2024-06-11T13:11:15.717 INFO:root:teuthology version: 0.0.1.dev303+g861a8dc 2024-06-11T13:11:15.717 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/teuthology-2024-06-10_21:08:02-orch-squid-distro-default-smithi/7749303 --verbose --owner scheduled_teuthology@teuthology --name teuthology-2024-06-10_21:08:02-orch-squid-distro-default-smithi --description orch/cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/root mon_election/classic msgr/async-v1only start tasks/rados_api_tests} -- /home/teuthworker/archive/teuthology-2024-06-10_21:08:02-orch-squid-distro-default-smithi/7749303/orig.config.yaml 2024-06-11T13:11:15.745 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-11T13:11:15.825 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-06-10_21:08:02-orch-squid-distro-default-smithi/7749303 branch: squid description: orch/cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/root mon_election/classic msgr/async-v1only start tasks/rados_api_tests} email: ceph-qa@ceph.com first_in_suite: false job_id: '7749303' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-06-10_21:08:02-orch-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: client: debug ms: 1 global: mon election default strategy: 1 ms bind msgr2: false ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon warn on pool no app: false osd: debug ms: 1 debug osd: 20 osd class default list: '*' osd class load list: '*' osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - reached quota - but it is still running - overall HEALTH_ - \(POOL_FULL\) - \(SMALLER_PGP_NUM\) - \(CACHE_POOL_NO_HIT_SET\) - \(CACHE_POOL_NEAR_FULL\) - \(POOL_APP_NOT_ENABLED\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) log-only-match: - CEPHADM_ mon_bind_msgr2: false sha1: 3f51ea3944d39ca13aa40153c9a4ee298306679d ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: cephadm_mode: root install: ceph: flavor: default sha1: 3f51ea3944d39ca13aa40153c9a4ee298306679d selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: squid sha1: 3f51ea3944d39ca13aa40153c9a4ee298306679d owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 6336 sha1: 3f51ea3944d39ca13aa40153c9a4ee298306679d sleep_before_teardown: 0 subset: 5/64 suite: orch suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_3f51ea3944d39ca13aa40153c9a4ee298306679d/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 3f51ea3944d39ca13aa40153c9a4ee298306679d targets: smithi057.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGren7CrU90JdcGxGy5LUKYkQwrwcxeZGAfsFqhuwuU6BCIUVPFW6TvqSzbAwP+SEVN5dYCdFGxgltBVfoIOx3E= smithi107.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAOmE9NUUp7M80RIPnW4AU2n7TdTTGuZZfRdFQjE+FOb6gduvn1aQSOFNHbu9ob/l1HV0R/OluZC7Jv2FOinCg8= tasks: - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - workunit: clients: client.0: - rados/test.sh - rados/test_pool_quota.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 861a8dcf7aa816a26e13f039336f7ed0a9aec0fa timestamp: 2024-06-10_21:08:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-06-11T13:11:15.825 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_3f51ea3944d39ca13aa40153c9a4ee298306679d/qa; will attempt to use it 2024-06-11T13:11:15.826 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_3f51ea3944d39ca13aa40153c9a4ee298306679d/qa/tasks 2024-06-11T13:11:15.826 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-06-11T13:11:15.827 INFO:teuthology.task.internal:Checking packages... 2024-06-11T13:11:15.848 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3f51ea3944d39ca13aa40153c9a4ee298306679d' 2024-06-11T13:11:15.848 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-06-11T13:11:15.848 INFO:teuthology.packaging:ref: None 2024-06-11T13:11:15.848 INFO:teuthology.packaging:tag: None 2024-06-11T13:11:15.848 INFO:teuthology.packaging:branch: squid 2024-06-11T13:11:15.848 INFO:teuthology.packaging:sha1: 3f51ea3944d39ca13aa40153c9a4ee298306679d 2024-06-11T13:11:15.849 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-06-11T13:11:16.038 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2976.g3f51ea39 2024-06-11T13:11:16.040 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-06-11T13:11:16.047 INFO:teuthology.task.internal:no buildpackages task found 2024-06-11T13:11:16.047 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-06-11T13:11:16.065 INFO:teuthology.task.internal:Saving configuration 2024-06-11T13:11:16.076 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-06-11T13:11:16.081 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-06-11T13:11:16.105 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi057.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-10_21:08:02-orch-squid-distro-default-smithi/7749303', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-11 13:06:15.693473', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGren7CrU90JdcGxGy5LUKYkQwrwcxeZGAfsFqhuwuU6BCIUVPFW6TvqSzbAwP+SEVN5dYCdFGxgltBVfoIOx3E='} 2024-06-11T13:11:16.125 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi107.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-10_21:08:02-orch-squid-distro-default-smithi/7749303', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-11 13:06:15.691795', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAOmE9NUUp7M80RIPnW4AU2n7TdTTGuZZfRdFQjE+FOb6gduvn1aQSOFNHbu9ob/l1HV0R/OluZC7Jv2FOinCg8='} 2024-06-11T13:11:16.125 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-06-11T13:11:16.131 INFO:teuthology.task.internal:roles: ubuntu@smithi057.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-06-11T13:11:16.131 INFO:teuthology.task.internal:roles: ubuntu@smithi107.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-06-11T13:11:16.131 INFO:teuthology.run_tasks:Running task console_log... 2024-06-11T13:11:16.229 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7feff75dfac0>, signals=[15]) 2024-06-11T13:11:16.229 INFO:teuthology.run_tasks:Running task internal.connect... 2024-06-11T13:11:16.235 INFO:teuthology.task.internal:Opening connections... 2024-06-11T13:11:16.235 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi057.front.sepia.ceph.com 2024-06-11T13:11:16.236 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-11T13:11:16.316 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi107.front.sepia.ceph.com 2024-06-11T13:11:16.316 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-11T13:11:16.391 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-06-11T13:11:16.397 DEBUG:teuthology.orchestra.run.smithi057:> uname -m 2024-06-11T13:11:16.416 INFO:teuthology.orchestra.run.smithi057.stdout:x86_64 2024-06-11T13:11:16.416 DEBUG:teuthology.orchestra.run.smithi057:> cat /etc/os-release 2024-06-11T13:11:16.474 INFO:teuthology.orchestra.run.smithi057.stdout:NAME="CentOS Stream" 2024-06-11T13:11:16.474 INFO:teuthology.orchestra.run.smithi057.stdout:VERSION="9" 2024-06-11T13:11:16.474 INFO:teuthology.orchestra.run.smithi057.stdout:ID="centos" 2024-06-11T13:11:16.474 INFO:teuthology.orchestra.run.smithi057.stdout:ID_LIKE="rhel fedora" 2024-06-11T13:11:16.474 INFO:teuthology.orchestra.run.smithi057.stdout:VERSION_ID="9" 2024-06-11T13:11:16.474 INFO:teuthology.orchestra.run.smithi057.stdout:PLATFORM_ID="platform:el9" 2024-06-11T13:11:16.474 INFO:teuthology.orchestra.run.smithi057.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-11T13:11:16.474 INFO:teuthology.orchestra.run.smithi057.stdout:ANSI_COLOR="0;31" 2024-06-11T13:11:16.475 INFO:teuthology.orchestra.run.smithi057.stdout:LOGO="fedora-logo-icon" 2024-06-11T13:11:16.475 INFO:teuthology.orchestra.run.smithi057.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-11T13:11:16.475 INFO:teuthology.orchestra.run.smithi057.stdout:HOME_URL="https://centos.org/" 2024-06-11T13:11:16.475 INFO:teuthology.orchestra.run.smithi057.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-11T13:11:16.475 INFO:teuthology.orchestra.run.smithi057.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-11T13:11:16.475 INFO:teuthology.orchestra.run.smithi057.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-11T13:11:16.475 INFO:teuthology.lock.ops:Updating smithi057.front.sepia.ceph.com on lock server 2024-06-11T13:11:16.495 DEBUG:teuthology.orchestra.run.smithi107:> uname -m 2024-06-11T13:11:16.512 INFO:teuthology.orchestra.run.smithi107.stdout:x86_64 2024-06-11T13:11:16.513 DEBUG:teuthology.orchestra.run.smithi107:> cat /etc/os-release 2024-06-11T13:11:16.569 INFO:teuthology.orchestra.run.smithi107.stdout:NAME="CentOS Stream" 2024-06-11T13:11:16.569 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION="9" 2024-06-11T13:11:16.569 INFO:teuthology.orchestra.run.smithi107.stdout:ID="centos" 2024-06-11T13:11:16.569 INFO:teuthology.orchestra.run.smithi107.stdout:ID_LIKE="rhel fedora" 2024-06-11T13:11:16.569 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION_ID="9" 2024-06-11T13:11:16.569 INFO:teuthology.orchestra.run.smithi107.stdout:PLATFORM_ID="platform:el9" 2024-06-11T13:11:16.569 INFO:teuthology.orchestra.run.smithi107.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-11T13:11:16.569 INFO:teuthology.orchestra.run.smithi107.stdout:ANSI_COLOR="0;31" 2024-06-11T13:11:16.569 INFO:teuthology.orchestra.run.smithi107.stdout:LOGO="fedora-logo-icon" 2024-06-11T13:11:16.569 INFO:teuthology.orchestra.run.smithi107.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-11T13:11:16.570 INFO:teuthology.orchestra.run.smithi107.stdout:HOME_URL="https://centos.org/" 2024-06-11T13:11:16.570 INFO:teuthology.orchestra.run.smithi107.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-11T13:11:16.570 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-11T13:11:16.570 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-11T13:11:16.570 INFO:teuthology.lock.ops:Updating smithi107.front.sepia.ceph.com on lock server 2024-06-11T13:11:16.586 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-06-11T13:11:16.595 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-06-11T13:11:16.601 INFO:teuthology.task.internal:Checking for old test directory... 2024-06-11T13:11:16.601 DEBUG:teuthology.orchestra.run.smithi057:> test '!' -e /home/ubuntu/cephtest 2024-06-11T13:11:16.603 DEBUG:teuthology.orchestra.run.smithi107:> test '!' -e /home/ubuntu/cephtest 2024-06-11T13:11:16.625 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-06-11T13:11:16.631 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-06-11T13:11:16.631 DEBUG:teuthology.orchestra.run.smithi057:> test -z $(ls -A /var/lib/ceph) 2024-06-11T13:11:16.661 DEBUG:teuthology.orchestra.run.smithi107:> test -z $(ls -A /var/lib/ceph) 2024-06-11T13:11:16.712 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-06-11T13:11:16.759 INFO:teuthology.run_tasks:Running task kernel... 2024-06-11T13:11:16.773 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-06-11T13:11:16.773 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-06-11T13:11:16.774 DEBUG:teuthology.orchestra.run.smithi057:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-11T13:11:16.774 DEBUG:teuthology.orchestra.run.smithi107:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-11T13:11:16.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-11T13:11:16.791 DEBUG:teuthology.orchestra.run.smithi107:> uname -r 2024-06-11T13:11:16.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-11T13:11:16.792 DEBUG:teuthology.orchestra.run.smithi057:> uname -r 2024-06-11T13:11:16.849 INFO:teuthology.orchestra.run.smithi107.stdout:5.14.0-437.el9.x86_64 2024-06-11T13:11:16.849 INFO:teuthology.task.kernel:Running kernel on smithi107: 5.14.0-437.el9.x86_64 2024-06-11T13:11:16.849 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum install -y kernel 2024-06-11T13:11:16.851 INFO:teuthology.orchestra.run.smithi057.stdout:5.14.0-437.el9.x86_64 2024-06-11T13:11:16.851 INFO:teuthology.task.kernel:Running kernel on smithi057: 5.14.0-437.el9.x86_64 2024-06-11T13:11:16.852 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum install -y kernel 2024-06-11T13:11:20.510 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - BaseOS 4.0 MB/s | 8.1 MB 00:02 2024-06-11T13:11:22.940 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS Stream 9 - BaseOS 1.8 MB/s | 8.1 MB 00:04 2024-06-11T13:11:24.799 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS Stream 9 - AppStream 28 MB/s | 19 MB 00:00 2024-06-11T13:11:29.166 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-06-11T13:11:30.749 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS Stream 9 - Extras packages 46 kB/s | 17 kB 00:00 2024-06-11T13:11:31.902 INFO:teuthology.orchestra.run.smithi057.stdout:Extra Packages for Enterprise Linux 21 MB/s | 21 MB 00:01 2024-06-11T13:11:36.788 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - AppStream 1.3 MB/s | 19 MB 00:15 2024-06-11T13:11:37.324 INFO:teuthology.orchestra.run.smithi057.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-06-11T13:11:39.364 INFO:teuthology.orchestra.run.smithi057.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-06-11T13:11:39.364 INFO:teuthology.orchestra.run.smithi057.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-06-11T13:11:39.470 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-06-11T13:11:39.477 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:11:39.478 INFO:teuthology.orchestra.run.smithi057.stdout: Package Architecture Version Repository Size 2024-06-11T13:11:39.478 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:11:39.478 INFO:teuthology.orchestra.run.smithi057.stdout:Installing: 2024-06-11T13:11:39.478 INFO:teuthology.orchestra.run.smithi057.stdout: kernel x86_64 5.14.0-452.el9 baseos 893 k 2024-06-11T13:11:39.478 INFO:teuthology.orchestra.run.smithi057.stdout:Installing dependencies: 2024-06-11T13:11:39.478 INFO:teuthology.orchestra.run.smithi057.stdout: kernel-core x86_64 5.14.0-452.el9 baseos 16 M 2024-06-11T13:11:39.478 INFO:teuthology.orchestra.run.smithi057.stdout: kernel-modules x86_64 5.14.0-452.el9 baseos 35 M 2024-06-11T13:11:39.478 INFO:teuthology.orchestra.run.smithi057.stdout: kernel-modules-core x86_64 5.14.0-452.el9 baseos 29 M 2024-06-11T13:11:39.478 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:11:39.478 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction Summary 2024-06-11T13:11:39.478 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:11:39.478 INFO:teuthology.orchestra.run.smithi057.stdout:Install 4 Packages 2024-06-11T13:11:39.478 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:11:39.479 INFO:teuthology.orchestra.run.smithi057.stdout:Total download size: 80 M 2024-06-11T13:11:39.479 INFO:teuthology.orchestra.run.smithi057.stdout:Installed size: 124 M 2024-06-11T13:11:39.479 INFO:teuthology.orchestra.run.smithi057.stdout:Downloading Packages: 2024-06-11T13:11:39.630 INFO:teuthology.orchestra.run.smithi057.stdout:[MIRROR] kernel-5.14.0-452.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-452.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-11T13:11:39.635 INFO:teuthology.orchestra.run.smithi057.stdout:[MIRROR] kernel-core-5.14.0-452.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-452.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-11T13:11:39.635 INFO:teuthology.orchestra.run.smithi057.stdout:[MIRROR] kernel-modules-5.14.0-452.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-452.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-11T13:11:39.809 INFO:teuthology.orchestra.run.smithi057.stdout:(1/4): kernel-5.14.0-452.el9.x86_64.rpm 3.3 MB/s | 893 kB 00:00 2024-06-11T13:11:40.101 INFO:teuthology.orchestra.run.smithi057.stdout:(2/4): kernel-core-5.14.0-452.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2024-06-11T13:11:40.501 INFO:teuthology.orchestra.run.smithi057.stdout:(3/4): kernel-modules-core-5.14.0-452.el9.x86_6 42 MB/s | 29 MB 00:00 2024-06-11T13:11:40.893 INFO:teuthology.orchestra.run.smithi057.stdout:(4/4): kernel-modules-5.14.0-452.el9.x86_64.rpm 26 MB/s | 35 MB 00:01 2024-06-11T13:11:40.895 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2024-06-11T13:11:40.896 INFO:teuthology.orchestra.run.smithi057.stdout:Total 57 MB/s | 80 MB 00:01 2024-06-11T13:11:41.113 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - CRB 9.6 MB/s | 6.2 MB 00:00 2024-06-11T13:11:41.626 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction check 2024-06-11T13:11:42.029 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction check succeeded. 2024-06-11T13:11:42.029 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction test 2024-06-11T13:11:42.830 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 17 kB 00:00 2024-06-11T13:11:43.001 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction test succeeded. 2024-06-11T13:11:43.002 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction 2024-06-11T13:11:43.658 INFO:teuthology.orchestra.run.smithi107.stdout:Extra Packages for Enterprise Linux 31 MB/s | 21 MB 00:00 2024-06-11T13:11:45.516 INFO:teuthology.orchestra.run.smithi057.stdout: Preparing : 1/1 2024-06-11T13:11:46.164 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : kernel-modules-core-5.14.0-452.el9.x86_64 1/4 2024-06-11T13:11:46.243 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-11T13:11:48.738 INFO:teuthology.orchestra.run.smithi107.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-06-11T13:11:49.039 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-11T13:11:49.185 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-11T13:11:50.736 INFO:teuthology.orchestra.run.smithi107.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-06-11T13:11:50.736 INFO:teuthology.orchestra.run.smithi107.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-06-11T13:11:50.828 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-06-11T13:11:50.836 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:11:50.836 INFO:teuthology.orchestra.run.smithi107.stdout: Package Architecture Version Repository Size 2024-06-11T13:11:50.836 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:11:50.836 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-06-11T13:11:50.836 INFO:teuthology.orchestra.run.smithi107.stdout: kernel x86_64 5.14.0-452.el9 baseos 893 k 2024-06-11T13:11:50.836 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-06-11T13:11:50.836 INFO:teuthology.orchestra.run.smithi107.stdout: kernel-core x86_64 5.14.0-452.el9 baseos 16 M 2024-06-11T13:11:50.836 INFO:teuthology.orchestra.run.smithi107.stdout: kernel-modules x86_64 5.14.0-452.el9 baseos 35 M 2024-06-11T13:11:50.837 INFO:teuthology.orchestra.run.smithi107.stdout: kernel-modules-core x86_64 5.14.0-452.el9 baseos 29 M 2024-06-11T13:11:50.837 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:11:50.837 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-06-11T13:11:50.837 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:11:50.837 INFO:teuthology.orchestra.run.smithi107.stdout:Install 4 Packages 2024-06-11T13:11:50.837 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:11:50.837 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 80 M 2024-06-11T13:11:50.838 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 124 M 2024-06-11T13:11:50.838 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-06-11T13:11:50.979 INFO:teuthology.orchestra.run.smithi107.stdout:[MIRROR] kernel-5.14.0-452.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-452.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-11T13:11:50.979 INFO:teuthology.orchestra.run.smithi107.stdout:[MIRROR] kernel-core-5.14.0-452.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-452.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-11T13:11:50.983 INFO:teuthology.orchestra.run.smithi107.stdout:[MIRROR] kernel-modules-5.14.0-452.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-452.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-11T13:11:51.765 INFO:teuthology.orchestra.run.smithi107.stdout:(1/4): kernel-5.14.0-452.el9.x86_64.rpm 1.0 MB/s | 893 kB 00:00 2024-06-11T13:11:52.506 INFO:teuthology.orchestra.run.smithi107.stdout:(2/4): kernel-core-5.14.0-452.el9.x86_64.rpm 9.9 MB/s | 16 MB 00:01 2024-06-11T13:11:53.166 INFO:teuthology.orchestra.run.smithi107.stdout:(3/4): kernel-modules-core-5.14.0-452.el9.x86_6 21 MB/s | 29 MB 00:01 2024-06-11T13:11:53.474 INFO:teuthology.orchestra.run.smithi107.stdout:(4/4): kernel-modules-5.14.0-452.el9.x86_64.rpm 13 MB/s | 35 MB 00:02 2024-06-11T13:11:53.479 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-06-11T13:11:53.479 INFO:teuthology.orchestra.run.smithi107.stdout:Total 30 MB/s | 80 MB 00:02 2024-06-11T13:11:54.183 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-06-11T13:11:54.286 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-11T13:11:54.349 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : kernel-5.14.0-452.el9.x86_64 4/4 2024-06-11T13:11:54.526 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-06-11T13:11:54.526 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-06-11T13:11:55.485 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-06-11T13:11:55.486 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-06-11T13:11:57.932 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-06-11T13:11:58.564 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : kernel-modules-core-5.14.0-452.el9.x86_64 1/4 2024-06-11T13:11:58.657 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-11T13:11:59.456 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: kernel-modules-core-5.14.0-452.el9.x86_64 4/4 2024-06-11T13:12:01.297 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-11T13:12:01.443 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-11T13:12:06.181 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-11T13:12:06.307 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : kernel-5.14.0-452.el9.x86_64 4/4 2024-06-11T13:12:11.058 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: kernel-modules-core-5.14.0-452.el9.x86_64 4/4 2024-06-11T13:12:30.846 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: kernel-core-5.14.0-452.el9.x86_64 4/4 2024-06-11T13:12:30.888 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: kernel-modules-5.14.0-452.el9.x86_64 4/4 2024-06-11T13:12:31.238 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: kernel-5.14.0-452.el9.x86_64 4/4 2024-06-11T13:12:31.238 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : kernel-5.14.0-452.el9.x86_64 1/4 2024-06-11T13:12:31.238 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-11T13:12:31.239 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-11T13:12:32.045 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : kernel-modules-core-5.14.0-452.el9.x86_64 4/4 2024-06-11T13:12:32.045 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:12:32.045 INFO:teuthology.orchestra.run.smithi057.stdout:Installed: 2024-06-11T13:12:32.045 INFO:teuthology.orchestra.run.smithi057.stdout: kernel-5.14.0-452.el9.x86_64 2024-06-11T13:12:32.045 INFO:teuthology.orchestra.run.smithi057.stdout: kernel-core-5.14.0-452.el9.x86_64 2024-06-11T13:12:32.045 INFO:teuthology.orchestra.run.smithi057.stdout: kernel-modules-5.14.0-452.el9.x86_64 2024-06-11T13:12:32.045 INFO:teuthology.orchestra.run.smithi057.stdout: kernel-modules-core-5.14.0-452.el9.x86_64 2024-06-11T13:12:32.045 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:12:32.045 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-06-11T13:12:32.384 DEBUG:teuthology.orchestra.run.smithi057:> rpm -q kernel | sort -rV | head -n 1 2024-06-11T13:12:32.460 INFO:teuthology.orchestra.run.smithi057.stdout:kernel-5.14.0-452.el9.x86_64 2024-06-11T13:12:32.460 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-452.el9.x86_64 2024-06-11T13:12:32.461 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-452.el9.x86_64 2024-06-11T13:12:32.461 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-06-11T13:12:32.461 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-06-11T13:12:32.461 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-06-11T13:12:32.461 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi057.front.sepia.ceph.com, path=None, version=distro) 2024-06-11T13:12:32.461 DEBUG:teuthology.orchestra.run.smithi057:> rpm -q kernel | sort -rV | head -n 1 2024-06-11T13:12:32.515 INFO:teuthology.orchestra.run.smithi057.stdout:kernel-5.14.0-452.el9.x86_64 2024-06-11T13:12:32.516 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-452.el9.x86_64 2024-06-11T13:12:32.516 DEBUG:teuthology.orchestra.run.smithi057:> sudo rpm -qi grub2-tools 2024-06-11T13:12:32.554 INFO:teuthology.orchestra.run.smithi057.stdout:Name : grub2-tools 2024-06-11T13:12:32.555 INFO:teuthology.orchestra.run.smithi057.stdout:Epoch : 1 2024-06-11T13:12:32.555 INFO:teuthology.orchestra.run.smithi057.stdout:Version : 2.06 2024-06-11T13:12:32.555 INFO:teuthology.orchestra.run.smithi057.stdout:Release : 68.el9 2024-06-11T13:12:32.555 INFO:teuthology.orchestra.run.smithi057.stdout:Architecture: x86_64 2024-06-11T13:12:32.555 INFO:teuthology.orchestra.run.smithi057.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-06-11T13:12:32.555 INFO:teuthology.orchestra.run.smithi057.stdout:Group : Unspecified 2024-06-11T13:12:32.555 INFO:teuthology.orchestra.run.smithi057.stdout:Size : 8264254 2024-06-11T13:12:32.555 INFO:teuthology.orchestra.run.smithi057.stdout:License : GPLv3+ 2024-06-11T13:12:32.555 INFO:teuthology.orchestra.run.smithi057.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-06-11T13:12:32.555 INFO:teuthology.orchestra.run.smithi057.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-06-11T13:12:32.556 INFO:teuthology.orchestra.run.smithi057.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-06-11T13:12:32.556 INFO:teuthology.orchestra.run.smithi057.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-06-11T13:12:32.556 INFO:teuthology.orchestra.run.smithi057.stdout:Packager : builder@centos.org 2024-06-11T13:12:32.556 INFO:teuthology.orchestra.run.smithi057.stdout:Vendor : CentOS 2024-06-11T13:12:32.556 INFO:teuthology.orchestra.run.smithi057.stdout:URL : http://www.gnu.org/software/grub/ 2024-06-11T13:12:32.556 INFO:teuthology.orchestra.run.smithi057.stdout:Summary : Support tools for GRUB. 2024-06-11T13:12:32.556 INFO:teuthology.orchestra.run.smithi057.stdout:Description : 2024-06-11T13:12:32.556 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:12:32.556 INFO:teuthology.orchestra.run.smithi057.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-06-11T13:12:32.556 INFO:teuthology.orchestra.run.smithi057.stdout:customizable bootloader with modular architecture. It supports a rich 2024-06-11T13:12:32.556 INFO:teuthology.orchestra.run.smithi057.stdout:variety of kernel formats, file systems, computer architectures and 2024-06-11T13:12:32.556 INFO:teuthology.orchestra.run.smithi057.stdout:hardware devices. 2024-06-11T13:12:32.556 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:12:32.556 INFO:teuthology.orchestra.run.smithi057.stdout:This subpackage provides tools for support of all platforms. 2024-06-11T13:12:32.557 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-06-11T13:12:32.558 INFO:teuthology.task.kernel:Updating grub on smithi057 to boot 5.14.0-452.el9.x86_64 2024-06-11T13:12:32.558 DEBUG:teuthology.orchestra.run.smithi057:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-06-11T13:12:33.236 INFO:teuthology.orchestra.run.smithi057.stderr:Generating grub configuration file ... 2024-06-11T13:12:34.687 INFO:teuthology.orchestra.run.smithi057.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-06-11T13:12:34.720 INFO:teuthology.orchestra.run.smithi057.stderr:done 2024-06-11T13:12:34.722 DEBUG:teuthology.orchestra.run.smithi057:> mktemp 2024-06-11T13:12:34.742 INFO:teuthology.orchestra.run.smithi057.stdout:/tmp/tmp.pUnNbtFAHD 2024-06-11T13:12:34.742 DEBUG:teuthology.orchestra.run.smithi057:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.pUnNbtFAHD 2024-06-11T13:12:34.808 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 0666 /tmp/tmp.pUnNbtFAHD 2024-06-11T13:12:34.950 DEBUG:teuthology.orchestra.remote:smithi057:/tmp/tmp.pUnNbtFAHD is 6KB 2024-06-11T13:12:34.998 DEBUG:teuthology.orchestra.run.smithi057:> rm -fr /tmp/tmp.pUnNbtFAHD 2024-06-11T13:12:35.013 DEBUG:teuthology.orchestra.run.smithi057:> sudo /bin/ls /boot/loader/entries || true 2024-06-11T13:12:35.074 INFO:teuthology.orchestra.run.smithi057.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-06-11T13:12:35.074 INFO:teuthology.orchestra.run.smithi057.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-06-11T13:12:35.074 INFO:teuthology.orchestra.run.smithi057.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-06-11T13:12:35.075 INFO:teuthology.orchestra.run.smithi057.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-452.el9.x86_64.conf 2024-06-11T13:12:35.076 DEBUG:teuthology.orchestra.run.smithi057:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-452.el9.x86_64 2024-06-11T13:12:35.232 DEBUG:teuthology.orchestra.run.smithi057:> sudo shutdown -r now 2024-06-11T13:12:40.539 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: kernel-core-5.14.0-452.el9.x86_64 4/4 2024-06-11T13:12:40.579 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: kernel-modules-5.14.0-452.el9.x86_64 4/4 2024-06-11T13:12:40.969 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: kernel-5.14.0-452.el9.x86_64 4/4 2024-06-11T13:12:40.969 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : kernel-5.14.0-452.el9.x86_64 1/4 2024-06-11T13:12:40.969 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-11T13:12:40.969 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-11T13:12:41.794 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : kernel-modules-core-5.14.0-452.el9.x86_64 4/4 2024-06-11T13:12:41.795 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:12:41.795 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-06-11T13:12:41.795 INFO:teuthology.orchestra.run.smithi107.stdout: kernel-5.14.0-452.el9.x86_64 2024-06-11T13:12:41.795 INFO:teuthology.orchestra.run.smithi107.stdout: kernel-core-5.14.0-452.el9.x86_64 2024-06-11T13:12:41.795 INFO:teuthology.orchestra.run.smithi107.stdout: kernel-modules-5.14.0-452.el9.x86_64 2024-06-11T13:12:41.795 INFO:teuthology.orchestra.run.smithi107.stdout: kernel-modules-core-5.14.0-452.el9.x86_64 2024-06-11T13:12:41.795 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:12:41.795 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-06-11T13:12:42.155 DEBUG:teuthology.orchestra.run.smithi107:> rpm -q kernel | sort -rV | head -n 1 2024-06-11T13:12:42.234 INFO:teuthology.orchestra.run.smithi107.stdout:kernel-5.14.0-452.el9.x86_64 2024-06-11T13:12:42.234 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-452.el9.x86_64 2024-06-11T13:12:42.234 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-452.el9.x86_64 2024-06-11T13:12:42.234 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-06-11T13:12:42.235 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-06-11T13:12:42.235 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-06-11T13:12:42.235 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi107.front.sepia.ceph.com, path=None, version=distro) 2024-06-11T13:12:42.235 DEBUG:teuthology.orchestra.run.smithi107:> rpm -q kernel | sort -rV | head -n 1 2024-06-11T13:12:42.288 INFO:teuthology.orchestra.run.smithi107.stdout:kernel-5.14.0-452.el9.x86_64 2024-06-11T13:12:42.288 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-452.el9.x86_64 2024-06-11T13:12:42.288 DEBUG:teuthology.orchestra.run.smithi107:> sudo rpm -qi grub2-tools 2024-06-11T13:12:42.319 INFO:teuthology.orchestra.run.smithi107.stdout:Name : grub2-tools 2024-06-11T13:12:42.319 INFO:teuthology.orchestra.run.smithi107.stdout:Epoch : 1 2024-06-11T13:12:42.319 INFO:teuthology.orchestra.run.smithi107.stdout:Version : 2.06 2024-06-11T13:12:42.319 INFO:teuthology.orchestra.run.smithi107.stdout:Release : 68.el9 2024-06-11T13:12:42.319 INFO:teuthology.orchestra.run.smithi107.stdout:Architecture: x86_64 2024-06-11T13:12:42.319 INFO:teuthology.orchestra.run.smithi107.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-06-11T13:12:42.319 INFO:teuthology.orchestra.run.smithi107.stdout:Group : Unspecified 2024-06-11T13:12:42.319 INFO:teuthology.orchestra.run.smithi107.stdout:Size : 8264254 2024-06-11T13:12:42.319 INFO:teuthology.orchestra.run.smithi107.stdout:License : GPLv3+ 2024-06-11T13:12:42.319 INFO:teuthology.orchestra.run.smithi107.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-06-11T13:12:42.319 INFO:teuthology.orchestra.run.smithi107.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-06-11T13:12:42.319 INFO:teuthology.orchestra.run.smithi107.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-06-11T13:12:42.320 INFO:teuthology.orchestra.run.smithi107.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-06-11T13:12:42.320 INFO:teuthology.orchestra.run.smithi107.stdout:Packager : builder@centos.org 2024-06-11T13:12:42.320 INFO:teuthology.orchestra.run.smithi107.stdout:Vendor : CentOS 2024-06-11T13:12:42.320 INFO:teuthology.orchestra.run.smithi107.stdout:URL : http://www.gnu.org/software/grub/ 2024-06-11T13:12:42.320 INFO:teuthology.orchestra.run.smithi107.stdout:Summary : Support tools for GRUB. 2024-06-11T13:12:42.320 INFO:teuthology.orchestra.run.smithi107.stdout:Description : 2024-06-11T13:12:42.320 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:12:42.320 INFO:teuthology.orchestra.run.smithi107.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-06-11T13:12:42.320 INFO:teuthology.orchestra.run.smithi107.stdout:customizable bootloader with modular architecture. It supports a rich 2024-06-11T13:12:42.320 INFO:teuthology.orchestra.run.smithi107.stdout:variety of kernel formats, file systems, computer architectures and 2024-06-11T13:12:42.320 INFO:teuthology.orchestra.run.smithi107.stdout:hardware devices. 2024-06-11T13:12:42.320 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:12:42.320 INFO:teuthology.orchestra.run.smithi107.stdout:This subpackage provides tools for support of all platforms. 2024-06-11T13:12:42.321 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-06-11T13:12:42.321 INFO:teuthology.task.kernel:Updating grub on smithi107 to boot 5.14.0-452.el9.x86_64 2024-06-11T13:12:42.321 DEBUG:teuthology.orchestra.run.smithi107:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-06-11T13:12:42.911 INFO:teuthology.orchestra.run.smithi107.stderr:Generating grub configuration file ... 2024-06-11T13:12:44.348 INFO:teuthology.orchestra.run.smithi107.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-06-11T13:12:44.378 INFO:teuthology.orchestra.run.smithi107.stderr:done 2024-06-11T13:12:44.380 DEBUG:teuthology.orchestra.run.smithi107:> mktemp 2024-06-11T13:12:44.395 INFO:teuthology.orchestra.run.smithi107.stdout:/tmp/tmp.mnDMrhklYK 2024-06-11T13:12:44.395 DEBUG:teuthology.orchestra.run.smithi107:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.mnDMrhklYK 2024-06-11T13:12:44.460 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0666 /tmp/tmp.mnDMrhklYK 2024-06-11T13:12:44.597 DEBUG:teuthology.orchestra.remote:smithi107:/tmp/tmp.mnDMrhklYK is 6KB 2024-06-11T13:12:44.644 DEBUG:teuthology.orchestra.run.smithi107:> rm -fr /tmp/tmp.mnDMrhklYK 2024-06-11T13:12:44.658 DEBUG:teuthology.orchestra.run.smithi107:> sudo /bin/ls /boot/loader/entries || true 2024-06-11T13:12:44.720 INFO:teuthology.orchestra.run.smithi107.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-06-11T13:12:44.720 INFO:teuthology.orchestra.run.smithi107.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-06-11T13:12:44.720 INFO:teuthology.orchestra.run.smithi107.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-06-11T13:12:44.720 INFO:teuthology.orchestra.run.smithi107.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-452.el9.x86_64.conf 2024-06-11T13:12:44.721 DEBUG:teuthology.orchestra.run.smithi107:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-452.el9.x86_64 2024-06-11T13:12:44.856 DEBUG:teuthology.orchestra.run.smithi107:> sudo shutdown -r now 2024-06-11T13:13:05.254 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-06-11T13:13:05.255 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com' 2024-06-11T13:13:05.256 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-11T13:13:14.866 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-06-11T13:13:14.866 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-06-11T13:13:14.867 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-11T13:13:23.742 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.57 2024-06-11T13:13:32.750 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com' 2024-06-11T13:13:32.751 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-11T13:13:33.218 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.107 2024-06-11T13:13:35.806 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.57 2024-06-11T13:13:42.226 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-06-11T13:13:42.227 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-11T13:13:45.278 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.107 2024-06-11T13:13:47.809 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com' 2024-06-11T13:13:47.810 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-11T13:13:57.286 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-06-11T13:13:57.287 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-11T13:14:15.710 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.107 2024-06-11T13:14:30.722 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-06-11T13:14:30.723 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-11T13:14:31.037 DEBUG:teuthology.orchestra.run.smithi107:> true 2024-06-11T13:14:31.405 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-06-11T13:14:31.405 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-452.el9.x86_64"... 2024-06-11T13:14:31.405 DEBUG:teuthology.orchestra.run.smithi107:> uname -r 2024-06-11T13:14:31.459 INFO:teuthology.orchestra.run.smithi107.stdout:5.14.0-452.el9.x86_64 2024-06-11T13:14:31.460 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-452.el9.x86_64 vs 5.14.0-452.el9.x86_64 2024-06-11T13:14:31.460 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-06-11T13:14:31.460 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-06-11T13:14:32.461 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-06-11T13:14:32.461 DEBUG:teuthology.orchestra.run.smithi107:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-06-11T13:14:32.530 INFO:teuthology.orchestra.run.smithi107.stdout:ttyS1 2024-06-11T13:14:32.588 DEBUG:teuthology.parallel:result is None 2024-06-11T13:14:47.812 DEBUG:teuthology.orchestra.remote:timed out 2024-06-11T13:15:02.813 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com' 2024-06-11T13:15:02.814 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-11T13:15:03.119 DEBUG:teuthology.orchestra.run.smithi057:> true 2024-06-11T13:15:03.486 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi057.front.sepia.ceph.com' 2024-06-11T13:15:03.486 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-452.el9.x86_64"... 2024-06-11T13:15:03.486 DEBUG:teuthology.orchestra.run.smithi057:> uname -r 2024-06-11T13:15:03.543 INFO:teuthology.orchestra.run.smithi057.stdout:5.14.0-452.el9.x86_64 2024-06-11T13:15:03.543 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-452.el9.x86_64 vs 5.14.0-452.el9.x86_64 2024-06-11T13:15:03.543 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-06-11T13:15:03.544 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-06-11T13:15:04.544 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-06-11T13:15:04.545 DEBUG:teuthology.orchestra.run.smithi057:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-06-11T13:15:04.620 INFO:teuthology.orchestra.run.smithi057.stdout:ttyS1 2024-06-11T13:15:04.656 DEBUG:teuthology.parallel:result is None 2024-06-11T13:15:04.656 INFO:teuthology.run_tasks:Running task internal.base... 2024-06-11T13:15:04.666 INFO:teuthology.task.internal:Creating test directory... 2024-06-11T13:15:04.666 DEBUG:teuthology.orchestra.run.smithi057:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-11T13:15:04.668 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-11T13:15:04.688 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-06-11T13:15:04.694 INFO:teuthology.run_tasks:Running task internal.archive... 2024-06-11T13:15:04.703 INFO:teuthology.task.internal:Creating archive directory... 2024-06-11T13:15:04.703 DEBUG:teuthology.orchestra.run.smithi057:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-11T13:15:04.728 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-11T13:15:04.797 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-06-11T13:15:04.804 INFO:teuthology.task.internal:Enabling coredump saving... 2024-06-11T13:15:04.804 DEBUG:teuthology.orchestra.run.smithi057:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-11T13:15:04.807 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-11T13:15:04.837 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-11T13:15:04.843 INFO:teuthology.orchestra.run.smithi057.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-11T13:15:04.848 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-11T13:15:04.855 INFO:teuthology.orchestra.run.smithi057.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-11T13:15:04.857 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-06-11T13:15:04.864 INFO:teuthology.task.internal:Configuring sudo... 2024-06-11T13:15:04.864 DEBUG:teuthology.orchestra.run.smithi057:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-11T13:15:04.900 DEBUG:teuthology.orchestra.run.smithi107:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-11T13:15:04.927 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-06-11T13:15:04.937 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-06-11T13:15:04.937 DEBUG:teuthology.orchestra.run.smithi057:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-11T13:15:04.970 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-11T13:15:04.987 DEBUG:teuthology.orchestra.run.smithi057:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-11T13:15:05.071 DEBUG:teuthology.orchestra.run.smithi057:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-11T13:15:05.118 DEBUG:teuthology.orchestra.run.smithi057:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-11T13:15:05.199 DEBUG:teuthology.orchestra.run.smithi057:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-11T13:15:05.245 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-06-11T13:15:05.245 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-11T13:15:05.312 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-11T13:15:05.355 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-11T13:15:05.408 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-11T13:15:05.493 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-11T13:15:05.537 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-06-11T13:15:05.537 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-11T13:15:05.605 DEBUG:teuthology.orchestra.run.smithi057:> sudo service rsyslog restart 2024-06-11T13:15:05.607 DEBUG:teuthology.orchestra.run.smithi107:> sudo service rsyslog restart 2024-06-11T13:15:05.649 INFO:teuthology.orchestra.run.smithi057.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-11T13:15:05.706 INFO:teuthology.orchestra.run.smithi107.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-11T13:15:06.085 INFO:teuthology.run_tasks:Running task internal.timer... 2024-06-11T13:15:06.093 INFO:teuthology.task.internal:Starting timer... 2024-06-11T13:15:06.093 INFO:teuthology.run_tasks:Running task pcp... 2024-06-11T13:15:06.110 INFO:teuthology.run_tasks:Running task selinux... 2024-06-11T13:15:06.118 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-06-11T13:15:06.119 DEBUG:teuthology.orchestra.run.smithi057:> sudo service auditd rotate 2024-06-11T13:15:06.173 INFO:teuthology.orchestra.run.smithi057.stdout:Rotating logs: 2024-06-11T13:15:06.175 DEBUG:teuthology.orchestra.run.smithi107:> sudo service auditd rotate 2024-06-11T13:15:06.246 INFO:teuthology.orchestra.run.smithi107.stdout:Rotating logs: 2024-06-11T13:15:06.248 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-06-11T13:15:06.248 DEBUG:teuthology.orchestra.run.smithi057:> /usr/sbin/getenforce 2024-06-11T13:15:06.286 INFO:teuthology.orchestra.run.smithi057.stdout:Permissive 2024-06-11T13:15:06.286 DEBUG:teuthology.orchestra.run.smithi107:> /usr/sbin/getenforce 2024-06-11T13:15:06.317 INFO:teuthology.orchestra.run.smithi107.stdout:Permissive 2024-06-11T13:15:06.317 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi057.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi107.front.sepia.ceph.com': 'permissive'} 2024-06-11T13:15:06.317 DEBUG:teuthology.orchestra.run.smithi057:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-06-11T13:15:06.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-11T13:15:06.355 DEBUG:teuthology.orchestra.run.smithi107:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-06-11T13:15:06.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-11T13:15:06.385 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-06-11T13:15:06.385 DEBUG:teuthology.orchestra.run.smithi057:> sudo /usr/sbin/setenforce permissive 2024-06-11T13:15:06.423 DEBUG:teuthology.orchestra.run.smithi107:> sudo /usr/sbin/setenforce permissive 2024-06-11T13:15:06.451 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-06-11T13:15:06.464 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-06-11T13:15:06.467 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-06-11T13:15:06.591 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-06-11T13:15:06.602 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-06-11T13:15:06.603 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi057.front.sepia.ceph.com,smithi107.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-06-11T13:19:31.861 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi057.front.sepia.ceph.com'), Remote(name='ubuntu@smithi107.front.sepia.ceph.com')] 2024-06-11T13:19:31.862 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com' 2024-06-11T13:19:31.863 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-11T13:19:31.947 DEBUG:teuthology.orchestra.run.smithi057:> true 2024-06-11T13:19:32.035 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi057.front.sepia.ceph.com' 2024-06-11T13:19:32.035 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-06-11T13:19:32.036 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-11T13:19:32.113 DEBUG:teuthology.orchestra.run.smithi107:> true 2024-06-11T13:19:32.203 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-06-11T13:19:32.203 INFO:teuthology.run_tasks:Running task clock... 2024-06-11T13:19:32.215 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-06-11T13:19:32.215 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-11T13:19:32.215 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-11T13:19:32.217 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-11T13:19:32.218 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-11T13:19:32.247 INFO:teuthology.orchestra.run.smithi057.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-11T13:19:32.263 INFO:teuthology.orchestra.run.smithi057.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-11T13:19:32.273 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-11T13:19:32.288 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-11T13:19:32.290 INFO:teuthology.orchestra.run.smithi057.stderr:sudo: ntpd: command not found 2024-06-11T13:19:32.302 INFO:teuthology.orchestra.run.smithi057.stdout:506 Cannot talk to daemon 2024-06-11T13:19:32.313 INFO:teuthology.orchestra.run.smithi107.stderr:sudo: ntpd: command not found 2024-06-11T13:19:32.317 INFO:teuthology.orchestra.run.smithi057.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-11T13:19:32.323 INFO:teuthology.orchestra.run.smithi107.stdout:506 Cannot talk to daemon 2024-06-11T13:19:32.331 INFO:teuthology.orchestra.run.smithi057.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-11T13:19:32.338 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-11T13:19:32.351 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-11T13:19:32.390 INFO:teuthology.orchestra.run.smithi057.stderr:bash: line 1: ntpq: command not found 2024-06-11T13:19:32.393 INFO:teuthology.orchestra.run.smithi057.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-11T13:19:32.393 INFO:teuthology.orchestra.run.smithi057.stdout:=============================================================================== 2024-06-11T13:19:32.393 INFO:teuthology.orchestra.run.smithi057.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-11T13:19:32.393 INFO:teuthology.orchestra.run.smithi057.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-11T13:19:32.393 INFO:teuthology.orchestra.run.smithi057.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-11T13:19:32.394 INFO:teuthology.orchestra.run.smithi057.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-11T13:19:32.405 INFO:teuthology.orchestra.run.smithi107.stderr:bash: line 1: ntpq: command not found 2024-06-11T13:19:32.408 INFO:teuthology.orchestra.run.smithi107.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-11T13:19:32.409 INFO:teuthology.orchestra.run.smithi107.stdout:=============================================================================== 2024-06-11T13:19:32.409 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-11T13:19:32.409 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-11T13:19:32.409 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-11T13:19:32.409 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-11T13:19:32.410 INFO:teuthology.run_tasks:Running task install... 2024-06-11T13:19:32.418 DEBUG:teuthology.task.install:project ceph 2024-06-11T13:19:32.418 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3f51ea3944d39ca13aa40153c9a4ee298306679d'}} 2024-06-11T13:19:32.418 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '3f51ea3944d39ca13aa40153c9a4ee298306679d'} 2024-06-11T13:19:32.418 INFO:teuthology.task.install:Using flavor: default 2024-06-11T13:19:32.424 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-06-11T13:19:32.424 INFO:teuthology.task.install:extra packages: [] 2024-06-11T13:19:32.424 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3f51ea3944d39ca13aa40153c9a4ee298306679d', 'tag': None, 'wait_for_package': False} 2024-06-11T13:19:32.425 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3f51ea3944d39ca13aa40153c9a4ee298306679d 2024-06-11T13:19:32.426 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3f51ea3944d39ca13aa40153c9a4ee298306679d', 'tag': None, 'wait_for_package': False} 2024-06-11T13:19:32.427 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3f51ea3944d39ca13aa40153c9a4ee298306679d 2024-06-11T13:19:32.587 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/3f51ea3944d39ca13aa40153c9a4ee298306679d/centos/9/flavors/default/ 2024-06-11T13:19:32.587 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2976.g3f51ea39 2024-06-11T13:19:32.592 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/3f51ea3944d39ca13aa40153c9a4ee298306679d/centos/9/flavors/default/ 2024-06-11T13:19:32.592 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2976.g3f51ea39 2024-06-11T13:19:32.706 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/3f51ea3944d39ca13aa40153c9a4ee298306679d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/3f51ea3944d39ca13aa40153c9a4ee298306679d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/3f51ea3944d39ca13aa40153c9a4ee298306679d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-11T13:19:32.706 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-06-11T13:19:32.706 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-11T13:19:32.732 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-06-11T13:19:32.733 DEBUG:teuthology.orchestra.run.smithi107:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3f51ea3944d39ca13aa40153c9a4ee298306679d/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-11T13:19:32.771 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/3f51ea3944d39ca13aa40153c9a4ee298306679d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/3f51ea3944d39ca13aa40153c9a4ee298306679d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/3f51ea3944d39ca13aa40153c9a4ee298306679d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-11T13:19:32.772 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-06-11T13:19:32.772 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-11T13:19:32.799 DEBUG:teuthology.orchestra.run.smithi107:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-11T13:19:32.800 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-06-11T13:19:32.800 DEBUG:teuthology.orchestra.run.smithi057:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3f51ea3944d39ca13aa40153c9a4ee298306679d/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-11T13:19:32.867 DEBUG:teuthology.orchestra.run.smithi057:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-11T13:19:32.873 DEBUG:teuthology.orchestra.run.smithi107:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-11T13:19:32.937 INFO:teuthology.orchestra.run.smithi107.stdout:check_obsoletes = 1 2024-06-11T13:19:32.939 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum clean all 2024-06-11T13:19:32.944 DEBUG:teuthology.orchestra.run.smithi057:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-11T13:19:33.009 INFO:teuthology.orchestra.run.smithi057.stdout:check_obsoletes = 1 2024-06-11T13:19:33.011 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum clean all 2024-06-11T13:19:33.190 INFO:teuthology.orchestra.run.smithi107.stdout:45 files removed 2024-06-11T13:19:33.218 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-radosgw 2024-06-11T13:19:33.253 INFO:teuthology.orchestra.run.smithi057.stdout:45 files removed 2024-06-11T13:19:33.280 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install ceph-radosgw 2024-06-11T13:19:33.957 INFO:teuthology.orchestra.run.smithi107.stdout:ceph packages for x86_64 179 kB/s | 83 kB 00:00 2024-06-11T13:19:34.048 INFO:teuthology.orchestra.run.smithi057.stdout:ceph packages for x86_64 175 kB/s | 83 kB 00:00 2024-06-11T13:19:34.283 INFO:teuthology.orchestra.run.smithi107.stdout:ceph noarch packages 42 kB/s | 11 kB 00:00 2024-06-11T13:19:34.599 INFO:teuthology.orchestra.run.smithi057.stdout:ceph noarch packages 23 kB/s | 11 kB 00:00 2024-06-11T13:19:34.658 INFO:teuthology.orchestra.run.smithi107.stdout:ceph source packages 5.7 kB/s | 1.9 kB 00:00 2024-06-11T13:19:35.025 INFO:teuthology.orchestra.run.smithi057.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-06-11T13:19:36.743 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS Stream 9 - BaseOS 4.8 MB/s | 8.1 MB 00:01 2024-06-11T13:19:36.752 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - BaseOS 3.9 MB/s | 8.1 MB 00:02 2024-06-11T13:19:40.295 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS Stream 9 - AppStream 8.0 MB/s | 19 MB 00:02 2024-06-11T13:19:44.553 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-06-11T13:19:46.336 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 17 kB 00:00 2024-06-11T13:19:47.340 INFO:teuthology.orchestra.run.smithi057.stdout:Extra Packages for Enterprise Linux 25 MB/s | 21 MB 00:00 2024-06-11T13:19:52.762 INFO:teuthology.orchestra.run.smithi057.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-06-11T13:19:54.579 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 19 MB 00:16 2024-06-11T13:19:55.232 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-06-11T13:19:55.243 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:19:55.244 INFO:teuthology.orchestra.run.smithi057.stdout: Package Arch Version Repository Size 2024-06-11T13:19:55.244 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:19:55.244 INFO:teuthology.orchestra.run.smithi057.stdout:Installing: 2024-06-11T13:19:55.244 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-radosgw x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 10 M 2024-06-11T13:19:55.244 INFO:teuthology.orchestra.run.smithi057.stdout:Upgrading: 2024-06-11T13:19:55.244 INFO:teuthology.orchestra.run.smithi057.stdout: librados2 x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 3.4 M 2024-06-11T13:19:55.244 INFO:teuthology.orchestra.run.smithi057.stdout: librbd1 x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 3.1 M 2024-06-11T13:19:55.244 INFO:teuthology.orchestra.run.smithi057.stdout:Installing dependencies: 2024-06-11T13:19:55.244 INFO:teuthology.orchestra.run.smithi057.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-11T13:19:55.244 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-base x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 5.4 M 2024-06-11T13:19:55.245 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-common x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 21 M 2024-06-11T13:19:55.245 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-selinux x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 25 k 2024-06-11T13:19:55.245 INFO:teuthology.orchestra.run.smithi057.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-11T13:19:55.245 INFO:teuthology.orchestra.run.smithi057.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-11T13:19:55.245 INFO:teuthology.orchestra.run.smithi057.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-11T13:19:55.245 INFO:teuthology.orchestra.run.smithi057.stdout: libcephfs2 x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 746 k 2024-06-11T13:19:55.245 INFO:teuthology.orchestra.run.smithi057.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-11T13:19:55.245 INFO:teuthology.orchestra.run.smithi057.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-11T13:19:55.245 INFO:teuthology.orchestra.run.smithi057.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-11T13:19:55.245 INFO:teuthology.orchestra.run.smithi057.stdout: libradosstriper1 x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 502 k 2024-06-11T13:19:55.245 INFO:teuthology.orchestra.run.smithi057.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-11T13:19:55.245 INFO:teuthology.orchestra.run.smithi057.stdout: librgw2 x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 5.3 M 2024-06-11T13:19:55.245 INFO:teuthology.orchestra.run.smithi057.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-11T13:19:55.246 INFO:teuthology.orchestra.run.smithi057.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-11T13:19:55.246 INFO:teuthology.orchestra.run.smithi057.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-11T13:19:55.246 INFO:teuthology.orchestra.run.smithi057.stdout: python3-ceph-argparse x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 45 k 2024-06-11T13:19:55.246 INFO:teuthology.orchestra.run.smithi057.stdout: python3-ceph-common x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 135 k 2024-06-11T13:19:55.246 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cephfs x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 162 k 2024-06-11T13:19:55.246 INFO:teuthology.orchestra.run.smithi057.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-11T13:19:55.246 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rados x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 321 k 2024-06-11T13:19:55.246 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rbd x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 300 k 2024-06-11T13:19:55.247 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rgw x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 99 k 2024-06-11T13:19:55.247 INFO:teuthology.orchestra.run.smithi057.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-11T13:19:55.247 INFO:teuthology.orchestra.run.smithi057.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-11T13:19:55.247 INFO:teuthology.orchestra.run.smithi057.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-11T13:19:55.247 INFO:teuthology.orchestra.run.smithi057.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-11T13:19:55.247 INFO:teuthology.orchestra.run.smithi057.stdout:Installing weak dependencies: 2024-06-11T13:19:55.247 INFO:teuthology.orchestra.run.smithi057.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-11T13:19:55.247 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:19:55.247 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction Summary 2024-06-11T13:19:55.247 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:19:55.247 INFO:teuthology.orchestra.run.smithi057.stdout:Install 30 Packages 2024-06-11T13:19:55.247 INFO:teuthology.orchestra.run.smithi057.stdout:Upgrade 2 Packages 2024-06-11T13:19:55.247 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:19:55.248 INFO:teuthology.orchestra.run.smithi057.stdout:Total download size: 60 M 2024-06-11T13:19:55.248 INFO:teuthology.orchestra.run.smithi057.stdout:Downloading Packages: 2024-06-11T13:19:56.263 INFO:teuthology.orchestra.run.smithi057.stdout:(1/32): ceph-base-19.0.0-2976.g3f51ea39.el9.x86 9.1 MB/s | 5.4 MB 00:00 2024-06-11T13:19:56.297 INFO:teuthology.orchestra.run.smithi057.stdout:(2/32): ceph-selinux-19.0.0-2976.g3f51ea39.el9. 758 kB/s | 25 kB 00:00 2024-06-11T13:19:56.372 INFO:teuthology.orchestra.run.smithi057.stdout:(3/32): libcephfs2-19.0.0-2976.g3f51ea39.el9.x8 9.7 MB/s | 746 kB 00:00 2024-06-11T13:19:56.431 INFO:teuthology.orchestra.run.smithi057.stdout:(4/32): libradosstriper1-19.0.0-2976.g3f51ea39. 8.4 MB/s | 502 kB 00:00 2024-06-11T13:19:56.889 INFO:teuthology.orchestra.run.smithi057.stdout:(5/32): librgw2-19.0.0-2976.g3f51ea39.el9.x86_6 12 MB/s | 5.3 MB 00:00 2024-06-11T13:19:56.931 INFO:teuthology.orchestra.run.smithi057.stdout:(6/32): python3-ceph-argparse-19.0.0-2976.g3f51 1.1 MB/s | 45 kB 00:00 2024-06-11T13:19:56.998 INFO:teuthology.orchestra.run.smithi057.stdout:(7/32): python3-ceph-common-19.0.0-2976.g3f51ea 2.0 MB/s | 135 kB 00:00 2024-06-11T13:19:57.065 INFO:teuthology.orchestra.run.smithi057.stdout:(8/32): python3-cephfs-19.0.0-2976.g3f51ea39.el 2.4 MB/s | 162 kB 00:00 2024-06-11T13:19:57.140 INFO:teuthology.orchestra.run.smithi057.stdout:(9/32): python3-rados-19.0.0-2976.g3f51ea39.el9 4.2 MB/s | 321 kB 00:00 2024-06-11T13:19:57.224 INFO:teuthology.orchestra.run.smithi057.stdout:(10/32): python3-rbd-19.0.0-2976.g3f51ea39.el9. 3.5 MB/s | 300 kB 00:00 2024-06-11T13:19:57.291 INFO:teuthology.orchestra.run.smithi057.stdout:(11/32): python3-rgw-19.0.0-2976.g3f51ea39.el9. 1.5 MB/s | 99 kB 00:00 2024-06-11T13:19:57.330 INFO:teuthology.orchestra.run.smithi057.stdout:[MIRROR] boost-program-options-1.75.0-8.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-program-options-1.75.0-8.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-11T13:19:57.474 INFO:teuthology.orchestra.run.smithi057.stdout:(12/32): boost-program-options-1.75.0-8.el9.x86 583 kB/s | 107 kB 00:00 2024-06-11T13:19:57.475 INFO:teuthology.orchestra.run.smithi057.stdout:[MIRROR] libpmemobj-1.12.1-1.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/libpmemobj-1.12.1-1.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-11T13:19:57.708 INFO:teuthology.orchestra.run.smithi057.stdout:(13/32): ceph-common-19.0.0-2976.g3f51ea39.el9. 10 MB/s | 21 MB 00:02 2024-06-11T13:19:57.808 INFO:teuthology.orchestra.run.smithi057.stdout:(14/32): ceph-radosgw-19.0.0-2976.g3f51ea39.el9 4.8 MB/s | 10 MB 00:02 2024-06-11T13:19:57.809 INFO:teuthology.orchestra.run.smithi057.stdout:[MIRROR] librabbitmq-0.11.0-7.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/librabbitmq-0.11.0-7.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-11T13:19:57.809 INFO:teuthology.orchestra.run.smithi057.stdout:[MIRROR] librdkafka-1.6.1-102.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/librdkafka-1.6.1-102.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-11T13:19:57.842 INFO:teuthology.orchestra.run.smithi057.stdout:(15/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 436 kB/s | 160 kB 00:00 2024-06-11T13:19:57.917 INFO:teuthology.orchestra.run.smithi057.stdout:(16/32): librdkafka-1.6.1-102.el9.x86_64.rpm 6.0 MB/s | 662 kB 00:00 2024-06-11T13:19:57.943 INFO:teuthology.orchestra.run.smithi057.stdout:(17/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 193 kB/s | 45 kB 00:00 2024-06-11T13:19:57.969 INFO:teuthology.orchestra.run.smithi057.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 842 kB/s | 42 kB 00:00 2024-06-11T13:19:58.018 INFO:teuthology.orchestra.run.smithi057.stdout:(19/32): qatlib-service-24.02.0-1.el9.x86_64.rp 720 kB/s | 36 kB 00:00 2024-06-11T13:19:58.043 INFO:teuthology.orchestra.run.smithi057.stdout:(20/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.4 MB/s | 292 kB 00:00 2024-06-11T13:19:58.069 INFO:teuthology.orchestra.run.smithi057.stdout:(21/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 940 kB/s | 47 kB 00:00 2024-06-11T13:19:58.094 INFO:teuthology.orchestra.run.smithi057.stdout:(22/32): qatlib-24.02.0-1.el9.x86_64.rpm 1.4 MB/s | 221 kB 00:00 2024-06-11T13:19:58.178 INFO:teuthology.orchestra.run.smithi057.stdout:(23/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 2.3 MB/s | 309 kB 00:00 2024-06-11T13:19:58.203 INFO:teuthology.orchestra.run.smithi057.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 2.1 MB/s | 49 kB 00:00 2024-06-11T13:19:58.228 INFO:teuthology.orchestra.run.smithi057.stdout:(25/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 246 kB/s | 26 kB 00:00 2024-06-11T13:19:58.254 INFO:teuthology.orchestra.run.smithi057.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-06-11T13:19:58.279 INFO:teuthology.orchestra.run.smithi057.stdout:(27/32): re2-20211101-3.el9.x86_64.rpm 7.5 MB/s | 192 kB 00:00 2024-06-11T13:19:58.329 INFO:teuthology.orchestra.run.smithi057.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.1 MB/s | 839 kB 00:00 2024-06-11T13:19:58.379 INFO:teuthology.orchestra.run.smithi057.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-06-11T13:19:58.463 INFO:teuthology.orchestra.run.smithi057.stdout:(30/32): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-06-11T13:19:58.705 INFO:teuthology.orchestra.run.smithi057.stdout:(31/32): librados2-19.0.0-2976.g3f51ea39.el9.x8 9.0 MB/s | 3.4 MB 00:00 2024-06-11T13:19:58.755 INFO:teuthology.orchestra.run.smithi057.stdout:(32/32): librbd1-19.0.0-2976.g3f51ea39.el9.x86_ 8.4 MB/s | 3.1 MB 00:00 2024-06-11T13:19:58.760 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2024-06-11T13:19:58.760 INFO:teuthology.orchestra.run.smithi057.stdout:Total 17 MB/s | 60 MB 00:03 2024-06-11T13:19:58.862 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction check 2024-06-11T13:19:58.878 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - CRB 8.8 MB/s | 6.2 MB 00:00 2024-06-11T13:19:58.903 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction check succeeded. 2024-06-11T13:19:58.903 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction test 2024-06-11T13:19:59.494 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction test succeeded. 2024-06-11T13:19:59.495 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction 2024-06-11T13:20:00.595 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 17 kB 00:00 2024-06-11T13:20:00.919 INFO:teuthology.orchestra.run.smithi057.stdout: Preparing : 1/1 2024-06-11T13:20:01.007 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-06-11T13:20:01.035 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-06-11T13:20:01.069 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-06-11T13:20:01.083 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-06-11T13:20:01.108 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-06-11T13:20:01.303 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-06-11T13:20:01.356 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-06-11T13:20:01.548 INFO:teuthology.orchestra.run.smithi107.stdout:Extra Packages for Enterprise Linux 26 MB/s | 21 MB 00:00 2024-06-11T13:20:01.697 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-06-11T13:20:01.820 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : librados2-2:19.0.0-2976.g3f51ea39.el9.x86_64 7/34 2024-06-11T13:20:01.913 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: librados2-2:19.0.0-2976.g3f51ea39.el9.x86_64 7/34 2024-06-11T13:20:02.002 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-rados-2:19.0.0-2976.g3f51ea39.el9.x86_64 8/34 2024-06-11T13:20:02.038 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libcephfs2-2:19.0.0-2976.g3f51ea39.el9.x86_64 9/34 2024-06-11T13:20:02.101 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: libcephfs2-2:19.0.0-2976.g3f51ea39.el9.x86_64 9/34 2024-06-11T13:20:02.130 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-06-11T13:20:02.175 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-ceph-argparse-2:19.0.0-2976.g3f51ea39.el9. 11/34 2024-06-11T13:20:02.245 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-cephfs-2:19.0.0-2976.g3f51ea39.el9.x86_64 12/34 2024-06-11T13:20:02.266 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libradosstriper1-2:19.0.0-2976.g3f51ea39.el9.x86_6 13/34 2024-06-11T13:20:02.330 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2976.g3f51ea39.el9.x86_6 13/34 2024-06-11T13:20:02.356 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-06-11T13:20:02.468 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-06-11T13:20:02.500 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-06-11T13:20:02.540 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-06-11T13:20:02.580 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-06-11T13:20:02.610 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-06-11T13:20:02.643 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-06-11T13:20:02.976 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-06-11T13:20:03.022 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : librbd1-2:19.0.0-2976.g3f51ea39.el9.x86_64 22/34 2024-06-11T13:20:03.121 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: librbd1-2:19.0.0-2976.g3f51ea39.el9.x86_64 22/34 2024-06-11T13:20:03.151 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-rbd-2:19.0.0-2976.g3f51ea39.el9.x86_64 23/34 2024-06-11T13:20:03.188 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-06-11T13:20:03.686 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-06-11T13:20:03.716 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : librgw2-2:19.0.0-2976.g3f51ea39.el9.x86_64 26/34 2024-06-11T13:20:03.779 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: librgw2-2:19.0.0-2976.g3f51ea39.el9.x86_64 26/34 2024-06-11T13:20:03.859 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-rgw-2:19.0.0-2976.g3f51ea39.el9.x86_64 27/34 2024-06-11T13:20:03.901 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-ceph-common-2:19.0.0-2976.g3f51ea39.el9.x8 28/34 2024-06-11T13:20:06.265 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-common-2:19.0.0-2976.g3f51ea39.el9.x86_64 29/34 2024-06-11T13:20:06.307 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-common-2:19.0.0-2976.g3f51ea39.el9.x86_64 29/34 2024-06-11T13:20:06.812 INFO:teuthology.orchestra.run.smithi107.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-06-11T13:20:06.904 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-common-2:19.0.0-2976.g3f51ea39.el9.x86_64 29/34 2024-06-11T13:20:06.943 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-base-2:19.0.0-2976.g3f51ea39.el9.x86_64 30/34 2024-06-11T13:20:07.037 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-base-2:19.0.0-2976.g3f51ea39.el9.x86_64 30/34 2024-06-11T13:20:07.037 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-11T13:20:07.037 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-11T13:20:07.037 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:20:07.070 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-selinux-2:19.0.0-2976.g3f51ea39.el9.x86_64 31/34 2024-06-11T13:20:09.236 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-06-11T13:20:09.240 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:20:09.240 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-06-11T13:20:09.240 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:20:09.240 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-06-11T13:20:09.240 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-radosgw x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 10 M 2024-06-11T13:20:09.240 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrading: 2024-06-11T13:20:09.240 INFO:teuthology.orchestra.run.smithi107.stdout: librados2 x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 3.4 M 2024-06-11T13:20:09.241 INFO:teuthology.orchestra.run.smithi107.stdout: librbd1 x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 3.1 M 2024-06-11T13:20:09.241 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-06-11T13:20:09.241 INFO:teuthology.orchestra.run.smithi107.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-11T13:20:09.241 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 5.4 M 2024-06-11T13:20:09.241 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-common x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 21 M 2024-06-11T13:20:09.241 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-selinux x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 25 k 2024-06-11T13:20:09.241 INFO:teuthology.orchestra.run.smithi107.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-11T13:20:09.241 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-11T13:20:09.241 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-11T13:20:09.241 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs2 x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 746 k 2024-06-11T13:20:09.241 INFO:teuthology.orchestra.run.smithi107.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-11T13:20:09.241 INFO:teuthology.orchestra.run.smithi107.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-11T13:20:09.241 INFO:teuthology.orchestra.run.smithi107.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-11T13:20:09.241 INFO:teuthology.orchestra.run.smithi107.stdout: libradosstriper1 x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 502 k 2024-06-11T13:20:09.242 INFO:teuthology.orchestra.run.smithi107.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-11T13:20:09.242 INFO:teuthology.orchestra.run.smithi107.stdout: librgw2 x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 5.3 M 2024-06-11T13:20:09.242 INFO:teuthology.orchestra.run.smithi107.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-11T13:20:09.242 INFO:teuthology.orchestra.run.smithi107.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-11T13:20:09.242 INFO:teuthology.orchestra.run.smithi107.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-11T13:20:09.242 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-argparse x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 45 k 2024-06-11T13:20:09.242 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-common x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 135 k 2024-06-11T13:20:09.242 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cephfs x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 162 k 2024-06-11T13:20:09.242 INFO:teuthology.orchestra.run.smithi107.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-11T13:20:09.242 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rados x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 321 k 2024-06-11T13:20:09.242 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rbd x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 300 k 2024-06-11T13:20:09.242 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rgw x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 99 k 2024-06-11T13:20:09.242 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-11T13:20:09.243 INFO:teuthology.orchestra.run.smithi107.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-11T13:20:09.243 INFO:teuthology.orchestra.run.smithi107.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-11T13:20:09.243 INFO:teuthology.orchestra.run.smithi107.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-11T13:20:09.243 INFO:teuthology.orchestra.run.smithi107.stdout:Installing weak dependencies: 2024-06-11T13:20:09.243 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-11T13:20:09.243 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:20:09.243 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-06-11T13:20:09.243 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:20:09.243 INFO:teuthology.orchestra.run.smithi107.stdout:Install 30 Packages 2024-06-11T13:20:09.243 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrade 2 Packages 2024-06-11T13:20:09.243 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:20:09.244 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 60 M 2024-06-11T13:20:09.244 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-06-11T13:20:09.888 INFO:teuthology.orchestra.run.smithi107.stdout:(1/32): ceph-base-19.0.0-2976.g3f51ea39.el9.x86 10 MB/s | 5.4 MB 00:00 2024-06-11T13:20:09.930 INFO:teuthology.orchestra.run.smithi107.stdout:(2/32): ceph-selinux-19.0.0-2976.g3f51ea39.el9. 603 kB/s | 25 kB 00:00 2024-06-11T13:20:10.014 INFO:teuthology.orchestra.run.smithi107.stdout:(3/32): libcephfs2-19.0.0-2976.g3f51ea39.el9.x8 8.8 MB/s | 746 kB 00:00 2024-06-11T13:20:10.064 INFO:teuthology.orchestra.run.smithi107.stdout:(4/32): libradosstriper1-19.0.0-2976.g3f51ea39. 9.7 MB/s | 502 kB 00:00 2024-06-11T13:20:10.365 INFO:teuthology.orchestra.run.smithi107.stdout:(5/32): librgw2-19.0.0-2976.g3f51ea39.el9.x86_6 18 MB/s | 5.3 MB 00:00 2024-06-11T13:20:10.406 INFO:teuthology.orchestra.run.smithi107.stdout:(6/32): python3-ceph-argparse-19.0.0-2976.g3f51 1.1 MB/s | 45 kB 00:00 2024-06-11T13:20:10.457 INFO:teuthology.orchestra.run.smithi107.stdout:(7/32): python3-ceph-common-19.0.0-2976.g3f51ea 2.6 MB/s | 135 kB 00:00 2024-06-11T13:20:10.516 INFO:teuthology.orchestra.run.smithi107.stdout:(8/32): python3-cephfs-19.0.0-2976.g3f51ea39.el 2.7 MB/s | 162 kB 00:00 2024-06-11T13:20:10.583 INFO:teuthology.orchestra.run.smithi107.stdout:(9/32): python3-rados-19.0.0-2976.g3f51ea39.el9 4.7 MB/s | 321 kB 00:00 2024-06-11T13:20:10.633 INFO:teuthology.orchestra.run.smithi107.stdout:(10/32): python3-rbd-19.0.0-2976.g3f51ea39.el9. 5.9 MB/s | 300 kB 00:00 2024-06-11T13:20:10.691 INFO:teuthology.orchestra.run.smithi107.stdout:(11/32): python3-rgw-19.0.0-2976.g3f51ea39.el9. 1.7 MB/s | 99 kB 00:00 2024-06-11T13:20:10.727 INFO:teuthology.orchestra.run.smithi107.stdout:[MIRROR] boost-program-options-1.75.0-8.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-program-options-1.75.0-8.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-11T13:20:11.150 INFO:teuthology.orchestra.run.smithi107.stdout:(12/32): boost-program-options-1.75.0-8.el9.x86 233 kB/s | 107 kB 00:00 2024-06-11T13:20:11.152 INFO:teuthology.orchestra.run.smithi107.stdout:[MIRROR] libpmemobj-1.12.1-1.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/libpmemobj-1.12.1-1.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-11T13:20:11.334 INFO:teuthology.orchestra.run.smithi107.stdout:(13/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 875 kB/s | 160 kB 00:00 2024-06-11T13:20:11.334 INFO:teuthology.orchestra.run.smithi107.stdout:[MIRROR] librabbitmq-0.11.0-7.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/librabbitmq-0.11.0-7.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-11T13:20:11.442 INFO:teuthology.orchestra.run.smithi107.stdout:(14/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 417 kB/s | 45 kB 00:00 2024-06-11T13:20:11.693 INFO:teuthology.orchestra.run.smithi107.stdout:(15/32): ceph-common-19.0.0-2976.g3f51ea39.el9. 9.0 MB/s | 21 MB 00:02 2024-06-11T13:20:11.834 INFO:teuthology.orchestra.run.smithi107.stdout:(16/32): librdkafka-1.6.1-102.el9.x86_64.rpm 1.7 MB/s | 662 kB 00:00 2024-06-11T13:20:11.943 INFO:teuthology.orchestra.run.smithi107.stdout:(17/32): ceph-radosgw-19.0.0-2976.g3f51ea39.el9 4.0 MB/s | 10 MB 00:02 2024-06-11T13:20:11.968 INFO:teuthology.orchestra.run.smithi107.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 316 kB/s | 42 kB 00:00 2024-06-11T13:20:12.077 INFO:teuthology.orchestra.run.smithi107.stdout:(19/32): qatlib-service-24.02.0-1.el9.x86_64.rp 331 kB/s | 36 kB 00:00 2024-06-11T13:20:12.177 INFO:teuthology.orchestra.run.smithi107.stdout:(20/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 473 kB/s | 47 kB 00:00 2024-06-11T13:20:12.211 INFO:teuthology.orchestra.run.smithi107.stdout:(21/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 9.1 MB/s | 309 kB 00:00 2024-06-11T13:20:12.319 INFO:teuthology.orchestra.run.smithi107.stdout:(22/32): libarrow-9.0.0-11.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-06-11T13:20:12.345 INFO:teuthology.orchestra.run.smithi107.stdout:(23/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-06-11T13:20:12.370 INFO:teuthology.orchestra.run.smithi107.stdout:(24/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 432 kB/s | 292 kB 00:00 2024-06-11T13:20:12.387 INFO:teuthology.orchestra.run.smithi107.stdout:(25/32): liboath-2.6.7-2.el9.x86_64.rpm 1.1 MB/s | 49 kB 00:00 2024-06-11T13:20:12.446 INFO:teuthology.orchestra.run.smithi107.stdout:(26/32): qatlib-24.02.0-1.el9.x86_64.rpm 439 kB/s | 221 kB 00:00 2024-06-11T13:20:12.471 INFO:teuthology.orchestra.run.smithi107.stdout:(27/32): libunwind-1.6.2-1.el9.x86_64.rpm 670 kB/s | 67 kB 00:00 2024-06-11T13:20:12.530 INFO:teuthology.orchestra.run.smithi107.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 839 kB 00:00 2024-06-11T13:20:12.572 INFO:teuthology.orchestra.run.smithi107.stdout:(29/32): re2-20211101-3.el9.x86_64.rpm 1.5 MB/s | 192 kB 00:00 2024-06-11T13:20:12.613 INFO:teuthology.orchestra.run.smithi107.stdout:(30/32): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-06-11T13:20:12.914 INFO:teuthology.orchestra.run.smithi107.stdout:(31/32): librados2-19.0.0-2976.g3f51ea39.el9.x8 8.9 MB/s | 3.4 MB 00:00 2024-06-11T13:20:12.964 INFO:teuthology.orchestra.run.smithi107.stdout:(32/32): librbd1-19.0.0-2976.g3f51ea39.el9.x86_ 8.0 MB/s | 3.1 MB 00:00 2024-06-11T13:20:12.969 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-06-11T13:20:12.969 INFO:teuthology.orchestra.run.smithi107.stdout:Total 16 MB/s | 60 MB 00:03 2024-06-11T13:20:13.063 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-06-11T13:20:13.099 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-06-11T13:20:13.099 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-06-11T13:20:13.657 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-06-11T13:20:13.658 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-06-11T13:20:15.060 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-06-11T13:20:15.152 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-06-11T13:20:15.185 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-06-11T13:20:15.219 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-06-11T13:20:15.242 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-06-11T13:20:15.267 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-06-11T13:20:15.476 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-06-11T13:20:15.530 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-06-11T13:20:15.900 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-06-11T13:20:15.962 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : librados2-2:19.0.0-2976.g3f51ea39.el9.x86_64 7/34 2024-06-11T13:20:16.062 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados2-2:19.0.0-2976.g3f51ea39.el9.x86_64 7/34 2024-06-11T13:20:16.149 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rados-2:19.0.0-2976.g3f51ea39.el9.x86_64 8/34 2024-06-11T13:20:16.197 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephfs2-2:19.0.0-2976.g3f51ea39.el9.x86_64 9/34 2024-06-11T13:20:16.261 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephfs2-2:19.0.0-2976.g3f51ea39.el9.x86_64 9/34 2024-06-11T13:20:16.297 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-06-11T13:20:16.341 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ceph-argparse-2:19.0.0-2976.g3f51ea39.el9. 11/34 2024-06-11T13:20:16.451 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cephfs-2:19.0.0-2976.g3f51ea39.el9.x86_64 12/34 2024-06-11T13:20:16.474 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libradosstriper1-2:19.0.0-2976.g3f51ea39.el9.x86_6 13/34 2024-06-11T13:20:16.539 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2976.g3f51ea39.el9.x86_6 13/34 2024-06-11T13:20:16.573 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-06-11T13:20:16.679 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-06-11T13:20:16.708 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-06-11T13:20:16.749 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-06-11T13:20:16.781 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-06-11T13:20:16.810 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-06-11T13:20:16.842 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-06-11T13:20:17.162 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-06-11T13:20:17.206 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : librbd1-2:19.0.0-2976.g3f51ea39.el9.x86_64 22/34 2024-06-11T13:20:17.207 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2976.g3f51ea39.el9.x86_64 31/34 2024-06-11T13:20:17.207 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /sys 2024-06-11T13:20:17.207 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /proc 2024-06-11T13:20:17.207 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /mnt 2024-06-11T13:20:17.207 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /var/tmp 2024-06-11T13:20:17.207 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /home 2024-06-11T13:20:17.208 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /root 2024-06-11T13:20:17.208 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /tmp 2024-06-11T13:20:17.208 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:20:17.301 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librbd1-2:19.0.0-2976.g3f51ea39.el9.x86_64 22/34 2024-06-11T13:20:17.335 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rbd-2:19.0.0-2976.g3f51ea39.el9.x86_64 23/34 2024-06-11T13:20:17.371 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-06-11T13:20:17.848 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-06-11T13:20:17.875 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librgw2-2:19.0.0-2976.g3f51ea39.el9.x86_64 26/34 2024-06-11T13:20:17.946 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librgw2-2:19.0.0-2976.g3f51ea39.el9.x86_64 26/34 2024-06-11T13:20:18.014 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rgw-2:19.0.0-2976.g3f51ea39.el9.x86_64 27/34 2024-06-11T13:20:18.044 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ceph-common-2:19.0.0-2976.g3f51ea39.el9.x8 28/34 2024-06-11T13:20:18.178 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-radosgw-2:19.0.0-2976.g3f51ea39.el9.x86_64 32/34 2024-06-11T13:20:18.205 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2976.g3f51ea39.el9.x86_64 32/34 2024-06-11T13:20:18.205 INFO:teuthology.orchestra.run.smithi057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-11T13:20:18.205 INFO:teuthology.orchestra.run.smithi057.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-11T13:20:18.205 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-11T13:20:18.205 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-11T13:20:18.206 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:20:18.207 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-06-11T13:20:18.297 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-06-11T13:20:18.298 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-06-11T13:20:19.570 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-06-11T13:20:19.570 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-base-2:19.0.0-2976.g3f51ea39.el9.x86_64 1/34 2024-06-11T13:20:19.571 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-common-2:19.0.0-2976.g3f51ea39.el9.x86_64 2/34 2024-06-11T13:20:19.571 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-radosgw-2:19.0.0-2976.g3f51ea39.el9.x86_64 3/34 2024-06-11T13:20:19.571 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-selinux-2:19.0.0-2976.g3f51ea39.el9.x86_64 4/34 2024-06-11T13:20:19.571 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libcephfs2-2:19.0.0-2976.g3f51ea39.el9.x86_64 5/34 2024-06-11T13:20:19.571 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libradosstriper1-2:19.0.0-2976.g3f51ea39.el9.x86_6 6/34 2024-06-11T13:20:19.571 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librgw2-2:19.0.0-2976.g3f51ea39.el9.x86_64 7/34 2024-06-11T13:20:19.571 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2976.g3f51ea39.el9. 8/34 2024-06-11T13:20:19.571 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-ceph-common-2:19.0.0-2976.g3f51ea39.el9.x8 9/34 2024-06-11T13:20:19.571 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-cephfs-2:19.0.0-2976.g3f51ea39.el9.x86_64 10/34 2024-06-11T13:20:19.571 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-rados-2:19.0.0-2976.g3f51ea39.el9.x86_64 11/34 2024-06-11T13:20:19.571 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-rbd-2:19.0.0-2976.g3f51ea39.el9.x86_64 12/34 2024-06-11T13:20:19.571 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-rgw-2:19.0.0-2976.g3f51ea39.el9.x86_64 13/34 2024-06-11T13:20:19.571 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-06-11T13:20:19.572 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-06-11T13:20:19.572 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-06-11T13:20:19.572 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-06-11T13:20:19.572 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-06-11T13:20:19.572 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-06-11T13:20:19.572 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-06-11T13:20:19.572 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-06-11T13:20:19.572 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-06-11T13:20:19.572 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-06-11T13:20:19.572 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-06-11T13:20:19.572 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-06-11T13:20:19.572 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-06-11T13:20:19.572 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-06-11T13:20:19.573 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-06-11T13:20:19.573 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-06-11T13:20:19.573 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-06-11T13:20:19.573 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librados2-2:19.0.0-2976.g3f51ea39.el9.x86_64 31/34 2024-06-11T13:20:19.573 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-06-11T13:20:19.574 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librbd1-2:19.0.0-2976.g3f51ea39.el9.x86_64 33/34 2024-06-11T13:20:20.267 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-common-2:19.0.0-2976.g3f51ea39.el9.x86_64 29/34 2024-06-11T13:20:20.307 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-common-2:19.0.0-2976.g3f51ea39.el9.x86_64 29/34 2024-06-11T13:20:20.432 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-06-11T13:20:20.432 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:20:20.432 INFO:teuthology.orchestra.run.smithi057.stdout:Upgraded: 2024-06-11T13:20:20.432 INFO:teuthology.orchestra.run.smithi057.stdout: librados2-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:20:20.432 INFO:teuthology.orchestra.run.smithi057.stdout: librbd1-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:20:20.432 INFO:teuthology.orchestra.run.smithi057.stdout:Installed: 2024-06-11T13:20:20.432 INFO:teuthology.orchestra.run.smithi057.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-11T13:20:20.432 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-base-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:20:20.433 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-common-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:20:20.433 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-radosgw-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:20:20.433 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-selinux-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:20:20.433 INFO:teuthology.orchestra.run.smithi057.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-11T13:20:20.433 INFO:teuthology.orchestra.run.smithi057.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-11T13:20:20.433 INFO:teuthology.orchestra.run.smithi057.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-11T13:20:20.433 INFO:teuthology.orchestra.run.smithi057.stdout: libcephfs2-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:20:20.433 INFO:teuthology.orchestra.run.smithi057.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-11T13:20:20.433 INFO:teuthology.orchestra.run.smithi057.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-11T13:20:20.433 INFO:teuthology.orchestra.run.smithi057.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-11T13:20:20.433 INFO:teuthology.orchestra.run.smithi057.stdout: libradosstriper1-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:20:20.434 INFO:teuthology.orchestra.run.smithi057.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-11T13:20:20.434 INFO:teuthology.orchestra.run.smithi057.stdout: librgw2-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:20:20.434 INFO:teuthology.orchestra.run.smithi057.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-11T13:20:20.434 INFO:teuthology.orchestra.run.smithi057.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-11T13:20:20.434 INFO:teuthology.orchestra.run.smithi057.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-11T13:20:20.434 INFO:teuthology.orchestra.run.smithi057.stdout: python3-ceph-argparse-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:20:20.435 INFO:teuthology.orchestra.run.smithi057.stdout: python3-ceph-common-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:20:20.435 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cephfs-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:20:20.435 INFO:teuthology.orchestra.run.smithi057.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-11T13:20:20.435 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rados-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:20:20.435 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rbd-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:20:20.435 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rgw-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:20:20.435 INFO:teuthology.orchestra.run.smithi057.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-11T13:20:20.435 INFO:teuthology.orchestra.run.smithi057.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-11T13:20:20.435 INFO:teuthology.orchestra.run.smithi057.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-11T13:20:20.436 INFO:teuthology.orchestra.run.smithi057.stdout: re2-1:20211101-3.el9.x86_64 2024-06-11T13:20:20.436 INFO:teuthology.orchestra.run.smithi057.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-11T13:20:20.436 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:20:20.436 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-06-11T13:20:20.786 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install ceph-test 2024-06-11T13:20:20.894 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-common-2:19.0.0-2976.g3f51ea39.el9.x86_64 29/34 2024-06-11T13:20:20.928 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-base-2:19.0.0-2976.g3f51ea39.el9.x86_64 30/34 2024-06-11T13:20:21.012 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-base-2:19.0.0-2976.g3f51ea39.el9.x86_64 30/34 2024-06-11T13:20:21.013 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-11T13:20:21.013 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-11T13:20:21.013 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:20:21.045 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-selinux-2:19.0.0-2976.g3f51ea39.el9.x86_64 31/34 2024-06-11T13:20:21.383 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:00:29 ago on Tue 11 Jun 2024 01:19:52 PM UTC. 2024-06-11T13:20:21.507 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-06-11T13:20:21.508 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:20:21.508 INFO:teuthology.orchestra.run.smithi057.stdout: Package Arch Version Repository Size 2024-06-11T13:20:21.508 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:20:21.508 INFO:teuthology.orchestra.run.smithi057.stdout:Installing: 2024-06-11T13:20:21.508 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-test x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 48 M 2024-06-11T13:20:21.509 INFO:teuthology.orchestra.run.smithi057.stdout:Installing dependencies: 2024-06-11T13:20:21.509 INFO:teuthology.orchestra.run.smithi057.stdout: libcephsqlite x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 163 k 2024-06-11T13:20:21.509 INFO:teuthology.orchestra.run.smithi057.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-06-11T13:20:21.509 INFO:teuthology.orchestra.run.smithi057.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-11T13:20:21.509 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:20:21.509 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction Summary 2024-06-11T13:20:21.509 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:20:21.509 INFO:teuthology.orchestra.run.smithi057.stdout:Install 4 Packages 2024-06-11T13:20:21.509 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:20:21.510 INFO:teuthology.orchestra.run.smithi057.stdout:Total download size: 49 M 2024-06-11T13:20:21.510 INFO:teuthology.orchestra.run.smithi057.stdout:Installed size: 207 M 2024-06-11T13:20:21.510 INFO:teuthology.orchestra.run.smithi057.stdout:Downloading Packages: 2024-06-11T13:20:21.869 INFO:teuthology.orchestra.run.smithi057.stdout:[MIRROR] socat-1.7.4.1-5.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/socat-1.7.4.1-5.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-11T13:20:22.012 INFO:teuthology.orchestra.run.smithi057.stdout:(1/4): socat-1.7.4.1-5.el9.x86_64.rpm 1.3 MB/s | 305 kB 00:00 2024-06-11T13:20:22.013 INFO:teuthology.orchestra.run.smithi057.stdout:[MIRROR] xmlstarlet-1.6.1-20.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/xmlstarlet-1.6.1-20.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-11T13:20:22.054 INFO:teuthology.orchestra.run.smithi057.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-06-11T13:20:22.121 INFO:teuthology.orchestra.run.smithi057.stdout:(3/4): libcephsqlite-19.0.0-2976.g3f51ea39.el9. 484 kB/s | 163 kB 00:00 2024-06-11T13:20:27.321 INFO:teuthology.orchestra.run.smithi057.stdout:(4/4): ceph-test-19.0.0-2976.g3f51ea39.el9.x86_ 8.7 MB/s | 48 MB 00:05 2024-06-11T13:20:27.324 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2024-06-11T13:20:27.324 INFO:teuthology.orchestra.run.smithi057.stdout:Total 8.4 MB/s | 49 MB 00:05 2024-06-11T13:20:27.353 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction check 2024-06-11T13:20:27.375 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction check succeeded. 2024-06-11T13:20:27.375 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction test 2024-06-11T13:20:27.763 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction test succeeded. 2024-06-11T13:20:27.764 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction 2024-06-11T13:20:28.306 INFO:teuthology.orchestra.run.smithi057.stdout: Preparing : 1/1 2024-06-11T13:20:28.395 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-06-11T13:20:28.437 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-06-11T13:20:28.464 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libcephsqlite-2:19.0.0-2976.g3f51ea39.el9.x86_64 3/4 2024-06-11T13:20:30.649 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2976.g3f51ea39.el9.x86_64 31/34 2024-06-11T13:20:30.649 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /sys 2024-06-11T13:20:30.649 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /proc 2024-06-11T13:20:30.649 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /mnt 2024-06-11T13:20:30.649 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /var/tmp 2024-06-11T13:20:30.650 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /home 2024-06-11T13:20:30.650 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /root 2024-06-11T13:20:30.650 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /tmp 2024-06-11T13:20:30.650 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:20:31.615 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-radosgw-2:19.0.0-2976.g3f51ea39.el9.x86_64 32/34 2024-06-11T13:20:31.639 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2976.g3f51ea39.el9.x86_64 32/34 2024-06-11T13:20:31.640 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-11T13:20:31.640 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-11T13:20:31.640 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-11T13:20:31.640 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-11T13:20:31.640 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:20:31.642 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-06-11T13:20:31.725 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-06-11T13:20:31.725 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-06-11T13:20:32.966 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-06-11T13:20:32.966 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-base-2:19.0.0-2976.g3f51ea39.el9.x86_64 1/34 2024-06-11T13:20:32.966 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-common-2:19.0.0-2976.g3f51ea39.el9.x86_64 2/34 2024-06-11T13:20:32.966 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-radosgw-2:19.0.0-2976.g3f51ea39.el9.x86_64 3/34 2024-06-11T13:20:32.967 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-selinux-2:19.0.0-2976.g3f51ea39.el9.x86_64 4/34 2024-06-11T13:20:32.967 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephfs2-2:19.0.0-2976.g3f51ea39.el9.x86_64 5/34 2024-06-11T13:20:32.967 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libradosstriper1-2:19.0.0-2976.g3f51ea39.el9.x86_6 6/34 2024-06-11T13:20:32.967 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librgw2-2:19.0.0-2976.g3f51ea39.el9.x86_64 7/34 2024-06-11T13:20:32.967 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2976.g3f51ea39.el9. 8/34 2024-06-11T13:20:32.967 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ceph-common-2:19.0.0-2976.g3f51ea39.el9.x8 9/34 2024-06-11T13:20:32.967 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cephfs-2:19.0.0-2976.g3f51ea39.el9.x86_64 10/34 2024-06-11T13:20:32.970 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rados-2:19.0.0-2976.g3f51ea39.el9.x86_64 11/34 2024-06-11T13:20:32.970 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rbd-2:19.0.0-2976.g3f51ea39.el9.x86_64 12/34 2024-06-11T13:20:32.970 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rgw-2:19.0.0-2976.g3f51ea39.el9.x86_64 13/34 2024-06-11T13:20:32.970 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-06-11T13:20:32.970 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-06-11T13:20:32.970 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-06-11T13:20:32.970 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-06-11T13:20:32.970 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-06-11T13:20:32.970 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-06-11T13:20:32.970 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-06-11T13:20:32.971 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-06-11T13:20:32.971 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-06-11T13:20:32.971 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-06-11T13:20:32.971 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-06-11T13:20:32.971 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-06-11T13:20:32.971 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-06-11T13:20:32.971 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-06-11T13:20:32.971 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-06-11T13:20:32.971 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-06-11T13:20:32.971 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-06-11T13:20:32.971 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados2-2:19.0.0-2976.g3f51ea39.el9.x86_64 31/34 2024-06-11T13:20:32.971 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-06-11T13:20:32.971 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librbd1-2:19.0.0-2976.g3f51ea39.el9.x86_64 33/34 2024-06-11T13:20:33.302 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2976.g3f51ea39.el9.x86_64 3/4 2024-06-11T13:20:33.390 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-test-2:19.0.0-2976.g3f51ea39.el9.x86_64 4/4 2024-06-11T13:20:33.792 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-06-11T13:20:33.793 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:20:33.793 INFO:teuthology.orchestra.run.smithi107.stdout:Upgraded: 2024-06-11T13:20:33.793 INFO:teuthology.orchestra.run.smithi107.stdout: librados2-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:20:33.793 INFO:teuthology.orchestra.run.smithi107.stdout: librbd1-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:20:33.793 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-06-11T13:20:33.793 INFO:teuthology.orchestra.run.smithi107.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-11T13:20:33.793 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:20:33.793 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-common-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:20:33.793 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-radosgw-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:20:33.793 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-selinux-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:20:33.793 INFO:teuthology.orchestra.run.smithi107.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-11T13:20:33.793 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-11T13:20:33.793 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-11T13:20:33.794 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs2-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:20:33.794 INFO:teuthology.orchestra.run.smithi107.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-11T13:20:33.794 INFO:teuthology.orchestra.run.smithi107.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-11T13:20:33.794 INFO:teuthology.orchestra.run.smithi107.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-11T13:20:33.794 INFO:teuthology.orchestra.run.smithi107.stdout: libradosstriper1-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:20:33.794 INFO:teuthology.orchestra.run.smithi107.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-11T13:20:33.794 INFO:teuthology.orchestra.run.smithi107.stdout: librgw2-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:20:33.794 INFO:teuthology.orchestra.run.smithi107.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-11T13:20:33.794 INFO:teuthology.orchestra.run.smithi107.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-11T13:20:33.794 INFO:teuthology.orchestra.run.smithi107.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-11T13:20:33.794 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-argparse-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:20:33.794 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-common-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:20:33.794 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cephfs-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:20:33.795 INFO:teuthology.orchestra.run.smithi107.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-11T13:20:33.795 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rados-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:20:33.795 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rbd-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:20:33.795 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rgw-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:20:33.795 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-11T13:20:33.795 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-11T13:20:33.795 INFO:teuthology.orchestra.run.smithi107.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-11T13:20:33.795 INFO:teuthology.orchestra.run.smithi107.stdout: re2-1:20211101-3.el9.x86_64 2024-06-11T13:20:33.795 INFO:teuthology.orchestra.run.smithi107.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-11T13:20:33.795 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:20:33.795 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-06-11T13:20:34.131 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-test 2024-06-11T13:20:34.711 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:28 ago on Tue 11 Jun 2024 01:20:06 PM UTC. 2024-06-11T13:20:34.833 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-06-11T13:20:34.833 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:20:34.833 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-06-11T13:20:34.833 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:20:34.833 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-06-11T13:20:34.833 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-test x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 48 M 2024-06-11T13:20:34.834 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-06-11T13:20:34.834 INFO:teuthology.orchestra.run.smithi107.stdout: libcephsqlite x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 163 k 2024-06-11T13:20:34.834 INFO:teuthology.orchestra.run.smithi107.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-06-11T13:20:34.834 INFO:teuthology.orchestra.run.smithi107.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-11T13:20:34.834 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:20:34.834 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-06-11T13:20:34.834 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:20:34.834 INFO:teuthology.orchestra.run.smithi107.stdout:Install 4 Packages 2024-06-11T13:20:34.834 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:20:34.835 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 49 M 2024-06-11T13:20:34.835 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 207 M 2024-06-11T13:20:34.835 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-06-11T13:20:35.184 INFO:teuthology.orchestra.run.smithi107.stdout:[MIRROR] socat-1.7.4.1-5.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/socat-1.7.4.1-5.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-11T13:20:35.338 INFO:teuthology.orchestra.run.smithi107.stdout:(1/4): libcephsqlite-19.0.0-2976.g3f51ea39.el9. 711 kB/s | 163 kB 00:00 2024-06-11T13:20:35.339 INFO:teuthology.orchestra.run.smithi107.stdout:[MIRROR] xmlstarlet-1.6.1-20.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/xmlstarlet-1.6.1-20.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-11T13:20:35.396 INFO:teuthology.orchestra.run.smithi107.stdout:(2/4): socat-1.7.4.1-5.el9.x86_64.rpm 1.0 MB/s | 305 kB 00:00 2024-06-11T13:20:35.505 INFO:teuthology.orchestra.run.smithi107.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 382 kB/s | 64 kB 00:00 2024-06-11T13:20:35.644 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-test-2:19.0.0-2976.g3f51ea39.el9.x86_64 4/4 2024-06-11T13:20:35.644 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-test-2:19.0.0-2976.g3f51ea39.el9.x86_64 1/4 2024-06-11T13:20:35.644 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libcephsqlite-2:19.0.0-2976.g3f51ea39.el9.x86_64 2/4 2024-06-11T13:20:35.644 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-06-11T13:20:35.921 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-06-11T13:20:35.921 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:20:35.921 INFO:teuthology.orchestra.run.smithi057.stdout:Installed: 2024-06-11T13:20:35.921 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-test-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:20:35.921 INFO:teuthology.orchestra.run.smithi057.stdout: libcephsqlite-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:20:35.921 INFO:teuthology.orchestra.run.smithi057.stdout: socat-1.7.4.1-5.el9.x86_64 2024-06-11T13:20:35.921 INFO:teuthology.orchestra.run.smithi057.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-11T13:20:35.921 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:20:35.921 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-06-11T13:20:36.188 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install ceph 2024-06-11T13:20:36.779 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:00:44 ago on Tue 11 Jun 2024 01:19:52 PM UTC. 2024-06-11T13:20:36.909 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-06-11T13:20:36.912 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:20:36.912 INFO:teuthology.orchestra.run.smithi057.stdout: Package Arch Version Repository Size 2024-06-11T13:20:36.912 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:20:36.912 INFO:teuthology.orchestra.run.smithi057.stdout:Installing: 2024-06-11T13:20:36.912 INFO:teuthology.orchestra.run.smithi057.stdout: ceph x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 6.6 k 2024-06-11T13:20:36.912 INFO:teuthology.orchestra.run.smithi057.stdout:Installing dependencies: 2024-06-11T13:20:36.912 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mds x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 2.4 M 2024-06-11T13:20:36.912 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 1.5 M 2024-06-11T13:20:36.912 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2976.g3f51ea39.el9 ceph-noarch 249 k 2024-06-11T13:20:36.912 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mon x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 4.7 M 2024-06-11T13:20:36.912 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-osd x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 17 M 2024-06-11T13:20:36.912 INFO:teuthology.orchestra.run.smithi057.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-11T13:20:36.913 INFO:teuthology.orchestra.run.smithi057.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-11T13:20:36.913 INFO:teuthology.orchestra.run.smithi057.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-11T13:20:36.913 INFO:teuthology.orchestra.run.smithi057.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-11T13:20:36.913 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-11T13:20:36.913 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-11T13:20:36.913 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-11T13:20:36.913 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-11T13:20:36.913 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-11T13:20:36.913 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-11T13:20:36.913 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco-collections 2024-06-11T13:20:36.913 INFO:teuthology.orchestra.run.smithi057.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-06-11T13:20:36.913 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-11T13:20:36.913 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-11T13:20:36.913 INFO:teuthology.orchestra.run.smithi057.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-11T13:20:36.914 INFO:teuthology.orchestra.run.smithi057.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-11T13:20:36.914 INFO:teuthology.orchestra.run.smithi057.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-11T13:20:36.914 INFO:teuthology.orchestra.run.smithi057.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-11T13:20:36.914 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-11T13:20:36.914 INFO:teuthology.orchestra.run.smithi057.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-11T13:20:36.914 INFO:teuthology.orchestra.run.smithi057.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-11T13:20:36.914 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-11T13:20:36.914 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-11T13:20:36.914 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-11T13:20:36.914 INFO:teuthology.orchestra.run.smithi057.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-11T13:20:36.914 INFO:teuthology.orchestra.run.smithi057.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-11T13:20:36.914 INFO:teuthology.orchestra.run.smithi057.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-11T13:20:36.914 INFO:teuthology.orchestra.run.smithi057.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-11T13:20:36.915 INFO:teuthology.orchestra.run.smithi057.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-11T13:20:36.915 INFO:teuthology.orchestra.run.smithi057.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-11T13:20:36.915 INFO:teuthology.orchestra.run.smithi057.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-11T13:20:36.915 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:20:36.915 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction Summary 2024-06-11T13:20:36.915 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:20:36.915 INFO:teuthology.orchestra.run.smithi057.stdout:Install 36 Packages 2024-06-11T13:20:36.915 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:20:36.916 INFO:teuthology.orchestra.run.smithi057.stdout:Total download size: 30 M 2024-06-11T13:20:36.916 INFO:teuthology.orchestra.run.smithi057.stdout:Installed size: 107 M 2024-06-11T13:20:36.916 INFO:teuthology.orchestra.run.smithi057.stdout:Downloading Packages: 2024-06-11T13:20:37.563 INFO:teuthology.orchestra.run.smithi057.stdout:(1/36): ceph-19.0.0-2976.g3f51ea39.el9.x86_64.r 29 kB/s | 6.6 kB 00:00 2024-06-11T13:20:37.622 INFO:teuthology.orchestra.run.smithi107.stdout:(4/4): ceph-test-19.0.0-2976.g3f51ea39.el9.x86_ 19 MB/s | 48 MB 00:02 2024-06-11T13:20:37.624 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-06-11T13:20:37.625 INFO:teuthology.orchestra.run.smithi107.stdout:Total 17 MB/s | 49 MB 00:02 2024-06-11T13:20:37.652 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-06-11T13:20:37.673 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-06-11T13:20:37.674 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-06-11T13:20:37.730 INFO:teuthology.orchestra.run.smithi057.stdout:(2/36): ceph-mds-19.0.0-2976.g3f51ea39.el9.x86_ 6.0 MB/s | 2.4 MB 00:00 2024-06-11T13:20:37.772 INFO:teuthology.orchestra.run.smithi057.stdout:(3/36): ceph-mgr-19.0.0-2976.g3f51ea39.el9.x86_ 3.4 MB/s | 1.5 MB 00:00 2024-06-11T13:20:37.856 INFO:teuthology.orchestra.run.smithi057.stdout:(4/36): ceph-mgr-modules-core-19.0.0-2976.g3f51 2.9 MB/s | 249 kB 00:00 2024-06-11T13:20:37.893 INFO:teuthology.orchestra.run.smithi057.stdout:[MIRROR] python3-cffi-1.14.5-5.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/python3-cffi-1.14.5-5.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-11T13:20:38.043 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-06-11T13:20:38.044 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-06-11T13:20:38.098 INFO:teuthology.orchestra.run.smithi057.stdout:(5/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.0 MB/s | 253 kB 00:00 2024-06-11T13:20:38.099 INFO:teuthology.orchestra.run.smithi057.stdout:[MIRROR] python3-cryptography-36.0.1-4.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/python3-cryptography-36.0.1-4.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-11T13:20:38.373 INFO:teuthology.orchestra.run.smithi057.stdout:(6/36): python3-cryptography-36.0.1-4.el9.x86_6 4.5 MB/s | 1.2 MB 00:00 2024-06-11T13:20:38.374 INFO:teuthology.orchestra.run.smithi057.stdout:[MIRROR] python3-ply-3.11-14.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/python3-ply-3.11-14.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-11T13:20:38.540 INFO:teuthology.orchestra.run.smithi057.stdout:(7/36): ceph-osd-19.0.0-2976.g3f51ea39.el9.x86_ 21 MB/s | 17 MB 00:00 2024-06-11T13:20:38.564 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-06-11T13:20:38.565 INFO:teuthology.orchestra.run.smithi057.stdout:(8/36): python3-ply-3.11-14.el9.noarch.rpm 555 kB/s | 106 kB 00:00 2024-06-11T13:20:38.624 INFO:teuthology.orchestra.run.smithi057.stdout:(9/36): python3-requests-2.25.1-8.el9.noarch.rp 2.1 MB/s | 125 kB 00:00 2024-06-11T13:20:38.659 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-06-11T13:20:38.682 INFO:teuthology.orchestra.run.smithi057.stdout:(10/36): python3-urllib3-1.26.5-5.el9.noarch.rp 3.6 MB/s | 215 kB 00:00 2024-06-11T13:20:38.683 INFO:teuthology.orchestra.run.smithi057.stdout:[MIRROR] lua-5.4.4-4.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/lua-5.4.4-4.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-11T13:20:38.704 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-06-11T13:20:38.732 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephsqlite-2:19.0.0-2976.g3f51ea39.el9.x86_64 3/4 2024-06-11T13:20:38.741 INFO:teuthology.orchestra.run.smithi057.stdout:(11/36): python3-pycparser-2.20-6.el9.noarch.rp 674 kB/s | 135 kB 00:00 2024-06-11T13:20:38.742 INFO:teuthology.orchestra.run.smithi057.stdout:[MIRROR] python3-mako-1.1.4-6.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-mako-1.1.4-6.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-11T13:20:38.866 INFO:teuthology.orchestra.run.smithi057.stdout:(12/36): lua-5.4.4-4.el9.x86_64.rpm 1.0 MB/s | 188 kB 00:00 2024-06-11T13:20:38.892 INFO:teuthology.orchestra.run.smithi057.stdout:(13/36): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-06-11T13:20:38.892 INFO:teuthology.orchestra.run.smithi057.stdout:[MIRROR] python3-markupsafe-1.1.1-12.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-markupsafe-1.1.1-12.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-11T13:20:38.892 INFO:teuthology.orchestra.run.smithi057.stdout:[MIRROR] python3-pytz-2021.1-5.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-pytz-2021.1-5.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-11T13:20:38.944 INFO:teuthology.orchestra.run.smithi057.stdout:(14/36): python3-markupsafe-1.1.1-12.el9.x86_64 449 kB/s | 35 kB 00:00 2024-06-11T13:20:38.961 INFO:teuthology.orchestra.run.smithi057.stdout:(15/36): python3-pytz-2021.1-5.el9.noarch.rpm 738 kB/s | 51 kB 00:00 2024-06-11T13:20:38.961 INFO:teuthology.orchestra.run.smithi057.stdout:[MIRROR] lua-devel-5.4.4-4.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/lua-devel-5.4.4-4.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-11T13:20:38.986 INFO:teuthology.orchestra.run.smithi057.stdout:(16/36): python3-toml-0.10.2-6.el9.noarch.rpm 994 kB/s | 42 kB 00:00 2024-06-11T13:20:39.011 INFO:teuthology.orchestra.run.smithi057.stdout:(17/36): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-06-11T13:20:39.037 INFO:teuthology.orchestra.run.smithi057.stdout:(18/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 1.7 MB/s | 43 kB 00:00 2024-06-11T13:20:39.062 INFO:teuthology.orchestra.run.smithi057.stdout:(19/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 6.8 MB/s | 172 kB 00:00 2024-06-11T13:20:39.091 INFO:teuthology.orchestra.run.smithi057.stdout:(20/36): lua-devel-5.4.4-4.el9.x86_64.rpm 176 kB/s | 22 kB 00:00 2024-06-11T13:20:39.138 INFO:teuthology.orchestra.run.smithi057.stdout:(21/36): python3-cherrypy-18.6.1-2.el9.noarch.r 4.6 MB/s | 358 kB 00:00 2024-06-11T13:20:39.163 INFO:teuthology.orchestra.run.smithi057.stdout:(22/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 142 kB/s | 11 kB 00:00 2024-06-11T13:20:39.188 INFO:teuthology.orchestra.run.smithi057.stdout:(23/36): python3-jaraco-classes-3.2.1-5.el9.noa 370 kB/s | 18 kB 00:00 2024-06-11T13:20:39.213 INFO:teuthology.orchestra.run.smithi057.stdout:(24/36): python3-jaraco-collections-3.0.0-8.el9 930 kB/s | 23 kB 00:00 2024-06-11T13:20:39.238 INFO:teuthology.orchestra.run.smithi057.stdout:(25/36): python3-jaraco-functools-3.5.0-2.el9.n 389 kB/s | 19 kB 00:00 2024-06-11T13:20:39.264 INFO:teuthology.orchestra.run.smithi057.stdout:(26/36): python3-jaraco-text-3.2.0-6.el9.noarch 792 kB/s | 20 kB 00:00 2024-06-11T13:20:39.289 INFO:teuthology.orchestra.run.smithi057.stdout:(27/36): python3-logutils-0.3.5-21.el9.noarch.r 923 kB/s | 46 kB 00:00 2024-06-11T13:20:39.315 INFO:teuthology.orchestra.run.smithi057.stdout:(28/36): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-06-11T13:20:39.340 INFO:teuthology.orchestra.run.smithi057.stdout:(29/36): python3-portend-3.1.0-2.el9.noarch.rpm 657 kB/s | 16 kB 00:00 2024-06-11T13:20:39.407 INFO:teuthology.orchestra.run.smithi057.stdout:(30/36): ceph-mon-19.0.0-2976.g3f51ea39.el9.x86 2.5 MB/s | 4.7 MB 00:01 2024-06-11T13:20:39.432 INFO:teuthology.orchestra.run.smithi057.stdout:(31/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 979 kB/s | 90 kB 00:00 2024-06-11T13:20:39.482 INFO:teuthology.orchestra.run.smithi057.stdout:(32/36): python3-pecan-1.4.2-3.el9.noarch.rpm 1.4 MB/s | 272 kB 00:00 2024-06-11T13:20:39.508 INFO:teuthology.orchestra.run.smithi057.stdout:(33/36): python3-tempora-5.0.0-2.el9.noarch.rpm 355 kB/s | 36 kB 00:00 2024-06-11T13:20:39.533 INFO:teuthology.orchestra.run.smithi057.stdout:(34/36): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-06-11T13:20:39.558 INFO:teuthology.orchestra.run.smithi057.stdout:(35/36): python3-zc-lockfile-2.0-10.el9.noarch. 398 kB/s | 20 kB 00:00 2024-06-11T13:20:39.583 INFO:teuthology.orchestra.run.smithi057.stdout:(36/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-06-11T13:20:39.587 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2024-06-11T13:20:39.587 INFO:teuthology.orchestra.run.smithi057.stdout:Total 11 MB/s | 30 MB 00:02 2024-06-11T13:20:39.727 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction check 2024-06-11T13:20:39.769 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction check succeeded. 2024-06-11T13:20:39.770 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction test 2024-06-11T13:20:40.243 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction test succeeded. 2024-06-11T13:20:40.244 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction 2024-06-11T13:20:41.502 INFO:teuthology.orchestra.run.smithi057.stdout: Preparing : 1/1 2024-06-11T13:20:41.553 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-06-11T13:20:41.595 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-06-11T13:20:41.623 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-06-11T13:20:41.696 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-06-11T13:20:41.747 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/36 2024-06-11T13:20:41.780 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-06-11T13:20:41.876 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-06-11T13:20:41.963 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-06-11T13:20:42.008 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-06-11T13:20:42.043 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-06-11T13:20:42.072 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/36 2024-06-11T13:20:42.107 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/36 2024-06-11T13:20:42.140 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/36 2024-06-11T13:20:42.181 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/36 2024-06-11T13:20:42.225 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/36 2024-06-11T13:20:42.259 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/36 2024-06-11T13:20:42.297 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/36 2024-06-11T13:20:42.374 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 18/36 2024-06-11T13:20:42.524 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 19/36 2024-06-11T13:20:42.608 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 20/36 2024-06-11T13:20:42.659 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 21/36 2024-06-11T13:20:42.697 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 22/36 2024-06-11T13:20:42.756 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-ply-3.11-14.el9.noarch 23/36 2024-06-11T13:20:42.820 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 24/36 2024-06-11T13:20:43.015 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 25/36 2024-06-11T13:20:43.068 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 26/36 2024-06-11T13:20:43.149 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 27/36 2024-06-11T13:20:43.238 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 28/36 2024-06-11T13:20:43.276 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 29/36 2024-06-11T13:20:43.363 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2976.g3f51ea39.el9.x86_64 3/4 2024-06-11T13:20:43.400 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 30/36 2024-06-11T13:20:43.438 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-test-2:19.0.0-2976.g3f51ea39.el9.x86_64 4/4 2024-06-11T13:20:43.572 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2976.g3f51ea39.el9. 31/36 2024-06-11T13:20:43.602 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mgr-2:19.0.0-2976.g3f51ea39.el9.x86_64 32/36 2024-06-11T13:20:43.632 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2976.g3f51ea39.el9.x86_64 32/36 2024-06-11T13:20:43.632 INFO:teuthology.orchestra.run.smithi057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-11T13:20:43.632 INFO:teuthology.orchestra.run.smithi057.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-11T13:20:43.632 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-11T13:20:43.632 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-11T13:20:43.633 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:20:45.153 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-osd-2:19.0.0-2976.g3f51ea39.el9.x86_64 33/36 2024-06-11T13:20:45.182 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-osd-2:19.0.0-2976.g3f51ea39.el9.x86_64 33/36 2024-06-11T13:20:45.182 INFO:teuthology.orchestra.run.smithi057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-11T13:20:45.182 INFO:teuthology.orchestra.run.smithi057.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-11T13:20:45.183 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-11T13:20:45.183 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-11T13:20:45.183 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:20:45.654 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mon-2:19.0.0-2976.g3f51ea39.el9.x86_64 34/36 2024-06-11T13:20:45.679 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mon-2:19.0.0-2976.g3f51ea39.el9.x86_64 34/36 2024-06-11T13:20:45.679 INFO:teuthology.orchestra.run.smithi057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-11T13:20:45.679 INFO:teuthology.orchestra.run.smithi057.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-11T13:20:45.679 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-11T13:20:45.679 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-11T13:20:45.679 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:20:45.740 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-test-2:19.0.0-2976.g3f51ea39.el9.x86_64 4/4 2024-06-11T13:20:45.740 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-test-2:19.0.0-2976.g3f51ea39.el9.x86_64 1/4 2024-06-11T13:20:45.740 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephsqlite-2:19.0.0-2976.g3f51ea39.el9.x86_64 2/4 2024-06-11T13:20:45.740 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-06-11T13:20:45.913 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mds-2:19.0.0-2976.g3f51ea39.el9.x86_64 35/36 2024-06-11T13:20:45.937 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mds-2:19.0.0-2976.g3f51ea39.el9.x86_64 35/36 2024-06-11T13:20:45.938 INFO:teuthology.orchestra.run.smithi057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-11T13:20:45.938 INFO:teuthology.orchestra.run.smithi057.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-11T13:20:45.938 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-11T13:20:45.938 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-11T13:20:45.938 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:20:46.004 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-2:19.0.0-2976.g3f51ea39.el9.x86_64 36/36 2024-06-11T13:20:46.040 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-06-11T13:20:46.040 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:20:46.040 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-06-11T13:20:46.040 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-test-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:20:46.040 INFO:teuthology.orchestra.run.smithi107.stdout: libcephsqlite-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:20:46.040 INFO:teuthology.orchestra.run.smithi107.stdout: socat-1.7.4.1-5.el9.x86_64 2024-06-11T13:20:46.040 INFO:teuthology.orchestra.run.smithi107.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-11T13:20:46.040 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:20:46.041 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-06-11T13:20:46.306 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph 2024-06-11T13:20:46.885 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:40 ago on Tue 11 Jun 2024 01:20:06 PM UTC. 2024-06-11T13:20:47.008 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-06-11T13:20:47.011 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:20:47.011 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-06-11T13:20:47.011 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:20:47.011 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-06-11T13:20:47.011 INFO:teuthology.orchestra.run.smithi107.stdout: ceph x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 6.6 k 2024-06-11T13:20:47.011 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-06-11T13:20:47.011 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mds x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 2.4 M 2024-06-11T13:20:47.011 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 1.5 M 2024-06-11T13:20:47.011 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2976.g3f51ea39.el9 ceph-noarch 249 k 2024-06-11T13:20:47.011 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 4.7 M 2024-06-11T13:20:47.011 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-osd x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 17 M 2024-06-11T13:20:47.011 INFO:teuthology.orchestra.run.smithi107.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-11T13:20:47.012 INFO:teuthology.orchestra.run.smithi107.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-11T13:20:47.012 INFO:teuthology.orchestra.run.smithi107.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-11T13:20:47.012 INFO:teuthology.orchestra.run.smithi107.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-11T13:20:47.012 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-11T13:20:47.012 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-11T13:20:47.012 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-11T13:20:47.012 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-11T13:20:47.012 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-11T13:20:47.012 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-11T13:20:47.012 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-collections 2024-06-11T13:20:47.012 INFO:teuthology.orchestra.run.smithi107.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-06-11T13:20:47.012 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-11T13:20:47.012 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-11T13:20:47.012 INFO:teuthology.orchestra.run.smithi107.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-11T13:20:47.013 INFO:teuthology.orchestra.run.smithi107.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-11T13:20:47.013 INFO:teuthology.orchestra.run.smithi107.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-11T13:20:47.013 INFO:teuthology.orchestra.run.smithi107.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-11T13:20:47.013 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-11T13:20:47.013 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-11T13:20:47.013 INFO:teuthology.orchestra.run.smithi107.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-11T13:20:47.013 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-11T13:20:47.013 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-11T13:20:47.013 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-11T13:20:47.013 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-11T13:20:47.013 INFO:teuthology.orchestra.run.smithi107.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-11T13:20:47.013 INFO:teuthology.orchestra.run.smithi107.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-11T13:20:47.013 INFO:teuthology.orchestra.run.smithi107.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-11T13:20:47.013 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-11T13:20:47.014 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-11T13:20:47.014 INFO:teuthology.orchestra.run.smithi107.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-11T13:20:47.014 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:20:47.014 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-06-11T13:20:47.014 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:20:47.014 INFO:teuthology.orchestra.run.smithi107.stdout:Install 36 Packages 2024-06-11T13:20:47.014 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:20:47.014 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 30 M 2024-06-11T13:20:47.015 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 107 M 2024-06-11T13:20:47.015 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-06-11T13:20:47.614 INFO:teuthology.orchestra.run.smithi107.stdout:(1/36): ceph-19.0.0-2976.g3f51ea39.el9.x86_64.r 38 kB/s | 6.6 kB 00:00 2024-06-11T13:20:47.725 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-2:19.0.0-2976.g3f51ea39.el9.x86_64 36/36 2024-06-11T13:20:47.726 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-2:19.0.0-2976.g3f51ea39.el9.x86_64 1/36 2024-06-11T13:20:47.726 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mds-2:19.0.0-2976.g3f51ea39.el9.x86_64 2/36 2024-06-11T13:20:47.726 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mgr-2:19.0.0-2976.g3f51ea39.el9.x86_64 3/36 2024-06-11T13:20:47.726 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mon-2:19.0.0-2976.g3f51ea39.el9.x86_64 4/36 2024-06-11T13:20:47.726 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-osd-2:19.0.0-2976.g3f51ea39.el9.x86_64 5/36 2024-06-11T13:20:47.726 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2976.g3f51ea39.el9. 6/36 2024-06-11T13:20:47.726 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/36 2024-06-11T13:20:47.726 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/36 2024-06-11T13:20:47.726 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/36 2024-06-11T13:20:47.726 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/36 2024-06-11T13:20:47.726 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/36 2024-06-11T13:20:47.726 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/36 2024-06-11T13:20:47.727 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/36 2024-06-11T13:20:47.727 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/36 2024-06-11T13:20:47.727 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/36 2024-06-11T13:20:47.727 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/36 2024-06-11T13:20:47.727 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/36 2024-06-11T13:20:47.727 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-06-11T13:20:47.727 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 19/36 2024-06-11T13:20:47.727 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-06-11T13:20:47.727 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-06-11T13:20:47.727 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-06-11T13:20:47.727 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-06-11T13:20:47.728 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-06-11T13:20:47.728 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-06-11T13:20:47.728 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-06-11T13:20:47.728 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-06-11T13:20:47.728 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-06-11T13:20:47.729 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-06-11T13:20:47.729 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-06-11T13:20:47.729 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-06-11T13:20:47.729 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-06-11T13:20:47.729 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-06-11T13:20:47.729 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-06-11T13:20:47.729 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-06-11T13:20:47.822 INFO:teuthology.orchestra.run.smithi107.stdout:(2/36): ceph-mgr-19.0.0-2976.g3f51ea39.el9.x86_ 3.9 MB/s | 1.5 MB 00:00 2024-06-11T13:20:48.190 INFO:teuthology.orchestra.run.smithi107.stdout:(3/36): ceph-mon-19.0.0-2976.g3f51ea39.el9.x86_ 8.2 MB/s | 4.7 MB 00:00 2024-06-11T13:20:48.265 INFO:teuthology.orchestra.run.smithi107.stdout:(4/36): ceph-mgr-modules-core-19.0.0-2976.g3f51 3.3 MB/s | 249 kB 00:00 2024-06-11T13:20:48.302 INFO:teuthology.orchestra.run.smithi107.stdout:[MIRROR] python3-cffi-1.14.5-5.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/python3-cffi-1.14.5-5.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-11T13:20:48.715 INFO:teuthology.orchestra.run.smithi107.stdout:(5/36): ceph-osd-19.0.0-2976.g3f51ea39.el9.x86_ 19 MB/s | 17 MB 00:00 2024-06-11T13:20:48.716 INFO:teuthology.orchestra.run.smithi107.stdout:[MIRROR] python3-cryptography-36.0.1-4.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/python3-cryptography-36.0.1-4.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-11T13:20:48.801 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-06-11T13:20:48.802 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:20:48.802 INFO:teuthology.orchestra.run.smithi057.stdout:Installed: 2024-06-11T13:20:48.802 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:20:48.802 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mds-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:20:48.802 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:20:48.802 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-modules-core-2:19.0.0-2976.g3f51ea39.el9.noarch 2024-06-11T13:20:48.802 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mon-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:20:48.802 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-osd-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:20:48.802 INFO:teuthology.orchestra.run.smithi057.stdout: lua-5.4.4-4.el9.x86_64 2024-06-11T13:20:48.802 INFO:teuthology.orchestra.run.smithi057.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-11T13:20:48.802 INFO:teuthology.orchestra.run.smithi057.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-11T13:20:48.802 INFO:teuthology.orchestra.run.smithi057.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-11T13:20:48.803 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-11T13:20:48.803 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-11T13:20:48.803 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-11T13:20:48.803 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-11T13:20:48.803 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-11T13:20:48.803 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-11T13:20:48.803 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-11T13:20:48.803 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-11T13:20:48.803 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-11T13:20:48.803 INFO:teuthology.orchestra.run.smithi057.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-11T13:20:48.803 INFO:teuthology.orchestra.run.smithi057.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-11T13:20:48.803 INFO:teuthology.orchestra.run.smithi057.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-11T13:20:48.803 INFO:teuthology.orchestra.run.smithi057.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-11T13:20:48.804 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-11T13:20:48.804 INFO:teuthology.orchestra.run.smithi057.stdout: python3-ply-3.11-14.el9.noarch 2024-06-11T13:20:48.804 INFO:teuthology.orchestra.run.smithi057.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-11T13:20:48.804 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-11T13:20:48.804 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-11T13:20:48.804 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-11T13:20:48.804 INFO:teuthology.orchestra.run.smithi057.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-11T13:20:48.804 INFO:teuthology.orchestra.run.smithi057.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-11T13:20:48.804 INFO:teuthology.orchestra.run.smithi057.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-11T13:20:48.804 INFO:teuthology.orchestra.run.smithi057.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-11T13:20:48.804 INFO:teuthology.orchestra.run.smithi057.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-11T13:20:48.804 INFO:teuthology.orchestra.run.smithi057.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-11T13:20:48.804 INFO:teuthology.orchestra.run.smithi057.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-11T13:20:48.804 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:20:48.805 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-06-11T13:20:49.007 INFO:teuthology.orchestra.run.smithi107.stdout:(6/36): ceph-mds-19.0.0-2976.g3f51ea39.el9.x86_ 1.5 MB/s | 2.4 MB 00:01 2024-06-11T13:20:49.032 INFO:teuthology.orchestra.run.smithi107.stdout:(7/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 330 kB/s | 253 kB 00:00 2024-06-11T13:20:49.033 INFO:teuthology.orchestra.run.smithi107.stdout:[MIRROR] python3-ply-3.11-14.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/python3-ply-3.11-14.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-11T13:20:49.033 INFO:teuthology.orchestra.run.smithi107.stdout:[MIRROR] python3-pycparser-2.20-6.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/python3-pycparser-2.20-6.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-11T13:20:49.144 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install ceph-base 2024-06-11T13:20:49.249 INFO:teuthology.orchestra.run.smithi107.stdout:(8/36): python3-cryptography-36.0.1-4.el9.x86_6 2.3 MB/s | 1.2 MB 00:00 2024-06-11T13:20:49.274 INFO:teuthology.orchestra.run.smithi107.stdout:(9/36): python3-ply-3.11-14.el9.noarch.rpm 398 kB/s | 106 kB 00:00 2024-06-11T13:20:49.299 INFO:teuthology.orchestra.run.smithi107.stdout:(10/36): python3-pycparser-2.20-6.el9.noarch.rp 506 kB/s | 135 kB 00:00 2024-06-11T13:20:49.301 INFO:teuthology.orchestra.run.smithi107.stdout:[MIRROR] lua-5.4.4-4.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/lua-5.4.4-4.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-11T13:20:49.358 INFO:teuthology.orchestra.run.smithi107.stdout:(11/36): python3-requests-2.25.1-8.el9.noarch.r 1.1 MB/s | 125 kB 00:00 2024-06-11T13:20:49.359 INFO:teuthology.orchestra.run.smithi107.stdout:[MIRROR] python3-mako-1.1.4-6.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-mako-1.1.4-6.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-11T13:20:49.409 INFO:teuthology.orchestra.run.smithi107.stdout:(12/36): python3-urllib3-1.26.5-5.el9.noarch.rp 1.9 MB/s | 215 kB 00:00 2024-06-11T13:20:49.409 INFO:teuthology.orchestra.run.smithi107.stdout:[MIRROR] python3-markupsafe-1.1.1-12.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-markupsafe-1.1.1-12.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-11T13:20:49.467 INFO:teuthology.orchestra.run.smithi107.stdout:(13/36): lua-5.4.4-4.el9.x86_64.rpm 1.1 MB/s | 188 kB 00:00 2024-06-11T13:20:49.484 INFO:teuthology.orchestra.run.smithi107.stdout:(14/36): python3-markupsafe-1.1.1-12.el9.x86_64 460 kB/s | 35 kB 00:00 2024-06-11T13:20:49.510 INFO:teuthology.orchestra.run.smithi107.stdout:(15/36): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-06-11T13:20:49.526 INFO:teuthology.orchestra.run.smithi107.stdout:(16/36): python3-pytz-2021.1-5.el9.noarch.rpm 872 kB/s | 51 kB 00:00 2024-06-11T13:20:49.543 INFO:teuthology.orchestra.run.smithi107.stdout:(17/36): python3-toml-0.10.2-6.el9.noarch.rpm 711 kB/s | 42 kB 00:00 2024-06-11T13:20:49.571 INFO:teuthology.orchestra.run.smithi107.stdout:[MIRROR] lua-devel-5.4.4-4.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/lua-devel-5.4.4-4.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-11T13:20:49.585 INFO:teuthology.orchestra.run.smithi107.stdout:(18/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 2.9 MB/s | 43 kB 00:00 2024-06-11T13:20:49.619 INFO:teuthology.orchestra.run.smithi107.stdout:(19/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 5.4 MB/s | 172 kB 00:00 2024-06-11T13:20:49.644 INFO:teuthology.orchestra.run.smithi107.stdout:(20/36): luarocks-3.9.2-5.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-06-11T13:20:49.688 INFO:teuthology.orchestra.run.smithi107.stdout:(21/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 245 kB/s | 11 kB 00:00 2024-06-11T13:20:49.703 INFO:teuthology.orchestra.run.smithi107.stdout:(22/36): lua-devel-5.4.4-4.el9.x86_64.rpm 126 kB/s | 22 kB 00:00 2024-06-11T13:20:49.720 INFO:teuthology.orchestra.run.smithi107.stdout:(23/36): python3-jaraco-classes-3.2.1-5.el9.noa 562 kB/s | 18 kB 00:00 2024-06-11T13:20:49.751 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:00:57 ago on Tue 11 Jun 2024 01:19:52 PM UTC. 2024-06-11T13:20:49.770 INFO:teuthology.orchestra.run.smithi107.stdout:(24/36): python3-cherrypy-18.6.1-2.el9.noarch.r 2.3 MB/s | 358 kB 00:00 2024-06-11T13:20:49.796 INFO:teuthology.orchestra.run.smithi107.stdout:(25/36): python3-jaraco-functools-3.5.0-2.el9.n 402 kB/s | 19 kB 00:00 2024-06-11T13:20:49.821 INFO:teuthology.orchestra.run.smithi107.stdout:(26/36): python3-jaraco-text-3.2.0-6.el9.noarch 411 kB/s | 20 kB 00:00 2024-06-11T13:20:49.836 INFO:teuthology.orchestra.run.smithi057.stdout:Package ceph-base-2:19.0.0-2976.g3f51ea39.el9.x86_64 is already installed. 2024-06-11T13:20:49.846 INFO:teuthology.orchestra.run.smithi107.stdout:(27/36): python3-jaraco-collections-3.0.0-8.el9 162 kB/s | 23 kB 00:00 2024-06-11T13:20:49.871 INFO:teuthology.orchestra.run.smithi107.stdout:(28/36): python3-logutils-0.3.5-21.el9.noarch.r 613 kB/s | 46 kB 00:00 2024-06-11T13:20:49.879 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-06-11T13:20:49.881 INFO:teuthology.orchestra.run.smithi057.stdout:Nothing to do. 2024-06-11T13:20:49.881 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-06-11T13:20:49.897 INFO:teuthology.orchestra.run.smithi107.stdout:(29/36): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-06-11T13:20:49.922 INFO:teuthology.orchestra.run.smithi107.stdout:(30/36): python3-portend-3.1.0-2.el9.noarch.rpm 328 kB/s | 16 kB 00:00 2024-06-11T13:20:49.947 INFO:teuthology.orchestra.run.smithi107.stdout:(31/36): python3-tempora-5.0.0-2.el9.noarch.rpm 1.4 MB/s | 36 kB 00:00 2024-06-11T13:20:49.973 INFO:teuthology.orchestra.run.smithi107.stdout:(32/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-06-11T13:20:49.998 INFO:teuthology.orchestra.run.smithi107.stdout:(33/36): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-06-11T13:20:50.011 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install cephadm 2024-06-11T13:20:50.023 INFO:teuthology.orchestra.run.smithi107.stdout:(34/36): python3-zc-lockfile-2.0-10.el9.noarch. 806 kB/s | 20 kB 00:00 2024-06-11T13:20:50.048 INFO:teuthology.orchestra.run.smithi107.stdout:(35/36): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-06-11T13:20:50.074 INFO:teuthology.orchestra.run.smithi107.stdout:(36/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.1 MB/s | 427 kB 00:00 2024-06-11T13:20:50.079 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-06-11T13:20:50.079 INFO:teuthology.orchestra.run.smithi107.stdout:Total 9.8 MB/s | 30 MB 00:03 2024-06-11T13:20:50.221 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-06-11T13:20:50.260 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-06-11T13:20:50.260 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-06-11T13:20:50.607 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:00:58 ago on Tue 11 Jun 2024 01:19:52 PM UTC. 2024-06-11T13:20:50.721 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-06-11T13:20:50.722 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-06-11T13:20:50.735 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-06-11T13:20:50.736 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:20:50.736 INFO:teuthology.orchestra.run.smithi057.stdout: Package Arch Version Repository Size 2024-06-11T13:20:50.736 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:20:50.736 INFO:teuthology.orchestra.run.smithi057.stdout:Installing: 2024-06-11T13:20:50.736 INFO:teuthology.orchestra.run.smithi057.stdout: cephadm noarch 2:19.0.0-2976.g3f51ea39.el9 ceph-noarch 763 k 2024-06-11T13:20:50.736 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:20:50.736 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction Summary 2024-06-11T13:20:50.736 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:20:50.736 INFO:teuthology.orchestra.run.smithi057.stdout:Install 1 Package 2024-06-11T13:20:50.736 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:20:50.737 INFO:teuthology.orchestra.run.smithi057.stdout:Total download size: 763 k 2024-06-11T13:20:50.737 INFO:teuthology.orchestra.run.smithi057.stdout:Installed size: 769 k 2024-06-11T13:20:50.737 INFO:teuthology.orchestra.run.smithi057.stdout:Downloading Packages: 2024-06-11T13:20:51.197 INFO:teuthology.orchestra.run.smithi057.stdout:cephadm-19.0.0-2976.g3f51ea39.el9.noarch.rpm 1.6 MB/s | 763 kB 00:00 2024-06-11T13:20:51.197 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2024-06-11T13:20:51.198 INFO:teuthology.orchestra.run.smithi057.stdout:Total 1.6 MB/s | 763 kB 00:00 2024-06-11T13:20:51.198 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction check 2024-06-11T13:20:51.204 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction check succeeded. 2024-06-11T13:20:51.204 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction test 2024-06-11T13:20:51.210 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction test succeeded. 2024-06-11T13:20:51.210 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction 2024-06-11T13:20:51.294 INFO:teuthology.orchestra.run.smithi057.stdout: Preparing : 1/1 2024-06-11T13:20:51.763 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: cephadm-2:19.0.0-2976.g3f51ea39.el9.noarch 1/1 2024-06-11T13:20:51.855 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : cephadm-2:19.0.0-2976.g3f51ea39.el9.noarch 1/1 2024-06-11T13:20:51.968 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-06-11T13:20:52.034 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-06-11T13:20:52.067 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-06-11T13:20:52.095 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-06-11T13:20:52.168 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-06-11T13:20:52.186 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: cephadm-2:19.0.0-2976.g3f51ea39.el9.noarch 1/1 2024-06-11T13:20:52.219 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/36 2024-06-11T13:20:52.260 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-06-11T13:20:52.353 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-06-11T13:20:52.431 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-06-11T13:20:52.462 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : cephadm-2:19.0.0-2976.g3f51ea39.el9.noarch 1/1 2024-06-11T13:20:52.462 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:20:52.463 INFO:teuthology.orchestra.run.smithi057.stdout:Installed: 2024-06-11T13:20:52.463 INFO:teuthology.orchestra.run.smithi057.stdout: cephadm-2:19.0.0-2976.g3f51ea39.el9.noarch 2024-06-11T13:20:52.463 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:20:52.463 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-06-11T13:20:52.471 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-06-11T13:20:52.506 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-06-11T13:20:52.536 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/36 2024-06-11T13:20:52.570 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/36 2024-06-11T13:20:52.604 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/36 2024-06-11T13:20:52.644 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/36 2024-06-11T13:20:52.669 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install ceph-immutable-object-cache 2024-06-11T13:20:52.679 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/36 2024-06-11T13:20:52.714 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/36 2024-06-11T13:20:52.752 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/36 2024-06-11T13:20:52.825 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 18/36 2024-06-11T13:20:52.962 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 19/36 2024-06-11T13:20:53.044 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 20/36 2024-06-11T13:20:53.096 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 21/36 2024-06-11T13:20:53.143 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 22/36 2024-06-11T13:20:53.192 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ply-3.11-14.el9.noarch 23/36 2024-06-11T13:20:53.248 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 24/36 2024-06-11T13:20:53.274 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:01:01 ago on Tue 11 Jun 2024 01:19:52 PM UTC. 2024-06-11T13:20:53.402 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-06-11T13:20:53.403 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:20:53.403 INFO:teuthology.orchestra.run.smithi057.stdout: Package Arch Version Repo Size 2024-06-11T13:20:53.403 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:20:53.403 INFO:teuthology.orchestra.run.smithi057.stdout:Installing: 2024-06-11T13:20:53.403 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 145 k 2024-06-11T13:20:53.404 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:20:53.404 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction Summary 2024-06-11T13:20:53.404 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:20:53.404 INFO:teuthology.orchestra.run.smithi057.stdout:Install 1 Package 2024-06-11T13:20:53.404 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:20:53.404 INFO:teuthology.orchestra.run.smithi057.stdout:Total download size: 145 k 2024-06-11T13:20:53.404 INFO:teuthology.orchestra.run.smithi057.stdout:Installed size: 435 k 2024-06-11T13:20:53.404 INFO:teuthology.orchestra.run.smithi057.stdout:Downloading Packages: 2024-06-11T13:20:53.432 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 25/36 2024-06-11T13:20:53.483 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 26/36 2024-06-11T13:20:53.559 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 27/36 2024-06-11T13:20:53.648 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 28/36 2024-06-11T13:20:53.690 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 29/36 2024-06-11T13:20:53.804 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 30/36 2024-06-11T13:20:53.963 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-immutable-object-cache-19.0.0-2976.g3f51ea 259 kB/s | 145 kB 00:00 2024-06-11T13:20:53.964 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2024-06-11T13:20:53.964 INFO:teuthology.orchestra.run.smithi057.stdout:Total 259 kB/s | 145 kB 00:00 2024-06-11T13:20:53.964 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction check 2024-06-11T13:20:53.971 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2976.g3f51ea39.el9. 31/36 2024-06-11T13:20:53.972 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction check succeeded. 2024-06-11T13:20:53.973 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction test 2024-06-11T13:20:54.000 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-2:19.0.0-2976.g3f51ea39.el9.x86_64 32/36 2024-06-11T13:20:54.024 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2976.g3f51ea39.el9.x86_64 32/36 2024-06-11T13:20:54.024 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-11T13:20:54.025 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-11T13:20:54.025 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-11T13:20:54.025 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-11T13:20:54.025 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:20:54.034 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction test succeeded. 2024-06-11T13:20:54.034 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction 2024-06-11T13:20:54.184 INFO:teuthology.orchestra.run.smithi057.stdout: Preparing : 1/1 2024-06-11T13:20:54.255 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2976.g3f51ea39. 1/1 2024-06-11T13:20:54.281 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2976.g3f51ea39. 1/1 2024-06-11T13:20:54.281 INFO:teuthology.orchestra.run.smithi057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-11T13:20:54.281 INFO:teuthology.orchestra.run.smithi057.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-11T13:20:54.282 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:20:55.214 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2976.g3f51ea39. 1/1 2024-06-11T13:20:55.214 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:20:55.215 INFO:teuthology.orchestra.run.smithi057.stdout:Installed: 2024-06-11T13:20:55.215 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-immutable-object-cache-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:20:55.215 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:20:55.215 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-06-11T13:20:55.434 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install ceph-mgr 2024-06-11T13:20:55.459 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-osd-2:19.0.0-2976.g3f51ea39.el9.x86_64 33/36 2024-06-11T13:20:55.486 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-osd-2:19.0.0-2976.g3f51ea39.el9.x86_64 33/36 2024-06-11T13:20:55.486 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-11T13:20:55.486 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-11T13:20:55.487 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-11T13:20:55.487 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-11T13:20:55.487 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:20:55.927 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mon-2:19.0.0-2976.g3f51ea39.el9.x86_64 34/36 2024-06-11T13:20:55.950 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mon-2:19.0.0-2976.g3f51ea39.el9.x86_64 34/36 2024-06-11T13:20:55.951 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-11T13:20:55.951 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-11T13:20:55.951 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-11T13:20:55.951 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-11T13:20:55.951 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:20:56.048 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:01:04 ago on Tue 11 Jun 2024 01:19:52 PM UTC. 2024-06-11T13:20:56.132 INFO:teuthology.orchestra.run.smithi057.stdout:Package ceph-mgr-2:19.0.0-2976.g3f51ea39.el9.x86_64 is already installed. 2024-06-11T13:20:56.177 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-06-11T13:20:56.178 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mds-2:19.0.0-2976.g3f51ea39.el9.x86_64 35/36 2024-06-11T13:20:56.178 INFO:teuthology.orchestra.run.smithi057.stdout:Nothing to do. 2024-06-11T13:20:56.178 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-06-11T13:20:56.201 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mds-2:19.0.0-2976.g3f51ea39.el9.x86_64 35/36 2024-06-11T13:20:56.201 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-11T13:20:56.202 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-11T13:20:56.202 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-11T13:20:56.202 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-11T13:20:56.202 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:20:56.243 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install ceph-mgr-dashboard 2024-06-11T13:20:56.267 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-2:19.0.0-2976.g3f51ea39.el9.x86_64 36/36 2024-06-11T13:20:56.840 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:01:04 ago on Tue 11 Jun 2024 01:19:52 PM UTC. 2024-06-11T13:20:56.966 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-06-11T13:20:56.967 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:20:56.968 INFO:teuthology.orchestra.run.smithi057.stdout: Package Arch Version Repository Size 2024-06-11T13:20:56.968 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:20:56.968 INFO:teuthology.orchestra.run.smithi057.stdout:Installing: 2024-06-11T13:20:56.968 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2976.g3f51ea39.el9 ceph-noarch 3.5 M 2024-06-11T13:20:56.968 INFO:teuthology.orchestra.run.smithi057.stdout:Installing dependencies: 2024-06-11T13:20:56.968 INFO:teuthology.orchestra.run.smithi057.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-11T13:20:56.968 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2976.g3f51ea39.el9 ceph-noarch 32 k 2024-06-11T13:20:56.968 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2976.g3f51ea39.el9 ceph-noarch 16 k 2024-06-11T13:20:56.968 INFO:teuthology.orchestra.run.smithi057.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-11T13:20:56.968 INFO:teuthology.orchestra.run.smithi057.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-11T13:20:56.968 INFO:teuthology.orchestra.run.smithi057.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-11T13:20:56.969 INFO:teuthology.orchestra.run.smithi057.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-11T13:20:56.969 INFO:teuthology.orchestra.run.smithi057.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-11T13:20:56.969 INFO:teuthology.orchestra.run.smithi057.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-11T13:20:56.969 INFO:teuthology.orchestra.run.smithi057.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-11T13:20:56.969 INFO:teuthology.orchestra.run.smithi057.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-11T13:20:56.969 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:20:56.969 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction Summary 2024-06-11T13:20:56.969 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:20:56.969 INFO:teuthology.orchestra.run.smithi057.stdout:Install 12 Packages 2024-06-11T13:20:56.969 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:20:56.970 INFO:teuthology.orchestra.run.smithi057.stdout:Total download size: 8.6 M 2024-06-11T13:20:56.970 INFO:teuthology.orchestra.run.smithi057.stdout:Installed size: 103 M 2024-06-11T13:20:56.970 INFO:teuthology.orchestra.run.smithi057.stdout:Downloading Packages: 2024-06-11T13:20:57.434 INFO:teuthology.orchestra.run.smithi057.stdout:(1/12): ceph-grafana-dashboards-19.0.0-2976.g3f 157 kB/s | 32 kB 00:00 2024-06-11T13:20:57.460 INFO:teuthology.orchestra.run.smithi057.stdout:(2/12): ceph-prometheus-alerts-19.0.0-2976.g3f5 73 kB/s | 16 kB 00:00 2024-06-11T13:20:57.515 INFO:teuthology.orchestra.run.smithi057.stdout:[MIRROR] protobuf-3.14.0-13.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/protobuf-3.14.0-13.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-11T13:20:57.521 INFO:teuthology.orchestra.run.smithi057.stdout:[MIRROR] python3-protobuf-3.14.0-13.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-protobuf-3.14.0-13.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-11T13:20:57.693 INFO:teuthology.orchestra.run.smithi057.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.1 MB/s | 269 kB 00:00 2024-06-11T13:20:57.694 INFO:teuthology.orchestra.run.smithi057.stdout:[MIRROR] protobuf-compiler-3.14.0-13.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/protobuf-compiler-3.14.0-13.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-11T13:20:57.760 INFO:teuthology.orchestra.run.smithi057.stdout:(4/12): ceph-mgr-dashboard-19.0.0-2976.g3f51ea3 6.7 MB/s | 3.5 MB 00:00 2024-06-11T13:20:57.794 INFO:teuthology.orchestra.run.smithi057.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 16 MB/s | 548 kB 00:00 2024-06-11T13:20:57.819 INFO:teuthology.orchestra.run.smithi057.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 777 kB/s | 19 kB 00:00 2024-06-11T13:20:57.861 INFO:teuthology.orchestra.run.smithi057.stdout:(7/12): protobuf-3.14.0-13.el9.x86_64.rpm 2.4 MB/s | 1.0 MB 00:00 2024-06-11T13:20:57.895 INFO:teuthology.orchestra.run.smithi057.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 4.2 MB/s | 144 kB 00:00 2024-06-11T13:20:57.920 INFO:teuthology.orchestra.run.smithi057.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 1.2 MB/s | 31 kB 00:00 2024-06-11T13:20:57.962 INFO:teuthology.orchestra.run.smithi057.stdout:(10/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 14 MB/s | 2.0 MB 00:00 2024-06-11T13:20:57.996 INFO:teuthology.orchestra.run.smithi057.stdout:(11/12): protobuf-compiler-3.14.0-13.el9.x86_64 2.8 MB/s | 863 kB 00:00 2024-06-11T13:20:58.001 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-2:19.0.0-2976.g3f51ea39.el9.x86_64 36/36 2024-06-11T13:20:58.002 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-2:19.0.0-2976.g3f51ea39.el9.x86_64 1/36 2024-06-11T13:20:58.002 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mds-2:19.0.0-2976.g3f51ea39.el9.x86_64 2/36 2024-06-11T13:20:58.002 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-2:19.0.0-2976.g3f51ea39.el9.x86_64 3/36 2024-06-11T13:20:58.002 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mon-2:19.0.0-2976.g3f51ea39.el9.x86_64 4/36 2024-06-11T13:20:58.002 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-osd-2:19.0.0-2976.g3f51ea39.el9.x86_64 5/36 2024-06-11T13:20:58.002 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2976.g3f51ea39.el9. 6/36 2024-06-11T13:20:58.002 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/36 2024-06-11T13:20:58.002 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/36 2024-06-11T13:20:58.002 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/36 2024-06-11T13:20:58.002 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/36 2024-06-11T13:20:58.002 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/36 2024-06-11T13:20:58.002 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/36 2024-06-11T13:20:58.002 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/36 2024-06-11T13:20:58.003 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/36 2024-06-11T13:20:58.003 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/36 2024-06-11T13:20:58.003 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/36 2024-06-11T13:20:58.003 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/36 2024-06-11T13:20:58.003 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-06-11T13:20:58.003 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 19/36 2024-06-11T13:20:58.003 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-06-11T13:20:58.003 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-06-11T13:20:58.003 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-06-11T13:20:58.003 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-06-11T13:20:58.003 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-06-11T13:20:58.003 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-06-11T13:20:58.003 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-06-11T13:20:58.004 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-06-11T13:20:58.004 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-06-11T13:20:58.004 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-06-11T13:20:58.004 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-06-11T13:20:58.004 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-06-11T13:20:58.004 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-06-11T13:20:58.004 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-06-11T13:20:58.004 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-06-11T13:20:58.004 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-06-11T13:20:58.021 INFO:teuthology.orchestra.run.smithi057.stdout:(12/12): python3-routes-2.5.1-5.el9.noarch.rpm 1.8 MB/s | 188 kB 00:00 2024-06-11T13:20:58.024 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2024-06-11T13:20:58.025 INFO:teuthology.orchestra.run.smithi057.stdout:Total 8.2 MB/s | 8.6 MB 00:01 2024-06-11T13:20:58.074 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction check 2024-06-11T13:20:58.092 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction check succeeded. 2024-06-11T13:20:58.092 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction test 2024-06-11T13:20:58.295 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction test succeeded. 2024-06-11T13:20:58.295 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction 2024-06-11T13:20:58.843 INFO:teuthology.orchestra.run.smithi057.stdout: Preparing : 1/1 2024-06-11T13:20:58.931 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-06-11T13:20:58.966 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-06-11T13:20:59.014 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-06-11T13:20:59.043 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-06-11T13:20:59.147 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-06-11T13:20:59.170 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-06-11T13:20:59.170 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:20:59.170 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-06-11T13:20:59.170 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:20:59.170 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mds-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:20:59.170 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:20:59.170 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-modules-core-2:19.0.0-2976.g3f51ea39.el9.noarch 2024-06-11T13:20:59.170 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:20:59.170 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-osd-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:20:59.170 INFO:teuthology.orchestra.run.smithi107.stdout: lua-5.4.4-4.el9.x86_64 2024-06-11T13:20:59.170 INFO:teuthology.orchestra.run.smithi107.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-11T13:20:59.171 INFO:teuthology.orchestra.run.smithi107.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-11T13:20:59.171 INFO:teuthology.orchestra.run.smithi107.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-11T13:20:59.171 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-11T13:20:59.171 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-11T13:20:59.171 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-11T13:20:59.171 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-11T13:20:59.171 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-11T13:20:59.171 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-11T13:20:59.171 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-11T13:20:59.171 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-11T13:20:59.171 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-11T13:20:59.171 INFO:teuthology.orchestra.run.smithi107.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-11T13:20:59.171 INFO:teuthology.orchestra.run.smithi107.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-11T13:20:59.171 INFO:teuthology.orchestra.run.smithi107.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-11T13:20:59.172 INFO:teuthology.orchestra.run.smithi107.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-11T13:20:59.172 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-11T13:20:59.172 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ply-3.11-14.el9.noarch 2024-06-11T13:20:59.172 INFO:teuthology.orchestra.run.smithi107.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-11T13:20:59.172 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-11T13:20:59.172 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-11T13:20:59.172 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-11T13:20:59.172 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-11T13:20:59.172 INFO:teuthology.orchestra.run.smithi107.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-11T13:20:59.172 INFO:teuthology.orchestra.run.smithi107.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-11T13:20:59.172 INFO:teuthology.orchestra.run.smithi107.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-11T13:20:59.172 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-11T13:20:59.172 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-11T13:20:59.172 INFO:teuthology.orchestra.run.smithi107.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-11T13:20:59.173 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:20:59.173 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-06-11T13:20:59.285 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-06-11T13:20:59.383 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-06-11T13:20:59.432 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-06-11T13:20:59.460 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-06-11T13:20:59.491 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2976.g3f51ea39.el9 10/12 2024-06-11T13:20:59.497 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-base 2024-06-11T13:21:00.083 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:54 ago on Tue 11 Jun 2024 01:20:06 PM UTC. 2024-06-11T13:21:00.164 INFO:teuthology.orchestra.run.smithi107.stdout:Package ceph-base-2:19.0.0-2976.g3f51ea39.el9.x86_64 is already installed. 2024-06-11T13:21:00.206 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-06-11T13:21:00.207 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-06-11T13:21:00.207 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-06-11T13:21:00.257 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install cephadm 2024-06-11T13:21:00.595 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2976.g3f51ea39.el 11/12 2024-06-11T13:21:00.633 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2976.g3f51ea39.el9.noa 12/12 2024-06-11T13:21:00.851 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:54 ago on Tue 11 Jun 2024 01:20:06 PM UTC. 2024-06-11T13:21:00.982 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-06-11T13:21:00.983 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:21:00.983 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-06-11T13:21:00.983 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:21:00.984 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-06-11T13:21:00.984 INFO:teuthology.orchestra.run.smithi107.stdout: cephadm noarch 2:19.0.0-2976.g3f51ea39.el9 ceph-noarch 763 k 2024-06-11T13:21:00.984 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:00.984 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-06-11T13:21:00.984 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:21:00.984 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-06-11T13:21:00.984 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:00.984 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 763 k 2024-06-11T13:21:00.984 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 769 k 2024-06-11T13:21:00.984 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-06-11T13:21:01.417 INFO:teuthology.orchestra.run.smithi107.stdout:cephadm-19.0.0-2976.g3f51ea39.el9.noarch.rpm 1.7 MB/s | 763 kB 00:00 2024-06-11T13:21:01.418 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-06-11T13:21:01.418 INFO:teuthology.orchestra.run.smithi107.stdout:Total 1.7 MB/s | 763 kB 00:00 2024-06-11T13:21:01.418 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-06-11T13:21:01.424 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-06-11T13:21:01.424 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-06-11T13:21:01.432 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-06-11T13:21:01.433 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-06-11T13:21:01.520 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-06-11T13:21:01.911 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2976.g3f51ea39.el9.noa 12/12 2024-06-11T13:21:01.911 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2976.g3f51ea39.el 1/12 2024-06-11T13:21:01.911 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2976.g3f51ea39.el9.noa 2/12 2024-06-11T13:21:01.911 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2976.g3f51ea39.el9 3/12 2024-06-11T13:21:01.911 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-06-11T13:21:01.912 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-06-11T13:21:01.912 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-06-11T13:21:01.912 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-06-11T13:21:01.912 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-06-11T13:21:01.912 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-06-11T13:21:01.912 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-06-11T13:21:01.912 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-06-11T13:21:01.952 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: cephadm-2:19.0.0-2976.g3f51ea39.el9.noarch 1/1 2024-06-11T13:21:02.030 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephadm-2:19.0.0-2976.g3f51ea39.el9.noarch 1/1 2024-06-11T13:21:02.237 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-06-11T13:21:02.237 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:21:02.237 INFO:teuthology.orchestra.run.smithi057.stdout:Installed: 2024-06-11T13:21:02.237 INFO:teuthology.orchestra.run.smithi057.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-11T13:21:02.237 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-grafana-dashboards-2:19.0.0-2976.g3f51ea39.el9.noarch 2024-06-11T13:21:02.237 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-dashboard-2:19.0.0-2976.g3f51ea39.el9.noarch 2024-06-11T13:21:02.237 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-prometheus-alerts-2:19.0.0-2976.g3f51ea39.el9.noarch 2024-06-11T13:21:02.238 INFO:teuthology.orchestra.run.smithi057.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-11T13:21:02.238 INFO:teuthology.orchestra.run.smithi057.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-11T13:21:02.238 INFO:teuthology.orchestra.run.smithi057.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-11T13:21:02.238 INFO:teuthology.orchestra.run.smithi057.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-11T13:21:02.238 INFO:teuthology.orchestra.run.smithi057.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-11T13:21:02.238 INFO:teuthology.orchestra.run.smithi057.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-11T13:21:02.238 INFO:teuthology.orchestra.run.smithi057.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-11T13:21:02.238 INFO:teuthology.orchestra.run.smithi057.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-11T13:21:02.238 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:21:02.239 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-06-11T13:21:02.374 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: cephadm-2:19.0.0-2976.g3f51ea39.el9.noarch 1/1 2024-06-11T13:21:02.497 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install ceph-mgr-diskprediction-local 2024-06-11T13:21:02.630 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephadm-2:19.0.0-2976.g3f51ea39.el9.noarch 1/1 2024-06-11T13:21:02.630 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:02.630 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-06-11T13:21:02.630 INFO:teuthology.orchestra.run.smithi107.stdout: cephadm-2:19.0.0-2976.g3f51ea39.el9.noarch 2024-06-11T13:21:02.630 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:02.630 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-06-11T13:21:02.828 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-immutable-object-cache 2024-06-11T13:21:03.096 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:01:11 ago on Tue 11 Jun 2024 01:19:52 PM UTC. 2024-06-11T13:21:03.225 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-06-11T13:21:03.226 INFO:teuthology.orchestra.run.smithi057.stdout:======================================================================================= 2024-06-11T13:21:03.226 INFO:teuthology.orchestra.run.smithi057.stdout: Package Arch Version Repository Size 2024-06-11T13:21:03.226 INFO:teuthology.orchestra.run.smithi057.stdout:======================================================================================= 2024-06-11T13:21:03.226 INFO:teuthology.orchestra.run.smithi057.stdout:Installing: 2024-06-11T13:21:03.226 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2976.g3f51ea39.el9 ceph-noarch 7.4 M 2024-06-11T13:21:03.226 INFO:teuthology.orchestra.run.smithi057.stdout:Installing dependencies: 2024-06-11T13:21:03.226 INFO:teuthology.orchestra.run.smithi057.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-11T13:21:03.226 INFO:teuthology.orchestra.run.smithi057.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-11T13:21:03.226 INFO:teuthology.orchestra.run.smithi057.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-11T13:21:03.226 INFO:teuthology.orchestra.run.smithi057.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-11T13:21:03.226 INFO:teuthology.orchestra.run.smithi057.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-11T13:21:03.226 INFO:teuthology.orchestra.run.smithi057.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-11T13:21:03.227 INFO:teuthology.orchestra.run.smithi057.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-11T13:21:03.227 INFO:teuthology.orchestra.run.smithi057.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-11T13:21:03.227 INFO:teuthology.orchestra.run.smithi057.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-11T13:21:03.227 INFO:teuthology.orchestra.run.smithi057.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-11T13:21:03.227 INFO:teuthology.orchestra.run.smithi057.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-11T13:21:03.227 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:21:03.227 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction Summary 2024-06-11T13:21:03.227 INFO:teuthology.orchestra.run.smithi057.stdout:======================================================================================= 2024-06-11T13:21:03.227 INFO:teuthology.orchestra.run.smithi057.stdout:Install 12 Packages 2024-06-11T13:21:03.227 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:21:03.228 INFO:teuthology.orchestra.run.smithi057.stdout:Total download size: 38 M 2024-06-11T13:21:03.228 INFO:teuthology.orchestra.run.smithi057.stdout:Installed size: 209 M 2024-06-11T13:21:03.228 INFO:teuthology.orchestra.run.smithi057.stdout:Downloading Packages: 2024-06-11T13:21:03.412 INFO:teuthology.orchestra.run.smithi057.stdout:[MIRROR] libgfortran-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-11T13:21:03.413 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:57 ago on Tue 11 Jun 2024 01:20:06 PM UTC. 2024-06-11T13:21:03.413 INFO:teuthology.orchestra.run.smithi057.stdout:[MIRROR] libquadmath-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-11T13:21:03.538 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-06-11T13:21:03.539 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:21:03.539 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repo Size 2024-06-11T13:21:03.539 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:21:03.539 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-06-11T13:21:03.539 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 145 k 2024-06-11T13:21:03.539 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:03.539 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-06-11T13:21:03.539 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:21:03.539 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-06-11T13:21:03.539 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:03.540 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 145 k 2024-06-11T13:21:03.540 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 435 k 2024-06-11T13:21:03.540 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-06-11T13:21:03.765 INFO:teuthology.orchestra.run.smithi107.stdout:ceph-immutable-object-cache-19.0.0-2976.g3f51ea 644 kB/s | 145 kB 00:00 2024-06-11T13:21:03.766 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-06-11T13:21:03.766 INFO:teuthology.orchestra.run.smithi107.stdout:Total 640 kB/s | 145 kB 00:00 2024-06-11T13:21:03.766 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-06-11T13:21:03.773 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-06-11T13:21:03.773 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-06-11T13:21:03.830 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-06-11T13:21:03.830 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-06-11T13:21:03.969 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-06-11T13:21:04.041 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2976.g3f51ea39. 1/1 2024-06-11T13:21:04.069 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2976.g3f51ea39. 1/1 2024-06-11T13:21:04.070 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-11T13:21:04.070 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-11T13:21:04.070 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:04.081 INFO:teuthology.orchestra.run.smithi057.stdout:(1/12): ceph-mgr-diskprediction-local-19.0.0-29 9.9 MB/s | 7.4 MB 00:00 2024-06-11T13:21:04.106 INFO:teuthology.orchestra.run.smithi057.stdout:(2/12): libquadmath-11.4.1-3.el9.x86_64.rpm 252 kB/s | 195 kB 00:00 2024-06-11T13:21:04.107 INFO:teuthology.orchestra.run.smithi057.stdout:[MIRROR] flexiblas-3.0.4-8.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/flexiblas-3.0.4-8.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-11T13:21:04.107 INFO:teuthology.orchestra.run.smithi057.stdout:[MIRROR] flexiblas-netlib-3.0.4-8.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/flexiblas-netlib-3.0.4-8.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-11T13:21:04.265 INFO:teuthology.orchestra.run.smithi057.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 178 kB/s | 33 kB 00:00 2024-06-11T13:21:04.266 INFO:teuthology.orchestra.run.smithi057.stdout:[MIRROR] flexiblas-openblas-openmp-3.0.4-8.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/flexiblas-openblas-openmp-3.0.4-8.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-11T13:21:04.349 INFO:teuthology.orchestra.run.smithi057.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 12 MB/s | 3.0 MB 00:00 2024-06-11T13:21:04.374 INFO:teuthology.orchestra.run.smithi057.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 167 kB/s | 18 kB 00:00 2024-06-11T13:21:04.482 INFO:teuthology.orchestra.run.smithi057.stdout:(6/12): libgfortran-11.4.1-3.el9.x86_64.rpm 697 kB/s | 803 kB 00:01 2024-06-11T13:21:04.641 INFO:teuthology.orchestra.run.smithi057.stdout:(7/12): openblas-0.3.21-2.el9.x86_64.rpm 121 kB/s | 35 kB 00:00 2024-06-11T13:21:04.716 INFO:teuthology.orchestra.run.smithi057.stdout:(8/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 14 MB/s | 4.7 MB 00:00 2024-06-11T13:21:04.742 INFO:teuthology.orchestra.run.smithi057.stdout:(9/12): python3-devel-3.9.18-3.el9.x86_64.rpm 943 kB/s | 244 kB 00:00 2024-06-11T13:21:04.767 INFO:teuthology.orchestra.run.smithi057.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 4.9 MB/s | 250 kB 00:00 2024-06-11T13:21:04.984 INFO:teuthology.orchestra.run.smithi057.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 15 MB/s | 5.1 MB 00:00 2024-06-11T13:21:05.005 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2976.g3f51ea39. 1/1 2024-06-11T13:21:05.006 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:05.006 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-06-11T13:21:05.006 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-immutable-object-cache-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:21:05.006 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:05.006 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-06-11T13:21:05.233 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-mgr 2024-06-11T13:21:05.451 INFO:teuthology.orchestra.run.smithi057.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-06-11T13:21:05.453 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2024-06-11T13:21:05.454 INFO:teuthology.orchestra.run.smithi057.stdout:Total 17 MB/s | 38 MB 00:02 2024-06-11T13:21:05.820 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:59 ago on Tue 11 Jun 2024 01:20:06 PM UTC. 2024-06-11T13:21:05.823 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction check 2024-06-11T13:21:05.852 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction check succeeded. 2024-06-11T13:21:05.852 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction test 2024-06-11T13:21:05.905 INFO:teuthology.orchestra.run.smithi107.stdout:Package ceph-mgr-2:19.0.0-2976.g3f51ea39.el9.x86_64 is already installed. 2024-06-11T13:21:05.947 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-06-11T13:21:05.948 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-06-11T13:21:05.948 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-06-11T13:21:06.016 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-mgr-dashboard 2024-06-11T13:21:06.298 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction test succeeded. 2024-06-11T13:21:06.298 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction 2024-06-11T13:21:06.580 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:00 ago on Tue 11 Jun 2024 01:20:06 PM UTC. 2024-06-11T13:21:06.700 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-06-11T13:21:06.701 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:21:06.702 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-06-11T13:21:06.702 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:21:06.702 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-06-11T13:21:06.702 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2976.g3f51ea39.el9 ceph-noarch 3.5 M 2024-06-11T13:21:06.702 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-06-11T13:21:06.702 INFO:teuthology.orchestra.run.smithi107.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-11T13:21:06.702 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2976.g3f51ea39.el9 ceph-noarch 32 k 2024-06-11T13:21:06.702 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2976.g3f51ea39.el9 ceph-noarch 16 k 2024-06-11T13:21:06.702 INFO:teuthology.orchestra.run.smithi107.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-11T13:21:06.703 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-11T13:21:06.703 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-11T13:21:06.703 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-11T13:21:06.703 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-11T13:21:06.703 INFO:teuthology.orchestra.run.smithi107.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-11T13:21:06.703 INFO:teuthology.orchestra.run.smithi107.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-11T13:21:06.703 INFO:teuthology.orchestra.run.smithi107.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-11T13:21:06.703 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:06.703 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-06-11T13:21:06.703 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:21:06.703 INFO:teuthology.orchestra.run.smithi107.stdout:Install 12 Packages 2024-06-11T13:21:06.703 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:06.704 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 8.6 M 2024-06-11T13:21:06.704 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 103 M 2024-06-11T13:21:06.704 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-06-11T13:21:07.103 INFO:teuthology.orchestra.run.smithi057.stdout: Preparing : 1/1 2024-06-11T13:21:07.195 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-06-11T13:21:07.233 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-06-11T13:21:07.337 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-06-11T13:21:07.346 INFO:teuthology.orchestra.run.smithi107.stdout:(1/12): ceph-grafana-dashboards-19.0.0-2976.g3f 123 kB/s | 32 kB 00:00 2024-06-11T13:21:07.376 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-06-11T13:21:07.393 INFO:teuthology.orchestra.run.smithi107.stdout:(2/12): ceph-prometheus-alerts-19.0.0-2976.g3f5 53 kB/s | 16 kB 00:00 2024-06-11T13:21:07.396 INFO:teuthology.orchestra.run.smithi107.stdout:[MIRROR] protobuf-3.14.0-13.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/protobuf-3.14.0-13.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-11T13:21:07.429 INFO:teuthology.orchestra.run.smithi107.stdout:[MIRROR] python3-protobuf-3.14.0-13.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-protobuf-3.14.0-13.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-11T13:21:07.560 INFO:teuthology.orchestra.run.smithi107.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.6 MB/s | 269 kB 00:00 2024-06-11T13:21:07.561 INFO:teuthology.orchestra.run.smithi107.stdout:[MIRROR] protobuf-compiler-3.14.0-13.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/protobuf-compiler-3.14.0-13.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-11T13:21:07.561 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-06-11T13:21:07.591 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-06-11T13:21:07.602 INFO:teuthology.orchestra.run.smithi107.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 3.9 MB/s | 1.0 MB 00:00 2024-06-11T13:21:07.636 INFO:teuthology.orchestra.run.smithi107.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 16 MB/s | 548 kB 00:00 2024-06-11T13:21:07.661 INFO:teuthology.orchestra.run.smithi107.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 779 kB/s | 19 kB 00:00 2024-06-11T13:21:07.679 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-06-11T13:21:07.745 INFO:teuthology.orchestra.run.smithi107.stdout:(7/12): ceph-mgr-dashboard-19.0.0-2976.g3f51ea3 5.3 MB/s | 3.5 MB 00:00 2024-06-11T13:21:07.778 INFO:teuthology.orchestra.run.smithi107.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 4.2 MB/s | 144 kB 00:00 2024-06-11T13:21:07.795 INFO:teuthology.orchestra.run.smithi107.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 2.0 MB/s | 31 kB 00:00 2024-06-11T13:21:07.837 INFO:teuthology.orchestra.run.smithi107.stdout:(10/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 13 MB/s | 2.0 MB 00:00 2024-06-11T13:21:07.862 INFO:teuthology.orchestra.run.smithi107.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 2.7 MB/s | 188 kB 00:00 2024-06-11T13:21:08.371 INFO:teuthology.orchestra.run.smithi107.stdout:(12/12): protobuf-compiler-3.14.0-13.el9.x86_64 1.0 MB/s | 863 kB 00:00 2024-06-11T13:21:08.371 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-06-11T13:21:08.375 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-06-11T13:21:08.376 INFO:teuthology.orchestra.run.smithi107.stdout:Total 5.2 MB/s | 8.6 MB 00:01 2024-06-11T13:21:08.425 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-06-11T13:21:08.442 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-06-11T13:21:08.442 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-06-11T13:21:08.484 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-06-11T13:21:08.622 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-06-11T13:21:08.623 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-06-11T13:21:09.147 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-06-11T13:21:09.230 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-06-11T13:21:09.265 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-06-11T13:21:09.311 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-06-11T13:21:09.342 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-06-11T13:21:09.453 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-06-11T13:21:09.576 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-06-11T13:21:09.661 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-06-11T13:21:09.729 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-06-11T13:21:09.758 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-06-11T13:21:09.789 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2976.g3f51ea39.el9 10/12 2024-06-11T13:21:09.816 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-06-11T13:21:10.823 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2976.g3f51ea39.el 11/12 2024-06-11T13:21:10.849 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2976.g3f51ea39.el9.noa 12/12 2024-06-11T13:21:10.902 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-06-11T13:21:10.934 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2976.g3f51e 12/12 2024-06-11T13:21:12.111 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2976.g3f51ea39.el9.noa 12/12 2024-06-11T13:21:12.111 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2976.g3f51ea39.el 1/12 2024-06-11T13:21:12.111 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2976.g3f51ea39.el9.noa 2/12 2024-06-11T13:21:12.111 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2976.g3f51ea39.el9 3/12 2024-06-11T13:21:12.111 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-06-11T13:21:12.112 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-06-11T13:21:12.112 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-06-11T13:21:12.112 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-06-11T13:21:12.112 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-06-11T13:21:12.112 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-06-11T13:21:12.112 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-06-11T13:21:12.112 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-06-11T13:21:12.411 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-06-11T13:21:12.411 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:12.412 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-06-11T13:21:12.412 INFO:teuthology.orchestra.run.smithi107.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-11T13:21:12.412 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-grafana-dashboards-2:19.0.0-2976.g3f51ea39.el9.noarch 2024-06-11T13:21:12.412 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-dashboard-2:19.0.0-2976.g3f51ea39.el9.noarch 2024-06-11T13:21:12.412 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-prometheus-alerts-2:19.0.0-2976.g3f51ea39.el9.noarch 2024-06-11T13:21:12.412 INFO:teuthology.orchestra.run.smithi107.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-11T13:21:12.412 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-11T13:21:12.412 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-11T13:21:12.413 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-11T13:21:12.413 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-11T13:21:12.413 INFO:teuthology.orchestra.run.smithi107.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-11T13:21:12.413 INFO:teuthology.orchestra.run.smithi107.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-11T13:21:12.413 INFO:teuthology.orchestra.run.smithi107.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-11T13:21:12.413 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:12.413 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-06-11T13:21:12.642 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-mgr-diskprediction-local 2024-06-11T13:21:13.221 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:07 ago on Tue 11 Jun 2024 01:20:06 PM UTC. 2024-06-11T13:21:13.345 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-06-11T13:21:13.346 INFO:teuthology.orchestra.run.smithi107.stdout:======================================================================================= 2024-06-11T13:21:13.346 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-06-11T13:21:13.346 INFO:teuthology.orchestra.run.smithi107.stdout:======================================================================================= 2024-06-11T13:21:13.347 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-06-11T13:21:13.347 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2976.g3f51ea39.el9 ceph-noarch 7.4 M 2024-06-11T13:21:13.347 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-06-11T13:21:13.347 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-11T13:21:13.347 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-11T13:21:13.347 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-11T13:21:13.347 INFO:teuthology.orchestra.run.smithi107.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-11T13:21:13.347 INFO:teuthology.orchestra.run.smithi107.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-11T13:21:13.347 INFO:teuthology.orchestra.run.smithi107.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-11T13:21:13.347 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-11T13:21:13.347 INFO:teuthology.orchestra.run.smithi107.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-11T13:21:13.348 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-11T13:21:13.348 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-11T13:21:13.348 INFO:teuthology.orchestra.run.smithi107.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-11T13:21:13.348 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:13.348 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-06-11T13:21:13.348 INFO:teuthology.orchestra.run.smithi107.stdout:======================================================================================= 2024-06-11T13:21:13.348 INFO:teuthology.orchestra.run.smithi107.stdout:Install 12 Packages 2024-06-11T13:21:13.348 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:13.349 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 38 M 2024-06-11T13:21:13.349 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 209 M 2024-06-11T13:21:13.349 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-06-11T13:21:13.434 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2976.g3f51e 12/12 2024-06-11T13:21:13.434 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2976.g3f51e 1/12 2024-06-11T13:21:13.434 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-06-11T13:21:13.434 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-06-11T13:21:13.434 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-06-11T13:21:13.434 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-06-11T13:21:13.434 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-06-11T13:21:13.435 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-06-11T13:21:13.435 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-06-11T13:21:13.435 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-06-11T13:21:13.435 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-06-11T13:21:13.435 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-06-11T13:21:13.588 INFO:teuthology.orchestra.run.smithi107.stdout:[MIRROR] libgfortran-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-11T13:21:13.589 INFO:teuthology.orchestra.run.smithi107.stdout:[MIRROR] libquadmath-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-11T13:21:13.761 INFO:teuthology.orchestra.run.smithi107.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 782 kB/s | 195 kB 00:00 2024-06-11T13:21:13.762 INFO:teuthology.orchestra.run.smithi107.stdout:[MIRROR] flexiblas-3.0.4-8.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/flexiblas-3.0.4-8.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-11T13:21:13.819 INFO:teuthology.orchestra.run.smithi107.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 803 kB 00:00 2024-06-11T13:21:13.820 INFO:teuthology.orchestra.run.smithi107.stdout:[MIRROR] flexiblas-netlib-3.0.4-8.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/flexiblas-netlib-3.0.4-8.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-11T13:21:13.829 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-06-11T13:21:13.829 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:21:13.829 INFO:teuthology.orchestra.run.smithi057.stdout:Installed: 2024-06-11T13:21:13.829 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2976.g3f51ea39.el9.noarch 2024-06-11T13:21:13.829 INFO:teuthology.orchestra.run.smithi057.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-11T13:21:13.830 INFO:teuthology.orchestra.run.smithi057.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-11T13:21:13.830 INFO:teuthology.orchestra.run.smithi057.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-11T13:21:13.830 INFO:teuthology.orchestra.run.smithi057.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-11T13:21:13.830 INFO:teuthology.orchestra.run.smithi057.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-11T13:21:13.830 INFO:teuthology.orchestra.run.smithi057.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-11T13:21:13.830 INFO:teuthology.orchestra.run.smithi057.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-11T13:21:13.830 INFO:teuthology.orchestra.run.smithi057.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-11T13:21:13.830 INFO:teuthology.orchestra.run.smithi057.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-11T13:21:13.830 INFO:teuthology.orchestra.run.smithi057.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-11T13:21:13.830 INFO:teuthology.orchestra.run.smithi057.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-11T13:21:13.830 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:21:13.830 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-06-11T13:21:13.911 INFO:teuthology.orchestra.run.smithi107.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 217 kB/s | 33 kB 00:00 2024-06-11T13:21:13.912 INFO:teuthology.orchestra.run.smithi107.stdout:[MIRROR] flexiblas-openblas-openmp-3.0.4-8.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/flexiblas-openblas-openmp-3.0.4-8.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-11T13:21:13.962 INFO:teuthology.orchestra.run.smithi107.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 365 kB/s | 18 kB 00:00 2024-06-11T13:21:14.020 INFO:teuthology.orchestra.run.smithi107.stdout:(5/12): openblas-0.3.21-2.el9.x86_64.rpm 604 kB/s | 35 kB 00:00 2024-06-11T13:21:14.238 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install ceph-mgr-rook 2024-06-11T13:21:14.345 INFO:teuthology.orchestra.run.smithi107.stdout:(6/12): ceph-mgr-diskprediction-local-19.0.0-29 8.9 MB/s | 7.4 MB 00:00 2024-06-11T13:21:14.615 INFO:teuthology.orchestra.run.smithi107.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 916 kB/s | 244 kB 00:00 2024-06-11T13:21:14.822 INFO:teuthology.orchestra.run.smithi107.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 3.0 MB/s | 3.0 MB 00:01 2024-06-11T13:21:14.843 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:01:22 ago on Tue 11 Jun 2024 01:19:52 PM UTC. 2024-06-11T13:21:14.973 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-06-11T13:21:14.974 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:21:14.974 INFO:teuthology.orchestra.run.smithi057.stdout: Package Arch Version Repository Size 2024-06-11T13:21:14.975 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:21:14.975 INFO:teuthology.orchestra.run.smithi057.stdout:Installing: 2024-06-11T13:21:14.975 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-rook noarch 2:19.0.0-2976.g3f51ea39.el9 ceph-noarch 49 k 2024-06-11T13:21:14.975 INFO:teuthology.orchestra.run.smithi057.stdout:Installing dependencies: 2024-06-11T13:21:14.975 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-11T13:21:14.975 INFO:teuthology.orchestra.run.smithi057.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-11T13:21:14.975 INFO:teuthology.orchestra.run.smithi057.stdout: python3-google-auth noarch 1:2.29.0-2.el9 epel 205 k 2024-06-11T13:21:14.975 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-11T13:21:14.976 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-11T13:21:14.976 INFO:teuthology.orchestra.run.smithi057.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-11T13:21:14.976 INFO:teuthology.orchestra.run.smithi057.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-11T13:21:14.976 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-11T13:21:14.976 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-11T13:21:14.976 INFO:teuthology.orchestra.run.smithi057.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-11T13:21:14.976 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-11T13:21:14.976 INFO:teuthology.orchestra.run.smithi057.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-11T13:21:14.976 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:21:14.976 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction Summary 2024-06-11T13:21:14.976 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:21:14.976 INFO:teuthology.orchestra.run.smithi057.stdout:Install 13 Packages 2024-06-11T13:21:14.976 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:21:14.977 INFO:teuthology.orchestra.run.smithi057.stdout:Total download size: 2.2 M 2024-06-11T13:21:14.977 INFO:teuthology.orchestra.run.smithi057.stdout:Installed size: 26 M 2024-06-11T13:21:14.977 INFO:teuthology.orchestra.run.smithi057.stdout:Downloading Packages: 2024-06-11T13:21:15.013 INFO:teuthology.orchestra.run.smithi107.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 1.3 MB/s | 250 kB 00:00 2024-06-11T13:21:15.548 INFO:teuthology.orchestra.run.smithi057.stdout:[MIRROR] python3-jsonpatch-1.21-16.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-jsonpatch-1.21-16.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-11T13:21:15.550 INFO:teuthology.orchestra.run.smithi057.stdout:[MIRROR] python3-jsonpointer-2.0-4.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-jsonpointer-2.0-4.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-11T13:21:15.676 INFO:teuthology.orchestra.run.smithi057.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 122 kB/s | 26 kB 00:00 2024-06-11T13:21:15.701 INFO:teuthology.orchestra.run.smithi057.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 81 kB/s | 19 kB 00:00 2024-06-11T13:21:15.702 INFO:teuthology.orchestra.run.smithi057.stdout:[MIRROR] python3-oauthlib-3.1.1-5.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-oauthlib-3.1.1-5.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-11T13:21:15.702 INFO:teuthology.orchestra.run.smithi057.stdout:[MIRROR] python3-pyasn1-0.4.8-6.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-pyasn1-0.4.8-6.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-11T13:21:15.759 INFO:teuthology.orchestra.run.smithi057.stdout:(3/13): ceph-mgr-rook-19.0.0-2976.g3f51ea39.el9 168 kB/s | 49 kB 00:00 2024-06-11T13:21:15.796 INFO:teuthology.orchestra.run.smithi107.stdout:(10/12): openblas-openmp-0.3.21-2.el9.x86_64.rp 2.7 MB/s | 4.7 MB 00:01 2024-06-11T13:21:15.851 INFO:teuthology.orchestra.run.smithi057.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.0 MB/s | 159 kB 00:00 2024-06-11T13:21:15.876 INFO:teuthology.orchestra.run.smithi057.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.2 MB/s | 222 kB 00:00 2024-06-11T13:21:15.902 INFO:teuthology.orchestra.run.smithi057.stdout:(6/13): python3-cachetools-4.2.4-1.el9.noarch.r 1.3 MB/s | 32 kB 00:00 2024-06-11T13:21:15.927 INFO:teuthology.orchestra.run.smithi057.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 710 kB/s | 54 kB 00:00 2024-06-11T13:21:15.953 INFO:teuthology.orchestra.run.smithi057.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 297 kB/s | 15 kB 00:00 2024-06-11T13:21:15.978 INFO:teuthology.orchestra.run.smithi057.stdout:(9/13): python3-google-auth-2.29.0-2.el9.noarch 4.0 MB/s | 205 kB 00:00 2024-06-11T13:21:16.003 INFO:teuthology.orchestra.run.smithi057.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-06-11T13:21:16.028 INFO:teuthology.orchestra.run.smithi057.stdout:(11/13): python3-pyasn1-modules-0.4.8-6.el9.noa 1.0 MB/s | 279 kB 00:00 2024-06-11T13:21:16.053 INFO:teuthology.orchestra.run.smithi057.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.8 MB/s | 90 kB 00:00 2024-06-11T13:21:16.087 INFO:teuthology.orchestra.run.smithi057.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.6 MB/s | 1.0 MB 00:00 2024-06-11T13:21:16.089 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2024-06-11T13:21:16.089 INFO:teuthology.orchestra.run.smithi057.stdout:Total 2.0 MB/s | 2.2 MB 00:01 2024-06-11T13:21:16.155 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction check 2024-06-11T13:21:16.171 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction check succeeded. 2024-06-11T13:21:16.171 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction test 2024-06-11T13:21:16.265 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction test succeeded. 2024-06-11T13:21:16.265 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction 2024-06-11T13:21:16.763 INFO:teuthology.orchestra.run.smithi107.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.4 MB/s | 5.1 MB 00:02 2024-06-11T13:21:16.779 INFO:teuthology.orchestra.run.smithi057.stdout: Preparing : 1/1 2024-06-11T13:21:16.962 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-06-11T13:21:17.023 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-06-11T13:21:17.079 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-06-11T13:21:17.115 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-06-11T13:21:17.157 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-06-11T13:21:17.262 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-06-11T13:21:17.372 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-google-auth-1:2.29.0-2.el9.noarch 7/13 2024-06-11T13:21:17.425 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-06-11T13:21:18.188 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-06-11T13:21:18.246 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-06-11T13:21:18.280 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-06-11T13:21:18.329 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-06-11T13:21:18.357 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mgr-rook-2:19.0.0-2976.g3f51ea39.el9.noarch 13/13 2024-06-11T13:21:19.156 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2976.g3f51ea39.el9.noarch 13/13 2024-06-11T13:21:19.156 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2976.g3f51ea39.el9.noarch 1/13 2024-06-11T13:21:19.156 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-06-11T13:21:19.156 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-06-11T13:21:19.156 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-06-11T13:21:19.156 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-06-11T13:21:19.156 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-06-11T13:21:19.156 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-06-11T13:21:19.157 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-06-11T13:21:19.157 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-06-11T13:21:19.157 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-google-auth-1:2.29.0-2.el9.noarch 10/13 2024-06-11T13:21:19.157 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-06-11T13:21:19.157 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-06-11T13:21:19.501 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-06-11T13:21:19.501 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:21:19.501 INFO:teuthology.orchestra.run.smithi057.stdout:Installed: 2024-06-11T13:21:19.501 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-rook-2:19.0.0-2976.g3f51ea39.el9.noarch 2024-06-11T13:21:19.501 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-11T13:21:19.501 INFO:teuthology.orchestra.run.smithi057.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-11T13:21:19.501 INFO:teuthology.orchestra.run.smithi057.stdout: python3-google-auth-1:2.29.0-2.el9.noarch 2024-06-11T13:21:19.501 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-11T13:21:19.501 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-11T13:21:19.502 INFO:teuthology.orchestra.run.smithi057.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-11T13:21:19.502 INFO:teuthology.orchestra.run.smithi057.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-11T13:21:19.502 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-11T13:21:19.502 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-11T13:21:19.502 INFO:teuthology.orchestra.run.smithi057.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-11T13:21:19.502 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-11T13:21:19.502 INFO:teuthology.orchestra.run.smithi057.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-11T13:21:19.502 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:21:19.502 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-06-11T13:21:19.768 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install ceph-mgr-cephadm 2024-06-11T13:21:20.370 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:01:28 ago on Tue 11 Jun 2024 01:19:52 PM UTC. 2024-06-11T13:21:20.498 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-06-11T13:21:20.499 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:21:20.499 INFO:teuthology.orchestra.run.smithi057.stdout: Package Arch Version Repository Size 2024-06-11T13:21:20.499 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:21:20.499 INFO:teuthology.orchestra.run.smithi057.stdout:Installing: 2024-06-11T13:21:20.499 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2976.g3f51ea39.el9 ceph-noarch 142 k 2024-06-11T13:21:20.499 INFO:teuthology.orchestra.run.smithi057.stdout:Installing dependencies: 2024-06-11T13:21:20.499 INFO:teuthology.orchestra.run.smithi057.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-11T13:21:20.500 INFO:teuthology.orchestra.run.smithi057.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-11T13:21:20.500 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-11T13:21:20.500 INFO:teuthology.orchestra.run.smithi057.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-11T13:21:20.500 INFO:teuthology.orchestra.run.smithi057.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-11T13:21:20.500 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:21:20.500 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction Summary 2024-06-11T13:21:20.500 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:21:20.500 INFO:teuthology.orchestra.run.smithi057.stdout:Install 6 Packages 2024-06-11T13:21:20.500 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:21:20.501 INFO:teuthology.orchestra.run.smithi057.stdout:Total download size: 7.0 M 2024-06-11T13:21:20.501 INFO:teuthology.orchestra.run.smithi057.stdout:Installed size: 33 M 2024-06-11T13:21:20.501 INFO:teuthology.orchestra.run.smithi057.stdout:Downloading Packages: 2024-06-11T13:21:20.813 INFO:teuthology.orchestra.run.smithi057.stdout:[MIRROR] python3-babel-2.9.1-2.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-babel-2.9.1-2.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-11T13:21:20.819 INFO:teuthology.orchestra.run.smithi057.stdout:[MIRROR] python3-jinja2-2.11.3-6.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-jinja2-2.11.3-6.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-11T13:21:21.264 INFO:teuthology.orchestra.run.smithi057.stdout:(1/6): ceph-mgr-cephadm-19.0.0-2976.g3f51ea39.e 281 kB/s | 142 kB 00:00 2024-06-11T13:21:21.297 INFO:teuthology.orchestra.run.smithi057.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 16 MB/s | 548 kB 00:00 2024-06-11T13:21:21.314 INFO:teuthology.orchestra.run.smithi057.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 3.4 MB/s | 58 kB 00:00 2024-06-11T13:21:21.331 INFO:teuthology.orchestra.run.smithi057.stdout:(4/6): python3-typing-extensions-4.4.0-2.el9.no 3.0 MB/s | 51 kB 00:00 2024-06-11T13:21:21.456 INFO:teuthology.orchestra.run.smithi057.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 355 kB/s | 248 kB 00:00 2024-06-11T13:21:22.021 INFO:teuthology.orchestra.run.smithi107.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 2.3 MB/s | 16 MB 00:07 2024-06-11T13:21:22.023 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-06-11T13:21:22.023 INFO:teuthology.orchestra.run.smithi107.stdout:Total 4.4 MB/s | 38 MB 00:08 2024-06-11T13:21:22.380 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-06-11T13:21:22.407 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-06-11T13:21:22.407 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-06-11T13:21:22.837 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-06-11T13:21:22.837 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-06-11T13:21:23.431 INFO:teuthology.orchestra.run.smithi057.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 2.2 MB/s | 6.0 MB 00:02 2024-06-11T13:21:23.434 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2024-06-11T13:21:23.434 INFO:teuthology.orchestra.run.smithi057.stdout:Total 2.4 MB/s | 7.0 MB 00:02 2024-06-11T13:21:23.514 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction check 2024-06-11T13:21:23.523 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction check succeeded. 2024-06-11T13:21:23.524 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction test 2024-06-11T13:21:23.587 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-06-11T13:21:23.629 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction test succeeded. 2024-06-11T13:21:23.630 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction 2024-06-11T13:21:23.662 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-06-11T13:21:23.710 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-06-11T13:21:23.809 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-06-11T13:21:23.852 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-06-11T13:21:23.938 INFO:teuthology.orchestra.run.smithi057.stdout: Preparing : 1/1 2024-06-11T13:21:24.030 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-06-11T13:21:24.060 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-06-11T13:21:24.106 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-06-11T13:21:24.141 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-06-11T13:21:24.168 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-06-11T13:21:24.719 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-06-11T13:21:24.788 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-06-11T13:21:24.806 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-06-11T13:21:24.877 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-06-11T13:21:24.903 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2976.g3f51ea39.el9.noarch 6/6 2024-06-11T13:21:24.905 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-06-11T13:21:25.666 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2976.g3f51ea39.el9.noarch 6/6 2024-06-11T13:21:25.666 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2976.g3f51ea39.el9.noarch 1/6 2024-06-11T13:21:25.666 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-06-11T13:21:25.666 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-06-11T13:21:25.666 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-06-11T13:21:25.666 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-06-11T13:21:25.941 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-06-11T13:21:25.941 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:21:25.941 INFO:teuthology.orchestra.run.smithi057.stdout:Installed: 2024-06-11T13:21:25.941 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-cephadm-2:19.0.0-2976.g3f51ea39.el9.noarch 2024-06-11T13:21:25.941 INFO:teuthology.orchestra.run.smithi057.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-11T13:21:25.941 INFO:teuthology.orchestra.run.smithi057.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-11T13:21:25.941 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-11T13:21:25.941 INFO:teuthology.orchestra.run.smithi057.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-11T13:21:25.941 INFO:teuthology.orchestra.run.smithi057.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-11T13:21:25.941 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:21:25.941 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-06-11T13:21:26.124 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-06-11T13:21:26.173 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install ceph-fuse 2024-06-11T13:21:26.775 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:01:34 ago on Tue 11 Jun 2024 01:19:52 PM UTC. 2024-06-11T13:21:26.905 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-06-11T13:21:26.905 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:21:26.906 INFO:teuthology.orchestra.run.smithi057.stdout: Package Arch Version Repository Size 2024-06-11T13:21:26.906 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:21:26.906 INFO:teuthology.orchestra.run.smithi057.stdout:Installing: 2024-06-11T13:21:26.906 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-fuse x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 890 k 2024-06-11T13:21:26.906 INFO:teuthology.orchestra.run.smithi057.stdout:Installing dependencies: 2024-06-11T13:21:26.906 INFO:teuthology.orchestra.run.smithi057.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-11T13:21:26.906 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:21:26.906 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction Summary 2024-06-11T13:21:26.906 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:21:26.907 INFO:teuthology.orchestra.run.smithi057.stdout:Install 2 Packages 2024-06-11T13:21:26.907 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:21:26.907 INFO:teuthology.orchestra.run.smithi057.stdout:Total download size: 970 k 2024-06-11T13:21:26.907 INFO:teuthology.orchestra.run.smithi057.stdout:Installed size: 2.8 M 2024-06-11T13:21:26.907 INFO:teuthology.orchestra.run.smithi057.stdout:Downloading Packages: 2024-06-11T13:21:27.011 INFO:teuthology.orchestra.run.smithi057.stdout:[MIRROR] fuse-2.9.9-15.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/fuse-2.9.9-15.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-11T13:21:27.296 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-06-11T13:21:27.329 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2976.g3f51e 12/12 2024-06-11T13:21:27.399 INFO:teuthology.orchestra.run.smithi057.stdout:(1/2): ceph-fuse-19.0.0-2976.g3f51ea39.el9.x86_ 2.0 MB/s | 890 kB 00:00 2024-06-11T13:21:27.540 INFO:teuthology.orchestra.run.smithi057.stdout:(2/2): fuse-2.9.9-15.el9.x86_64.rpm 142 kB/s | 80 kB 00:00 2024-06-11T13:21:27.543 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2024-06-11T13:21:27.543 INFO:teuthology.orchestra.run.smithi057.stdout:Total 1.5 MB/s | 970 kB 00:00 2024-06-11T13:21:27.555 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction check 2024-06-11T13:21:27.579 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction check succeeded. 2024-06-11T13:21:27.580 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction test 2024-06-11T13:21:27.664 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction test succeeded. 2024-06-11T13:21:27.664 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction 2024-06-11T13:21:27.868 INFO:teuthology.orchestra.run.smithi057.stdout: Preparing : 1/1 2024-06-11T13:21:28.024 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-06-11T13:21:28.093 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-fuse-2:19.0.0-2976.g3f51ea39.el9.x86_64 2/2 2024-06-11T13:21:28.724 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-fuse-2:19.0.0-2976.g3f51ea39.el9.x86_64 2/2 2024-06-11T13:21:28.725 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-fuse-2:19.0.0-2976.g3f51ea39.el9.x86_64 1/2 2024-06-11T13:21:29.024 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-06-11T13:21:29.024 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:21:29.024 INFO:teuthology.orchestra.run.smithi057.stdout:Installed: 2024-06-11T13:21:29.024 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-fuse-2:19.0.0-2976.g3f51ea39.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-06-11T13:21:29.024 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:21:29.024 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-06-11T13:21:29.284 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install ceph-volume 2024-06-11T13:21:29.832 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2976.g3f51e 12/12 2024-06-11T13:21:29.832 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2976.g3f51e 1/12 2024-06-11T13:21:29.832 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-06-11T13:21:29.832 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-06-11T13:21:29.832 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-06-11T13:21:29.832 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-06-11T13:21:29.833 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-06-11T13:21:29.833 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-06-11T13:21:29.833 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-06-11T13:21:29.833 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-06-11T13:21:29.833 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-06-11T13:21:29.833 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-06-11T13:21:29.890 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:01:37 ago on Tue 11 Jun 2024 01:19:52 PM UTC. 2024-06-11T13:21:30.022 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-06-11T13:21:30.022 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:21:30.022 INFO:teuthology.orchestra.run.smithi057.stdout: Package Arch Version Repository Size 2024-06-11T13:21:30.023 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:21:30.023 INFO:teuthology.orchestra.run.smithi057.stdout:Installing: 2024-06-11T13:21:30.023 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-volume noarch 2:19.0.0-2976.g3f51ea39.el9 ceph-noarch 264 k 2024-06-11T13:21:30.023 INFO:teuthology.orchestra.run.smithi057.stdout:Installing dependencies: 2024-06-11T13:21:30.023 INFO:teuthology.orchestra.run.smithi057.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-11T13:21:30.023 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:21:30.023 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction Summary 2024-06-11T13:21:30.023 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:21:30.023 INFO:teuthology.orchestra.run.smithi057.stdout:Install 2 Packages 2024-06-11T13:21:30.023 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:21:30.024 INFO:teuthology.orchestra.run.smithi057.stdout:Total download size: 341 k 2024-06-11T13:21:30.024 INFO:teuthology.orchestra.run.smithi057.stdout:Installed size: 1.5 M 2024-06-11T13:21:30.024 INFO:teuthology.orchestra.run.smithi057.stdout:Downloading Packages: 2024-06-11T13:21:30.150 INFO:teuthology.orchestra.run.smithi057.stdout:[MIRROR] python3-packaging-20.9-5.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-packaging-20.9-5.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-11T13:21:30.203 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-06-11T13:21:30.204 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:30.204 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-06-11T13:21:30.204 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2976.g3f51ea39.el9.noarch 2024-06-11T13:21:30.204 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-11T13:21:30.204 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-11T13:21:30.204 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-11T13:21:30.204 INFO:teuthology.orchestra.run.smithi107.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-11T13:21:30.204 INFO:teuthology.orchestra.run.smithi107.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-11T13:21:30.204 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-11T13:21:30.204 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-11T13:21:30.204 INFO:teuthology.orchestra.run.smithi107.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-11T13:21:30.205 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-11T13:21:30.205 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-11T13:21:30.205 INFO:teuthology.orchestra.run.smithi107.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-11T13:21:30.205 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:30.205 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-06-11T13:21:30.336 INFO:teuthology.orchestra.run.smithi057.stdout:(1/2): ceph-volume-19.0.0-2976.g3f51ea39.el9.no 1.1 MB/s | 264 kB 00:00 2024-06-11T13:21:30.570 INFO:teuthology.orchestra.run.smithi057.stdout:(2/2): python3-packaging-20.9-5.el9.noarch.rpm 162 kB/s | 77 kB 00:00 2024-06-11T13:21:30.571 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2024-06-11T13:21:30.571 INFO:teuthology.orchestra.run.smithi057.stdout:Total 622 kB/s | 341 kB 00:00 2024-06-11T13:21:30.583 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction check 2024-06-11T13:21:30.587 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-mgr-rook 2024-06-11T13:21:30.594 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction check succeeded. 2024-06-11T13:21:30.594 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction test 2024-06-11T13:21:30.633 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction test succeeded. 2024-06-11T13:21:30.633 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction 2024-06-11T13:21:30.783 INFO:teuthology.orchestra.run.smithi057.stdout: Preparing : 1/1 2024-06-11T13:21:30.979 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-06-11T13:21:31.013 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-volume-2:19.0.0-2976.g3f51ea39.el9.noarch 2/2 2024-06-11T13:21:31.030 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-volume-2:19.0.0-2976.g3f51ea39.el9.noarch 2/2 2024-06-11T13:21:31.031 INFO:teuthology.orchestra.run.smithi057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-11T13:21:31.031 INFO:teuthology.orchestra.run.smithi057.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-11T13:21:31.031 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:21:31.179 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:25 ago on Tue 11 Jun 2024 01:20:06 PM UTC. 2024-06-11T13:21:31.306 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-06-11T13:21:31.307 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:21:31.307 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-06-11T13:21:31.307 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:21:31.307 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-06-11T13:21:31.307 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-rook noarch 2:19.0.0-2976.g3f51ea39.el9 ceph-noarch 49 k 2024-06-11T13:21:31.307 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-06-11T13:21:31.308 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-11T13:21:31.308 INFO:teuthology.orchestra.run.smithi107.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-11T13:21:31.308 INFO:teuthology.orchestra.run.smithi107.stdout: python3-google-auth noarch 1:2.29.0-2.el9 epel 205 k 2024-06-11T13:21:31.308 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-11T13:21:31.308 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-11T13:21:31.308 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-11T13:21:31.308 INFO:teuthology.orchestra.run.smithi107.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-11T13:21:31.308 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-11T13:21:31.309 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-11T13:21:31.309 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-11T13:21:31.309 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-11T13:21:31.309 INFO:teuthology.orchestra.run.smithi107.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-11T13:21:31.309 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:31.309 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-06-11T13:21:31.309 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:21:31.309 INFO:teuthology.orchestra.run.smithi107.stdout:Install 13 Packages 2024-06-11T13:21:31.309 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:31.310 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 2.2 M 2024-06-11T13:21:31.310 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 26 M 2024-06-11T13:21:31.310 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-06-11T13:21:31.623 INFO:teuthology.orchestra.run.smithi107.stdout:[MIRROR] python3-jsonpatch-1.21-16.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-jsonpatch-1.21-16.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-11T13:21:31.623 INFO:teuthology.orchestra.run.smithi107.stdout:[MIRROR] python3-jsonpointer-2.0-4.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-jsonpointer-2.0-4.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-11T13:21:31.735 INFO:teuthology.orchestra.run.smithi107.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 97 kB/s | 19 kB 00:00 2024-06-11T13:21:31.760 INFO:teuthology.orchestra.run.smithi107.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 115 kB/s | 26 kB 00:00 2024-06-11T13:21:31.761 INFO:teuthology.orchestra.run.smithi107.stdout:[MIRROR] python3-oauthlib-3.1.1-5.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-oauthlib-3.1.1-5.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-11T13:21:31.761 INFO:teuthology.orchestra.run.smithi107.stdout:[MIRROR] python3-pyasn1-0.4.8-6.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-pyasn1-0.4.8-6.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-11T13:21:31.787 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-volume-2:19.0.0-2976.g3f51ea39.el9.noarch 1/2 2024-06-11T13:21:31.827 INFO:teuthology.orchestra.run.smithi107.stdout:(3/13): ceph-mgr-rook-19.0.0-2976.g3f51ea39.el9 170 kB/s | 49 kB 00:00 2024-06-11T13:21:31.878 INFO:teuthology.orchestra.run.smithi107.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-06-11T13:21:31.904 INFO:teuthology.orchestra.run.smithi107.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.3 MB/s | 222 kB 00:00 2024-06-11T13:21:31.928 INFO:teuthology.orchestra.run.smithi107.stdout:(6/13): python3-cachetools-4.2.4-1.el9.noarch.r 1.3 MB/s | 32 kB 00:00 2024-06-11T13:21:31.954 INFO:teuthology.orchestra.run.smithi107.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 709 kB/s | 54 kB 00:00 2024-06-11T13:21:31.979 INFO:teuthology.orchestra.run.smithi107.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 297 kB/s | 15 kB 00:00 2024-06-11T13:21:32.004 INFO:teuthology.orchestra.run.smithi107.stdout:(9/13): python3-google-auth-2.29.0-2.el9.noarch 4.0 MB/s | 205 kB 00:00 2024-06-11T13:21:32.030 INFO:teuthology.orchestra.run.smithi107.stdout:(10/13): python3-pyasn1-modules-0.4.8-6.el9.noa 1.3 MB/s | 279 kB 00:00 2024-06-11T13:21:32.055 INFO:teuthology.orchestra.run.smithi107.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-06-11T13:21:32.113 INFO:teuthology.orchestra.run.smithi107.stdout:(12/13): python3-kubernetes-26.1.0-2.el9.noarch 7.6 MB/s | 1.0 MB 00:00 2024-06-11T13:21:32.122 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-06-11T13:21:32.128 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:21:32.128 INFO:teuthology.orchestra.run.smithi057.stdout:Installed: 2024-06-11T13:21:32.128 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-volume-2:19.0.0-2976.g3f51ea39.el9.noarch 2024-06-11T13:21:32.128 INFO:teuthology.orchestra.run.smithi057.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-11T13:21:32.128 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:21:32.128 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-06-11T13:21:32.139 INFO:teuthology.orchestra.run.smithi107.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 823 kB/s | 90 kB 00:00 2024-06-11T13:21:32.142 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-06-11T13:21:32.142 INFO:teuthology.orchestra.run.smithi107.stdout:Total 2.6 MB/s | 2.2 MB 00:00 2024-06-11T13:21:32.207 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-06-11T13:21:32.221 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-06-11T13:21:32.222 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-06-11T13:21:32.309 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-06-11T13:21:32.309 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-06-11T13:21:32.394 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install librados-devel 2024-06-11T13:21:32.828 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-06-11T13:21:33.009 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:01:41 ago on Tue 11 Jun 2024 01:19:52 PM UTC. 2024-06-11T13:21:33.012 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-06-11T13:21:33.082 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-06-11T13:21:33.137 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-06-11T13:21:33.139 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-06-11T13:21:33.139 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:21:33.139 INFO:teuthology.orchestra.run.smithi057.stdout: Package Arch Version Repo Size 2024-06-11T13:21:33.139 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:21:33.139 INFO:teuthology.orchestra.run.smithi057.stdout:Installing: 2024-06-11T13:21:33.139 INFO:teuthology.orchestra.run.smithi057.stdout: librados-devel x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 127 k 2024-06-11T13:21:33.140 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:21:33.140 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction Summary 2024-06-11T13:21:33.140 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:21:33.140 INFO:teuthology.orchestra.run.smithi057.stdout:Install 1 Package 2024-06-11T13:21:33.140 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:21:33.140 INFO:teuthology.orchestra.run.smithi057.stdout:Total download size: 127 k 2024-06-11T13:21:33.140 INFO:teuthology.orchestra.run.smithi057.stdout:Installed size: 456 k 2024-06-11T13:21:33.141 INFO:teuthology.orchestra.run.smithi057.stdout:Downloading Packages: 2024-06-11T13:21:33.175 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-06-11T13:21:33.217 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-06-11T13:21:33.310 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-06-11T13:21:33.411 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-google-auth-1:2.29.0-2.el9.noarch 7/13 2024-06-11T13:21:33.467 INFO:teuthology.orchestra.run.smithi057.stdout:librados-devel-19.0.0-2976.g3f51ea39.el9.x86_64 390 kB/s | 127 kB 00:00 2024-06-11T13:21:33.467 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-06-11T13:21:33.467 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2024-06-11T13:21:33.468 INFO:teuthology.orchestra.run.smithi057.stdout:Total 388 kB/s | 127 kB 00:00 2024-06-11T13:21:33.468 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction check 2024-06-11T13:21:33.473 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction check succeeded. 2024-06-11T13:21:33.474 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction test 2024-06-11T13:21:33.543 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction test succeeded. 2024-06-11T13:21:33.543 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction 2024-06-11T13:21:33.696 INFO:teuthology.orchestra.run.smithi057.stdout: Preparing : 1/1 2024-06-11T13:21:33.810 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : librados-devel-2:19.0.0-2976.g3f51ea39.el9.x86_64 1/1 2024-06-11T13:21:34.180 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-06-11T13:21:34.232 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-06-11T13:21:34.266 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-06-11T13:21:34.275 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: librados-devel-2:19.0.0-2976.g3f51ea39.el9.x86_64 1/1 2024-06-11T13:21:34.314 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-06-11T13:21:34.336 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-rook-2:19.0.0-2976.g3f51ea39.el9.noarch 13/13 2024-06-11T13:21:34.514 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librados-devel-2:19.0.0-2976.g3f51ea39.el9.x86_64 1/1 2024-06-11T13:21:34.514 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:21:34.514 INFO:teuthology.orchestra.run.smithi057.stdout:Installed: 2024-06-11T13:21:34.514 INFO:teuthology.orchestra.run.smithi057.stdout: librados-devel-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:21:34.514 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:21:34.514 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-06-11T13:21:34.781 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install libcephfs2 2024-06-11T13:21:35.119 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2976.g3f51ea39.el9.noarch 13/13 2024-06-11T13:21:35.119 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2976.g3f51ea39.el9.noarch 1/13 2024-06-11T13:21:35.119 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-06-11T13:21:35.119 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-06-11T13:21:35.119 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-06-11T13:21:35.119 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-06-11T13:21:35.119 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-06-11T13:21:35.120 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-06-11T13:21:35.120 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-06-11T13:21:35.120 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-06-11T13:21:35.120 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-google-auth-1:2.29.0-2.el9.noarch 10/13 2024-06-11T13:21:35.120 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-06-11T13:21:35.120 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-06-11T13:21:35.390 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:01:43 ago on Tue 11 Jun 2024 01:19:52 PM UTC. 2024-06-11T13:21:35.440 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-06-11T13:21:35.440 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:35.440 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-06-11T13:21:35.440 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-rook-2:19.0.0-2976.g3f51ea39.el9.noarch 2024-06-11T13:21:35.440 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-11T13:21:35.440 INFO:teuthology.orchestra.run.smithi107.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-11T13:21:35.441 INFO:teuthology.orchestra.run.smithi107.stdout: python3-google-auth-1:2.29.0-2.el9.noarch 2024-06-11T13:21:35.441 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-11T13:21:35.441 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-11T13:21:35.441 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-11T13:21:35.441 INFO:teuthology.orchestra.run.smithi107.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-11T13:21:35.441 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-11T13:21:35.441 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-11T13:21:35.441 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-11T13:21:35.441 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-11T13:21:35.441 INFO:teuthology.orchestra.run.smithi107.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-11T13:21:35.441 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:35.441 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-06-11T13:21:35.474 INFO:teuthology.orchestra.run.smithi057.stdout:Package libcephfs2-2:19.0.0-2976.g3f51ea39.el9.x86_64 is already installed. 2024-06-11T13:21:35.518 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-06-11T13:21:35.519 INFO:teuthology.orchestra.run.smithi057.stdout:Nothing to do. 2024-06-11T13:21:35.520 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-06-11T13:21:35.612 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install libcephfs-devel 2024-06-11T13:21:35.671 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-mgr-cephadm 2024-06-11T13:21:36.207 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:01:44 ago on Tue 11 Jun 2024 01:19:52 PM UTC. 2024-06-11T13:21:36.244 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:30 ago on Tue 11 Jun 2024 01:20:06 PM UTC. 2024-06-11T13:21:36.337 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-06-11T13:21:36.338 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:21:36.338 INFO:teuthology.orchestra.run.smithi057.stdout: Package Arch Version Repo Size 2024-06-11T13:21:36.338 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:21:36.338 INFO:teuthology.orchestra.run.smithi057.stdout:Installing: 2024-06-11T13:21:36.338 INFO:teuthology.orchestra.run.smithi057.stdout: libcephfs-devel x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 32 k 2024-06-11T13:21:36.338 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:21:36.338 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction Summary 2024-06-11T13:21:36.338 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:21:36.338 INFO:teuthology.orchestra.run.smithi057.stdout:Install 1 Package 2024-06-11T13:21:36.338 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:21:36.338 INFO:teuthology.orchestra.run.smithi057.stdout:Total download size: 32 k 2024-06-11T13:21:36.338 INFO:teuthology.orchestra.run.smithi057.stdout:Installed size: 142 k 2024-06-11T13:21:36.339 INFO:teuthology.orchestra.run.smithi057.stdout:Downloading Packages: 2024-06-11T13:21:36.366 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-06-11T13:21:36.367 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:21:36.367 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-06-11T13:21:36.367 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:21:36.367 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-06-11T13:21:36.367 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2976.g3f51ea39.el9 ceph-noarch 142 k 2024-06-11T13:21:36.367 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-06-11T13:21:36.367 INFO:teuthology.orchestra.run.smithi107.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-11T13:21:36.368 INFO:teuthology.orchestra.run.smithi107.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-11T13:21:36.368 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-11T13:21:36.368 INFO:teuthology.orchestra.run.smithi107.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-11T13:21:36.368 INFO:teuthology.orchestra.run.smithi107.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-11T13:21:36.368 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:36.368 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-06-11T13:21:36.368 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:21:36.368 INFO:teuthology.orchestra.run.smithi107.stdout:Install 6 Packages 2024-06-11T13:21:36.368 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:36.369 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 7.0 M 2024-06-11T13:21:36.369 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 33 M 2024-06-11T13:21:36.369 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-06-11T13:21:36.585 INFO:teuthology.orchestra.run.smithi057.stdout:libcephfs-devel-19.0.0-2976.g3f51ea39.el9.x86_6 129 kB/s | 32 kB 00:00 2024-06-11T13:21:36.586 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2024-06-11T13:21:36.586 INFO:teuthology.orchestra.run.smithi057.stdout:Total 128 kB/s | 32 kB 00:00 2024-06-11T13:21:36.586 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction check 2024-06-11T13:21:36.589 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction check succeeded. 2024-06-11T13:21:36.590 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction test 2024-06-11T13:21:36.611 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction test succeeded. 2024-06-11T13:21:36.612 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction 2024-06-11T13:21:36.697 INFO:teuthology.orchestra.run.smithi057.stdout: Preparing : 1/1 2024-06-11T13:21:36.802 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libcephfs-devel-2:19.0.0-2976.g3f51ea39.el9.x86_64 1/1 2024-06-11T13:21:37.074 INFO:teuthology.orchestra.run.smithi107.stdout:[MIRROR] python3-babel-2.9.1-2.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-babel-2.9.1-2.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-11T13:21:37.074 INFO:teuthology.orchestra.run.smithi107.stdout:[MIRROR] python3-jinja2-2.11.3-6.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-jinja2-2.11.3-6.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-11T13:21:37.144 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-2976.g3f51ea39.el9.x86_64 1/1 2024-06-11T13:21:37.234 INFO:teuthology.orchestra.run.smithi107.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.0 MB/s | 248 kB 00:00 2024-06-11T13:21:37.259 INFO:teuthology.orchestra.run.smithi107.stdout:(2/6): ceph-mgr-cephadm-19.0.0-2976.g3f51ea39.e 537 kB/s | 142 kB 00:00 2024-06-11T13:21:37.293 INFO:teuthology.orchestra.run.smithi107.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.7 MB/s | 58 kB 00:00 2024-06-11T13:21:37.319 INFO:teuthology.orchestra.run.smithi107.stdout:(4/6): python3-typing-extensions-4.4.0-2.el9.no 2.0 MB/s | 51 kB 00:00 2024-06-11T13:21:37.344 INFO:teuthology.orchestra.run.smithi107.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 4.9 MB/s | 548 kB 00:00 2024-06-11T13:21:37.379 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libcephfs-devel-2:19.0.0-2976.g3f51ea39.el9.x86_64 1/1 2024-06-11T13:21:37.380 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:21:37.380 INFO:teuthology.orchestra.run.smithi057.stdout:Installed: 2024-06-11T13:21:37.380 INFO:teuthology.orchestra.run.smithi057.stdout: libcephfs-devel-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:21:37.380 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:21:37.380 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-06-11T13:21:37.452 INFO:teuthology.orchestra.run.smithi107.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 6.0 MB 00:00 2024-06-11T13:21:37.455 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-06-11T13:21:37.455 INFO:teuthology.orchestra.run.smithi107.stdout:Total 6.4 MB/s | 7.0 MB 00:01 2024-06-11T13:21:37.526 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-06-11T13:21:37.537 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-06-11T13:21:37.537 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-06-11T13:21:37.581 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install librados2 2024-06-11T13:21:37.635 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-06-11T13:21:37.635 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-06-11T13:21:37.938 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-06-11T13:21:38.108 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-06-11T13:21:38.162 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-06-11T13:21:38.177 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:01:46 ago on Tue 11 Jun 2024 01:19:52 PM UTC. 2024-06-11T13:21:38.263 INFO:teuthology.orchestra.run.smithi057.stdout:Package librados2-2:19.0.0-2976.g3f51ea39.el9.x86_64 is already installed. 2024-06-11T13:21:38.308 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-06-11T13:21:38.309 INFO:teuthology.orchestra.run.smithi057.stdout:Nothing to do. 2024-06-11T13:21:38.309 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-06-11T13:21:38.370 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install librbd1 2024-06-11T13:21:38.686 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-06-11T13:21:38.773 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-06-11T13:21:38.837 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-06-11T13:21:38.867 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2976.g3f51ea39.el9.noarch 6/6 2024-06-11T13:21:38.972 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:01:46 ago on Tue 11 Jun 2024 01:19:52 PM UTC. 2024-06-11T13:21:39.058 INFO:teuthology.orchestra.run.smithi057.stdout:Package librbd1-2:19.0.0-2976.g3f51ea39.el9.x86_64 is already installed. 2024-06-11T13:21:39.103 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-06-11T13:21:39.104 INFO:teuthology.orchestra.run.smithi057.stdout:Nothing to do. 2024-06-11T13:21:39.104 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-06-11T13:21:39.178 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install python3-rados 2024-06-11T13:21:39.649 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2976.g3f51ea39.el9.noarch 6/6 2024-06-11T13:21:39.649 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2976.g3f51ea39.el9.noarch 1/6 2024-06-11T13:21:39.649 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-06-11T13:21:39.649 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-06-11T13:21:39.649 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-06-11T13:21:39.650 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-06-11T13:21:39.780 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:01:47 ago on Tue 11 Jun 2024 01:19:52 PM UTC. 2024-06-11T13:21:39.863 INFO:teuthology.orchestra.run.smithi057.stdout:Package python3-rados-2:19.0.0-2976.g3f51ea39.el9.x86_64 is already installed. 2024-06-11T13:21:39.905 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-06-11T13:21:39.905 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:39.905 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-06-11T13:21:39.905 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-cephadm-2:19.0.0-2976.g3f51ea39.el9.noarch 2024-06-11T13:21:39.905 INFO:teuthology.orchestra.run.smithi107.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-11T13:21:39.906 INFO:teuthology.orchestra.run.smithi107.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-11T13:21:39.906 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-11T13:21:39.906 INFO:teuthology.orchestra.run.smithi107.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-11T13:21:39.906 INFO:teuthology.orchestra.run.smithi107.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-11T13:21:39.906 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:39.906 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-06-11T13:21:39.907 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-06-11T13:21:39.908 INFO:teuthology.orchestra.run.smithi057.stdout:Nothing to do. 2024-06-11T13:21:39.908 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-06-11T13:21:39.966 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install python3-rgw 2024-06-11T13:21:40.112 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-fuse 2024-06-11T13:21:40.566 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:01:48 ago on Tue 11 Jun 2024 01:19:52 PM UTC. 2024-06-11T13:21:40.651 INFO:teuthology.orchestra.run.smithi057.stdout:Package python3-rgw-2:19.0.0-2976.g3f51ea39.el9.x86_64 is already installed. 2024-06-11T13:21:40.688 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:34 ago on Tue 11 Jun 2024 01:20:06 PM UTC. 2024-06-11T13:21:40.696 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-06-11T13:21:40.697 INFO:teuthology.orchestra.run.smithi057.stdout:Nothing to do. 2024-06-11T13:21:40.697 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-06-11T13:21:40.761 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install python3-cephfs 2024-06-11T13:21:40.812 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-06-11T13:21:40.813 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:21:40.813 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-06-11T13:21:40.813 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:21:40.813 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-06-11T13:21:40.813 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-fuse x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 890 k 2024-06-11T13:21:40.813 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-06-11T13:21:40.813 INFO:teuthology.orchestra.run.smithi107.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-11T13:21:40.813 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:40.814 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-06-11T13:21:40.814 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:21:40.814 INFO:teuthology.orchestra.run.smithi107.stdout:Install 2 Packages 2024-06-11T13:21:40.814 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:40.814 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 970 k 2024-06-11T13:21:40.814 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 2.8 M 2024-06-11T13:21:40.815 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-06-11T13:21:40.918 INFO:teuthology.orchestra.run.smithi107.stdout:[MIRROR] fuse-2.9.9-15.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/fuse-2.9.9-15.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-11T13:21:41.321 INFO:teuthology.orchestra.run.smithi107.stdout:(1/2): ceph-fuse-19.0.0-2976.g3f51ea39.el9.x86_ 2.0 MB/s | 890 kB 00:00 2024-06-11T13:21:41.367 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:01:49 ago on Tue 11 Jun 2024 01:19:52 PM UTC. 2024-06-11T13:21:41.371 INFO:teuthology.orchestra.run.smithi107.stdout:(2/2): fuse-2.9.9-15.el9.x86_64.rpm 164 kB/s | 80 kB 00:00 2024-06-11T13:21:41.373 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-06-11T13:21:41.373 INFO:teuthology.orchestra.run.smithi107.stdout:Total 1.7 MB/s | 970 kB 00:00 2024-06-11T13:21:41.385 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-06-11T13:21:41.407 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-06-11T13:21:41.407 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-06-11T13:21:41.452 INFO:teuthology.orchestra.run.smithi057.stdout:Package python3-cephfs-2:19.0.0-2976.g3f51ea39.el9.x86_64 is already installed. 2024-06-11T13:21:41.485 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-06-11T13:21:41.486 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-06-11T13:21:41.496 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-06-11T13:21:41.497 INFO:teuthology.orchestra.run.smithi057.stdout:Nothing to do. 2024-06-11T13:21:41.497 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-06-11T13:21:41.558 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install python3-rbd 2024-06-11T13:21:41.690 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-06-11T13:21:41.832 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-06-11T13:21:41.905 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-fuse-2:19.0.0-2976.g3f51ea39.el9.x86_64 2/2 2024-06-11T13:21:42.158 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:01:50 ago on Tue 11 Jun 2024 01:19:52 PM UTC. 2024-06-11T13:21:42.243 INFO:teuthology.orchestra.run.smithi057.stdout:Package python3-rbd-2:19.0.0-2976.g3f51ea39.el9.x86_64 is already installed. 2024-06-11T13:21:42.288 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-06-11T13:21:42.289 INFO:teuthology.orchestra.run.smithi057.stdout:Nothing to do. 2024-06-11T13:21:42.289 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-06-11T13:21:42.354 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install rbd-fuse 2024-06-11T13:21:42.530 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-fuse-2:19.0.0-2976.g3f51ea39.el9.x86_64 2/2 2024-06-11T13:21:42.530 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-fuse-2:19.0.0-2976.g3f51ea39.el9.x86_64 1/2 2024-06-11T13:21:42.792 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-06-11T13:21:42.792 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:42.792 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-06-11T13:21:42.792 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-fuse-2:19.0.0-2976.g3f51ea39.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-06-11T13:21:42.792 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:42.792 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-06-11T13:21:42.980 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:01:50 ago on Tue 11 Jun 2024 01:19:52 PM UTC. 2024-06-11T13:21:43.059 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-volume 2024-06-11T13:21:43.109 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-06-11T13:21:43.110 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:21:43.110 INFO:teuthology.orchestra.run.smithi057.stdout: Package Architecture Version Repository Size 2024-06-11T13:21:43.110 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:21:43.110 INFO:teuthology.orchestra.run.smithi057.stdout:Installing: 2024-06-11T13:21:43.110 INFO:teuthology.orchestra.run.smithi057.stdout: rbd-fuse x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 86 k 2024-06-11T13:21:43.110 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:21:43.111 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction Summary 2024-06-11T13:21:43.111 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:21:43.111 INFO:teuthology.orchestra.run.smithi057.stdout:Install 1 Package 2024-06-11T13:21:43.111 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:21:43.111 INFO:teuthology.orchestra.run.smithi057.stdout:Total download size: 86 k 2024-06-11T13:21:43.111 INFO:teuthology.orchestra.run.smithi057.stdout:Installed size: 231 k 2024-06-11T13:21:43.111 INFO:teuthology.orchestra.run.smithi057.stdout:Downloading Packages: 2024-06-11T13:21:43.318 INFO:teuthology.orchestra.run.smithi057.stdout:rbd-fuse-19.0.0-2976.g3f51ea39.el9.x86_64.rpm 415 kB/s | 86 kB 00:00 2024-06-11T13:21:43.318 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2024-06-11T13:21:43.318 INFO:teuthology.orchestra.run.smithi057.stdout:Total 412 kB/s | 86 kB 00:00 2024-06-11T13:21:43.319 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction check 2024-06-11T13:21:43.326 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction check succeeded. 2024-06-11T13:21:43.326 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction test 2024-06-11T13:21:43.393 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction test succeeded. 2024-06-11T13:21:43.393 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction 2024-06-11T13:21:43.548 INFO:teuthology.orchestra.run.smithi057.stdout: Preparing : 1/1 2024-06-11T13:21:43.644 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : rbd-fuse-2:19.0.0-2976.g3f51ea39.el9.x86_64 1/1 2024-06-11T13:21:43.651 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:37 ago on Tue 11 Jun 2024 01:20:06 PM UTC. 2024-06-11T13:21:43.778 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-06-11T13:21:43.779 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:21:43.779 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-06-11T13:21:43.779 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:21:43.779 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-06-11T13:21:43.779 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-volume noarch 2:19.0.0-2976.g3f51ea39.el9 ceph-noarch 264 k 2024-06-11T13:21:43.779 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-06-11T13:21:43.779 INFO:teuthology.orchestra.run.smithi107.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-11T13:21:43.779 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:43.779 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-06-11T13:21:43.779 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:21:43.779 INFO:teuthology.orchestra.run.smithi107.stdout:Install 2 Packages 2024-06-11T13:21:43.779 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:43.780 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 341 k 2024-06-11T13:21:43.780 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 1.5 M 2024-06-11T13:21:43.780 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-06-11T13:21:44.034 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: rbd-fuse-2:19.0.0-2976.g3f51ea39.el9.x86_64 1/1 2024-06-11T13:21:44.126 INFO:teuthology.orchestra.run.smithi107.stdout:[MIRROR] python3-packaging-20.9-5.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-packaging-20.9-5.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-11T13:21:44.311 INFO:teuthology.orchestra.run.smithi107.stdout:(1/2): ceph-volume-19.0.0-2976.g3f51ea39.el9.no 1.1 MB/s | 264 kB 00:00 2024-06-11T13:21:44.321 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : rbd-fuse-2:19.0.0-2976.g3f51ea39.el9.x86_64 1/1 2024-06-11T13:21:44.321 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:21:44.321 INFO:teuthology.orchestra.run.smithi057.stdout:Installed: 2024-06-11T13:21:44.322 INFO:teuthology.orchestra.run.smithi057.stdout: rbd-fuse-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:21:44.322 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:21:44.322 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-06-11T13:21:44.336 INFO:teuthology.orchestra.run.smithi107.stdout:(2/2): python3-packaging-20.9-5.el9.noarch.rpm 292 kB/s | 77 kB 00:00 2024-06-11T13:21:44.337 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-06-11T13:21:44.337 INFO:teuthology.orchestra.run.smithi107.stdout:Total 611 kB/s | 341 kB 00:00 2024-06-11T13:21:44.349 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-06-11T13:21:44.359 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-06-11T13:21:44.359 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-06-11T13:21:44.396 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-06-11T13:21:44.396 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-06-11T13:21:44.541 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-06-11T13:21:44.564 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install rbd-mirror 2024-06-11T13:21:44.726 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-06-11T13:21:44.764 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-volume-2:19.0.0-2976.g3f51ea39.el9.noarch 2/2 2024-06-11T13:21:44.779 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-volume-2:19.0.0-2976.g3f51ea39.el9.noarch 2/2 2024-06-11T13:21:44.779 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-11T13:21:44.779 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-11T13:21:44.779 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:45.166 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:01:53 ago on Tue 11 Jun 2024 01:19:52 PM UTC. 2024-06-11T13:21:45.297 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-06-11T13:21:45.298 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:21:45.298 INFO:teuthology.orchestra.run.smithi057.stdout: Package Arch Version Repo Size 2024-06-11T13:21:45.298 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:21:45.298 INFO:teuthology.orchestra.run.smithi057.stdout:Installing: 2024-06-11T13:21:45.298 INFO:teuthology.orchestra.run.smithi057.stdout: rbd-mirror x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 3.1 M 2024-06-11T13:21:45.298 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:21:45.298 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction Summary 2024-06-11T13:21:45.298 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:21:45.298 INFO:teuthology.orchestra.run.smithi057.stdout:Install 1 Package 2024-06-11T13:21:45.298 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:21:45.299 INFO:teuthology.orchestra.run.smithi057.stdout:Total download size: 3.1 M 2024-06-11T13:21:45.299 INFO:teuthology.orchestra.run.smithi057.stdout:Installed size: 13 M 2024-06-11T13:21:45.299 INFO:teuthology.orchestra.run.smithi057.stdout:Downloading Packages: 2024-06-11T13:21:45.440 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-volume-2:19.0.0-2976.g3f51ea39.el9.noarch 1/2 2024-06-11T13:21:45.689 INFO:teuthology.orchestra.run.smithi057.stdout:rbd-mirror-19.0.0-2976.g3f51ea39.el9.x86_64.rpm 7.9 MB/s | 3.1 MB 00:00 2024-06-11T13:21:45.689 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2024-06-11T13:21:45.689 INFO:teuthology.orchestra.run.smithi057.stdout:Total 7.9 MB/s | 3.1 MB 00:00 2024-06-11T13:21:45.690 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction check 2024-06-11T13:21:45.700 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction check succeeded. 2024-06-11T13:21:45.700 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction test 2024-06-11T13:21:45.730 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-06-11T13:21:45.730 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:45.730 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-06-11T13:21:45.730 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-volume-2:19.0.0-2976.g3f51ea39.el9.noarch 2024-06-11T13:21:45.730 INFO:teuthology.orchestra.run.smithi107.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-11T13:21:45.731 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:45.731 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-06-11T13:21:45.785 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction test succeeded. 2024-06-11T13:21:45.785 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction 2024-06-11T13:21:46.043 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install librados-devel 2024-06-11T13:21:46.229 INFO:teuthology.orchestra.run.smithi057.stdout: Preparing : 1/1 2024-06-11T13:21:46.281 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : rbd-mirror-2:19.0.0-2976.g3f51ea39.el9.x86_64 1/1 2024-06-11T13:21:46.307 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2976.g3f51ea39.el9.x86_64 1/1 2024-06-11T13:21:46.307 INFO:teuthology.orchestra.run.smithi057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-11T13:21:46.307 INFO:teuthology.orchestra.run.smithi057.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-11T13:21:46.307 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-11T13:21:46.308 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-11T13:21:46.308 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:21:46.633 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:40 ago on Tue 11 Jun 2024 01:20:06 PM UTC. 2024-06-11T13:21:46.756 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-06-11T13:21:46.757 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:21:46.757 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repo Size 2024-06-11T13:21:46.757 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:21:46.757 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-06-11T13:21:46.758 INFO:teuthology.orchestra.run.smithi107.stdout: librados-devel x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 127 k 2024-06-11T13:21:46.758 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:46.758 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-06-11T13:21:46.758 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:21:46.758 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-06-11T13:21:46.758 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:46.758 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 127 k 2024-06-11T13:21:46.758 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 456 k 2024-06-11T13:21:46.758 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-06-11T13:21:46.968 INFO:teuthology.orchestra.run.smithi107.stdout:librados-devel-19.0.0-2976.g3f51ea39.el9.x86_64 606 kB/s | 127 kB 00:00 2024-06-11T13:21:46.968 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-06-11T13:21:46.969 INFO:teuthology.orchestra.run.smithi107.stdout:Total 602 kB/s | 127 kB 00:00 2024-06-11T13:21:46.969 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-06-11T13:21:46.974 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-06-11T13:21:46.974 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-06-11T13:21:47.040 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-06-11T13:21:47.040 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-06-11T13:21:47.188 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-06-11T13:21:47.300 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librados-devel-2:19.0.0-2976.g3f51ea39.el9.x86_64 1/1 2024-06-11T13:21:47.387 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : rbd-mirror-2:19.0.0-2976.g3f51ea39.el9.x86_64 1/1 2024-06-11T13:21:47.387 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:21:47.388 INFO:teuthology.orchestra.run.smithi057.stdout:Installed: 2024-06-11T13:21:47.388 INFO:teuthology.orchestra.run.smithi057.stdout: rbd-mirror-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:21:47.388 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:21:47.388 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-06-11T13:21:47.618 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install rbd-nbd 2024-06-11T13:21:47.716 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados-devel-2:19.0.0-2976.g3f51ea39.el9.x86_64 1/1 2024-06-11T13:21:47.970 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados-devel-2:19.0.0-2976.g3f51ea39.el9.x86_64 1/1 2024-06-11T13:21:47.970 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:47.970 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-06-11T13:21:47.970 INFO:teuthology.orchestra.run.smithi107.stdout: librados-devel-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:21:47.970 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:47.971 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-06-11T13:21:48.224 INFO:teuthology.orchestra.run.smithi057.stdout:Last metadata expiration check: 0:01:56 ago on Tue 11 Jun 2024 01:19:52 PM UTC. 2024-06-11T13:21:48.271 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install libcephfs2 2024-06-11T13:21:48.354 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-06-11T13:21:48.354 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:21:48.355 INFO:teuthology.orchestra.run.smithi057.stdout: Package Architecture Version Repository Size 2024-06-11T13:21:48.355 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:21:48.355 INFO:teuthology.orchestra.run.smithi057.stdout:Installing: 2024-06-11T13:21:48.355 INFO:teuthology.orchestra.run.smithi057.stdout: rbd-nbd x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 170 k 2024-06-11T13:21:48.355 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:21:48.355 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction Summary 2024-06-11T13:21:48.355 INFO:teuthology.orchestra.run.smithi057.stdout:================================================================================ 2024-06-11T13:21:48.355 INFO:teuthology.orchestra.run.smithi057.stdout:Install 1 Package 2024-06-11T13:21:48.355 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:21:48.356 INFO:teuthology.orchestra.run.smithi057.stdout:Total download size: 170 k 2024-06-11T13:21:48.356 INFO:teuthology.orchestra.run.smithi057.stdout:Installed size: 494 k 2024-06-11T13:21:48.356 INFO:teuthology.orchestra.run.smithi057.stdout:Downloading Packages: 2024-06-11T13:21:48.676 INFO:teuthology.orchestra.run.smithi057.stdout:rbd-nbd-19.0.0-2976.g3f51ea39.el9.x86_64.rpm 532 kB/s | 170 kB 00:00 2024-06-11T13:21:48.677 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2024-06-11T13:21:48.677 INFO:teuthology.orchestra.run.smithi057.stdout:Total 530 kB/s | 170 kB 00:00 2024-06-11T13:21:48.677 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction check 2024-06-11T13:21:48.685 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction check succeeded. 2024-06-11T13:21:48.686 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction test 2024-06-11T13:21:48.753 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction test succeeded. 2024-06-11T13:21:48.753 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction 2024-06-11T13:21:48.857 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:42 ago on Tue 11 Jun 2024 01:20:06 PM UTC. 2024-06-11T13:21:48.914 INFO:teuthology.orchestra.run.smithi057.stdout: Preparing : 1/1 2024-06-11T13:21:48.941 INFO:teuthology.orchestra.run.smithi107.stdout:Package libcephfs2-2:19.0.0-2976.g3f51ea39.el9.x86_64 is already installed. 2024-06-11T13:21:48.985 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-06-11T13:21:48.986 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-06-11T13:21:48.986 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-06-11T13:21:49.018 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : rbd-nbd-2:19.0.0-2976.g3f51ea39.el9.x86_64 1/1 2024-06-11T13:21:49.056 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install libcephfs-devel 2024-06-11T13:21:49.431 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: rbd-nbd-2:19.0.0-2976.g3f51ea39.el9.x86_64 1/1 2024-06-11T13:21:49.624 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:43 ago on Tue 11 Jun 2024 01:20:06 PM UTC. 2024-06-11T13:21:49.695 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : rbd-nbd-2:19.0.0-2976.g3f51ea39.el9.x86_64 1/1 2024-06-11T13:21:49.696 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:21:49.696 INFO:teuthology.orchestra.run.smithi057.stdout:Installed: 2024-06-11T13:21:49.696 INFO:teuthology.orchestra.run.smithi057.stdout: rbd-nbd-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:21:49.696 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:21:49.696 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-06-11T13:21:49.748 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-06-11T13:21:49.749 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:21:49.749 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repo Size 2024-06-11T13:21:49.749 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:21:49.749 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-06-11T13:21:49.749 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-devel x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 32 k 2024-06-11T13:21:49.749 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:49.749 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-06-11T13:21:49.750 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:21:49.750 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-06-11T13:21:49.750 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:49.750 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 32 k 2024-06-11T13:21:49.750 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 142 k 2024-06-11T13:21:49.750 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-06-11T13:21:49.930 DEBUG:teuthology.parallel:result is None 2024-06-11T13:21:50.023 INFO:teuthology.orchestra.run.smithi107.stdout:libcephfs-devel-19.0.0-2976.g3f51ea39.el9.x86_6 116 kB/s | 32 kB 00:00 2024-06-11T13:21:50.023 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-06-11T13:21:50.024 INFO:teuthology.orchestra.run.smithi107.stdout:Total 116 kB/s | 32 kB 00:00 2024-06-11T13:21:50.024 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-06-11T13:21:50.027 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-06-11T13:21:50.027 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-06-11T13:21:50.044 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-06-11T13:21:50.044 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-06-11T13:21:50.128 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-06-11T13:21:50.230 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephfs-devel-2:19.0.0-2976.g3f51ea39.el9.x86_64 1/1 2024-06-11T13:21:50.592 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-2976.g3f51ea39.el9.x86_64 1/1 2024-06-11T13:21:50.804 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephfs-devel-2:19.0.0-2976.g3f51ea39.el9.x86_64 1/1 2024-06-11T13:21:50.804 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:50.804 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-06-11T13:21:50.804 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-devel-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:21:50.805 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:50.805 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-06-11T13:21:50.981 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install librados2 2024-06-11T13:21:51.554 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:45 ago on Tue 11 Jun 2024 01:20:06 PM UTC. 2024-06-11T13:21:51.634 INFO:teuthology.orchestra.run.smithi107.stdout:Package librados2-2:19.0.0-2976.g3f51ea39.el9.x86_64 is already installed. 2024-06-11T13:21:51.676 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-06-11T13:21:51.677 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-06-11T13:21:51.677 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-06-11T13:21:51.734 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install librbd1 2024-06-11T13:21:52.303 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:46 ago on Tue 11 Jun 2024 01:20:06 PM UTC. 2024-06-11T13:21:52.382 INFO:teuthology.orchestra.run.smithi107.stdout:Package librbd1-2:19.0.0-2976.g3f51ea39.el9.x86_64 is already installed. 2024-06-11T13:21:52.425 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-06-11T13:21:52.426 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-06-11T13:21:52.426 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-06-11T13:21:52.496 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install python3-rados 2024-06-11T13:21:53.075 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:47 ago on Tue 11 Jun 2024 01:20:06 PM UTC. 2024-06-11T13:21:53.156 INFO:teuthology.orchestra.run.smithi107.stdout:Package python3-rados-2:19.0.0-2976.g3f51ea39.el9.x86_64 is already installed. 2024-06-11T13:21:53.198 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-06-11T13:21:53.200 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-06-11T13:21:53.200 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-06-11T13:21:53.269 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install python3-rgw 2024-06-11T13:21:53.849 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:47 ago on Tue 11 Jun 2024 01:20:06 PM UTC. 2024-06-11T13:21:53.929 INFO:teuthology.orchestra.run.smithi107.stdout:Package python3-rgw-2:19.0.0-2976.g3f51ea39.el9.x86_64 is already installed. 2024-06-11T13:21:53.971 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-06-11T13:21:53.972 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-06-11T13:21:53.972 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-06-11T13:21:54.039 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install python3-cephfs 2024-06-11T13:21:54.618 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:48 ago on Tue 11 Jun 2024 01:20:06 PM UTC. 2024-06-11T13:21:54.699 INFO:teuthology.orchestra.run.smithi107.stdout:Package python3-cephfs-2:19.0.0-2976.g3f51ea39.el9.x86_64 is already installed. 2024-06-11T13:21:54.741 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-06-11T13:21:54.742 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-06-11T13:21:54.742 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-06-11T13:21:54.805 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install python3-rbd 2024-06-11T13:21:55.377 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:49 ago on Tue 11 Jun 2024 01:20:06 PM UTC. 2024-06-11T13:21:55.457 INFO:teuthology.orchestra.run.smithi107.stdout:Package python3-rbd-2:19.0.0-2976.g3f51ea39.el9.x86_64 is already installed. 2024-06-11T13:21:55.500 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-06-11T13:21:55.501 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-06-11T13:21:55.501 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-06-11T13:21:55.557 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install rbd-fuse 2024-06-11T13:21:56.134 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:50 ago on Tue 11 Jun 2024 01:20:06 PM UTC. 2024-06-11T13:21:56.258 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-06-11T13:21:56.259 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:21:56.259 INFO:teuthology.orchestra.run.smithi107.stdout: Package Architecture Version Repository Size 2024-06-11T13:21:56.259 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:21:56.259 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-06-11T13:21:56.259 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-fuse x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 86 k 2024-06-11T13:21:56.259 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:56.259 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-06-11T13:21:56.259 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:21:56.260 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-06-11T13:21:56.261 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:56.261 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 86 k 2024-06-11T13:21:56.262 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 231 k 2024-06-11T13:21:56.262 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-06-11T13:21:56.631 INFO:teuthology.orchestra.run.smithi107.stdout:rbd-fuse-19.0.0-2976.g3f51ea39.el9.x86_64.rpm 231 kB/s | 86 kB 00:00 2024-06-11T13:21:56.632 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-06-11T13:21:56.632 INFO:teuthology.orchestra.run.smithi107.stdout:Total 230 kB/s | 86 kB 00:00 2024-06-11T13:21:56.632 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-06-11T13:21:56.639 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-06-11T13:21:56.639 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-06-11T13:21:56.704 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-06-11T13:21:56.704 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-06-11T13:21:56.857 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-06-11T13:21:56.954 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-fuse-2:19.0.0-2976.g3f51ea39.el9.x86_64 1/1 2024-06-11T13:21:57.315 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: rbd-fuse-2:19.0.0-2976.g3f51ea39.el9.x86_64 1/1 2024-06-11T13:21:57.555 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-fuse-2:19.0.0-2976.g3f51ea39.el9.x86_64 1/1 2024-06-11T13:21:57.556 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:57.556 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-06-11T13:21:57.556 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-fuse-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:21:57.556 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:57.556 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-06-11T13:21:57.834 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install rbd-mirror 2024-06-11T13:21:58.433 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:52 ago on Tue 11 Jun 2024 01:20:06 PM UTC. 2024-06-11T13:21:58.560 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-06-11T13:21:58.561 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:21:58.561 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repo Size 2024-06-11T13:21:58.561 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:21:58.561 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-06-11T13:21:58.562 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-mirror x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 3.1 M 2024-06-11T13:21:58.562 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:58.562 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-06-11T13:21:58.562 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:21:58.562 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-06-11T13:21:58.562 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:21:58.562 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 3.1 M 2024-06-11T13:21:58.562 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 13 M 2024-06-11T13:21:58.562 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-06-11T13:21:59.879 INFO:teuthology.orchestra.run.smithi107.stdout:rbd-mirror-19.0.0-2976.g3f51ea39.el9.x86_64.rpm 2.3 MB/s | 3.1 MB 00:01 2024-06-11T13:21:59.879 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-06-11T13:21:59.880 INFO:teuthology.orchestra.run.smithi107.stdout:Total 2.3 MB/s | 3.1 MB 00:01 2024-06-11T13:21:59.880 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-06-11T13:21:59.891 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-06-11T13:21:59.891 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-06-11T13:21:59.977 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-06-11T13:21:59.977 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-06-11T13:22:00.408 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-06-11T13:22:00.477 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-mirror-2:19.0.0-2976.g3f51ea39.el9.x86_64 1/1 2024-06-11T13:22:00.502 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2976.g3f51ea39.el9.x86_64 1/1 2024-06-11T13:22:00.502 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-11T13:22:00.502 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-11T13:22:00.502 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-11T13:22:00.502 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-11T13:22:00.502 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:22:01.547 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-mirror-2:19.0.0-2976.g3f51ea39.el9.x86_64 1/1 2024-06-11T13:22:01.547 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:22:01.547 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-06-11T13:22:01.548 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-mirror-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:22:01.548 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:22:01.548 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-06-11T13:22:01.772 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install rbd-nbd 2024-06-11T13:22:02.361 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:56 ago on Tue 11 Jun 2024 01:20:06 PM UTC. 2024-06-11T13:22:02.490 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-06-11T13:22:02.491 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:22:02.491 INFO:teuthology.orchestra.run.smithi107.stdout: Package Architecture Version Repository Size 2024-06-11T13:22:02.491 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:22:02.491 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-06-11T13:22:02.491 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-nbd x86_64 2:19.0.0-2976.g3f51ea39.el9 ceph 170 k 2024-06-11T13:22:02.491 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:22:02.491 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-06-11T13:22:02.491 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-06-11T13:22:02.491 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-06-11T13:22:02.491 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:22:02.492 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 170 k 2024-06-11T13:22:02.492 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 494 k 2024-06-11T13:22:02.492 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-06-11T13:22:02.840 INFO:teuthology.orchestra.run.smithi107.stdout:rbd-nbd-19.0.0-2976.g3f51ea39.el9.x86_64.rpm 490 kB/s | 170 kB 00:00 2024-06-11T13:22:02.841 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-06-11T13:22:02.841 INFO:teuthology.orchestra.run.smithi107.stdout:Total 488 kB/s | 170 kB 00:00 2024-06-11T13:22:02.841 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-06-11T13:22:02.850 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-06-11T13:22:02.850 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-06-11T13:22:02.915 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-06-11T13:22:02.916 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-06-11T13:22:03.078 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-06-11T13:22:03.181 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-nbd-2:19.0.0-2976.g3f51ea39.el9.x86_64 1/1 2024-06-11T13:22:03.616 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: rbd-nbd-2:19.0.0-2976.g3f51ea39.el9.x86_64 1/1 2024-06-11T13:22:03.859 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-nbd-2:19.0.0-2976.g3f51ea39.el9.x86_64 1/1 2024-06-11T13:22:03.860 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:22:03.860 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-06-11T13:22:03.860 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-nbd-2:19.0.0-2976.g3f51ea39.el9.x86_64 2024-06-11T13:22:03.860 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:22:03.860 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-06-11T13:22:04.094 DEBUG:teuthology.parallel:result is None 2024-06-11T13:22:04.095 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3f51ea3944d39ca13aa40153c9a4ee298306679d 2024-06-11T13:22:04.403 DEBUG:teuthology.orchestra.run.smithi057:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-11T13:22:04.431 INFO:teuthology.orchestra.run.smithi057.stdout:19.0.0-2976.g3f51ea39.el9 2024-06-11T13:22:04.432 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2976.g3f51ea39.el9 2024-06-11T13:22:04.432 INFO:teuthology.task.install:The correct ceph version 19.0.0-2976.g3f51ea39 is installed. 2024-06-11T13:22:04.434 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3f51ea3944d39ca13aa40153c9a4ee298306679d 2024-06-11T13:22:04.573 DEBUG:teuthology.orchestra.run.smithi107:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-11T13:22:04.595 INFO:teuthology.orchestra.run.smithi107.stdout:19.0.0-2976.g3f51ea39.el9 2024-06-11T13:22:04.595 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2976.g3f51ea39.el9 2024-06-11T13:22:04.595 INFO:teuthology.task.install:The correct ceph version 19.0.0-2976.g3f51ea39 is installed. 2024-06-11T13:22:04.597 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-06-11T13:22:04.598 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-06-11T13:22:04.598 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-11T13:22:04.629 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-06-11T13:22:04.629 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-11T13:22:04.663 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-06-11T13:22:04.663 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-06-11T13:22:04.663 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/usr/bin/daemon-helper 2024-06-11T13:22:04.700 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-11T13:22:04.770 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-06-11T13:22:04.770 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/daemon-helper 2024-06-11T13:22:04.801 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-11T13:22:04.866 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-06-11T13:22:04.867 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-06-11T13:22:04.867 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-11T13:22:04.896 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-11T13:22:04.965 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-06-11T13:22:04.966 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-11T13:22:04.991 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-11T13:22:05.055 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-06-11T13:22:05.055 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-06-11T13:22:05.055 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/usr/bin/stdin-killer 2024-06-11T13:22:05.081 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-11T13:22:05.147 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-06-11T13:22:05.147 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/stdin-killer 2024-06-11T13:22:05.171 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-11T13:22:05.235 INFO:teuthology.run_tasks:Running task cephadm... 2024-06-11T13:22:05.328 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'client': {'debug ms': 1}, 'global': {'mon election default strategy': 1, 'ms bind msgr2': False, 'ms type': 'async'}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'log-only-match': ['CEPHADM_'], 'mon_bind_msgr2': False, 'sha1': '3f51ea3944d39ca13aa40153c9a4ee298306679d', 'cephadm_mode': 'root'} 2024-06-11T13:22:05.328 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d 2024-06-11T13:22:05.328 INFO:tasks.cephadm:Cluster fsid is 98a6635e-27f5-11ef-bc9b-c7b262605968 2024-06-11T13:22:05.328 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-06-11T13:22:05.329 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '[v1:172.21.15.57:6789]', 'mon.c': '[v1:172.21.15.57:6790]', 'mon.b': '[v1:172.21.15.107:6789]'} 2024-06-11T13:22:05.329 INFO:tasks.cephadm:First mon is mon.a on smithi057 2024-06-11T13:22:05.329 INFO:tasks.cephadm:First mgr is y 2024-06-11T13:22:05.329 INFO:tasks.cephadm:Normalizing hostnames... 2024-06-11T13:22:05.329 DEBUG:teuthology.orchestra.run.smithi057:> sudo hostname $(hostname -s) 2024-06-11T13:22:05.355 DEBUG:teuthology.orchestra.run.smithi107:> sudo hostname $(hostname -s) 2024-06-11T13:22:05.380 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-06-11T13:22:05.380 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3f51ea3944d39ca13aa40153c9a4ee298306679d 2024-06-11T13:22:05.522 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '3f51ea3944d39ca13aa40153c9a4ee298306679d', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/55357/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2976-g3f51ea39', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2976.g3f51ea39'}, 'url': 'https://1.chacra.ceph.com/r/ceph/squid/3f51ea3944d39ca13aa40153c9a4ee298306679d/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-06-10 16:45:03.941060', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/squid/3f51ea3944d39ca13aa40153c9a4ee298306679d/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-06-11T13:22:05.663 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref squid, sha1 3f51ea3944d39ca13aa40153c9a4ee298306679d from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=3f51ea3944d39ca13aa40153c9a4ee298306679d 2024-06-11T13:22:05.664 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/squid/3f51ea3944d39ca13aa40153c9a4ee298306679d/centos/9/x86_64/flavors/default/cephadm 2024-06-11T13:22:05.665 DEBUG:teuthology.orchestra.run.smithi057:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/3f51ea3944d39ca13aa40153c9a4ee298306679d/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-11T13:22:06.181 INFO:teuthology.orchestra.run.smithi057.stdout:-rw-r--r--. 1 ubuntu ubuntu 781859 Jun 11 13:22 /home/ubuntu/cephtest/cephadm 2024-06-11T13:22:06.181 DEBUG:teuthology.orchestra.run.smithi107:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/3f51ea3944d39ca13aa40153c9a4ee298306679d/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-11T13:22:06.663 INFO:teuthology.orchestra.run.smithi107.stdout:-rw-r--r--. 1 ubuntu ubuntu 781859 Jun 11 13:22 /home/ubuntu/cephtest/cephadm 2024-06-11T13:22:06.663 DEBUG:teuthology.orchestra.run.smithi057:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-11T13:22:06.681 DEBUG:teuthology.orchestra.run.smithi107:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-11T13:22:06.711 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d on all hosts... 2024-06-11T13:22:06.711 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d pull 2024-06-11T13:22:06.723 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d pull 2024-06-11T13:22:07.557 INFO:teuthology.orchestra.run.smithi057.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d... 2024-06-11T13:22:07.698 INFO:teuthology.orchestra.run.smithi107.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d... 2024-06-11T13:22:42.864 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-06-11T13:22:42.864 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_version": "ceph version 19.0.0-2976-g3f51ea39 (3f51ea3944d39ca13aa40153c9a4ee298306679d) squid (rc)", 2024-06-11T13:22:42.864 INFO:teuthology.orchestra.run.smithi057.stdout: "image_id": "2d7459ec65a098c35c774154617f2893b0ad051f56ce89b72ab795f65a97fc22", 2024-06-11T13:22:42.864 INFO:teuthology.orchestra.run.smithi057.stdout: "repo_digests": [ 2024-06-11T13:22:42.864 INFO:teuthology.orchestra.run.smithi057.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310" 2024-06-11T13:22:42.864 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2024-06-11T13:22:42.864 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-06-11T13:22:43.499 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-06-11T13:22:43.499 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_version": "ceph version 19.0.0-2976-g3f51ea39 (3f51ea3944d39ca13aa40153c9a4ee298306679d) squid (rc)", 2024-06-11T13:22:43.499 INFO:teuthology.orchestra.run.smithi107.stdout: "image_id": "2d7459ec65a098c35c774154617f2893b0ad051f56ce89b72ab795f65a97fc22", 2024-06-11T13:22:43.499 INFO:teuthology.orchestra.run.smithi107.stdout: "repo_digests": [ 2024-06-11T13:22:43.499 INFO:teuthology.orchestra.run.smithi107.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310" 2024-06-11T13:22:43.499 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-06-11T13:22:43.499 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-06-11T13:22:43.526 DEBUG:teuthology.orchestra.run.smithi057:> sudo mkdir -p /etc/ceph 2024-06-11T13:22:43.561 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /etc/ceph 2024-06-11T13:22:43.590 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 777 /etc/ceph 2024-06-11T13:22:43.633 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 777 /etc/ceph 2024-06-11T13:22:43.659 INFO:tasks.cephadm:Writing seed config... 2024-06-11T13:22:43.660 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-06-11T13:22:43.660 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-06-11T13:22:43.660 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-06-11T13:22:43.660 INFO:tasks.cephadm: override: [global] mon election default strategy = 1 2024-06-11T13:22:43.660 INFO:tasks.cephadm: override: [global] ms bind msgr2 = False 2024-06-11T13:22:43.661 INFO:tasks.cephadm: override: [global] ms type = async 2024-06-11T13:22:43.661 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-06-11T13:22:43.661 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-06-11T13:22:43.661 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-06-11T13:22:43.661 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-06-11T13:22:43.661 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-06-11T13:22:43.661 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-06-11T13:22:43.661 INFO:tasks.cephadm: override: [osd] osd class default list = * 2024-06-11T13:22:43.661 INFO:tasks.cephadm: override: [osd] osd class load list = * 2024-06-11T13:22:43.661 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-06-11T13:22:43.662 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-06-11T13:22:43.662 DEBUG:teuthology.orchestra.run.smithi057:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-06-11T13:22:43.692 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 98a6635e-27f5-11ef-bc9b-c7b262605968 mon election default strategy = 1 ms bind msgr2 = False ms type = async [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 osd class default list = * osd class load list = * [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] debug ms = 1 2024-06-11T13:22:43.692 DEBUG:teuthology.orchestra.run.smithi057:mon.a> sudo journalctl -f -n 0 -u ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@mon.a.service 2024-06-11T13:22:43.735 DEBUG:teuthology.orchestra.run.smithi057:mgr.y> sudo journalctl -f -n 0 -u ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@mgr.y.service 2024-06-11T13:22:43.779 INFO:tasks.cephadm:Bootstrapping... 2024-06-11T13:22:43.779 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d -v bootstrap --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-addrv '[v1:172.21.15.57:6789]' --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-06-11T13:22:43.986 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2024-06-11T13:22:43.986 INFO:teuthology.orchestra.run.smithi057.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d', '-v', 'bootstrap', '--fsid', '98a6635e-27f5-11ef-bc9b-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-addrv', '[v1:172.21.15.57:6789]', '--skip-admin-label'] 2024-06-11T13:22:43.986 INFO:teuthology.orchestra.run.smithi057.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-06-11T13:22:43.987 INFO:teuthology.orchestra.run.smithi057.stdout:Verifying podman|docker is present... 2024-06-11T13:22:44.012 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stdout 5.0.2 2024-06-11T13:22:44.012 INFO:teuthology.orchestra.run.smithi057.stdout:Verifying lvm2 is present... 2024-06-11T13:22:44.012 INFO:teuthology.orchestra.run.smithi057.stdout:Verifying time synchronization is in place... 2024-06-11T13:22:44.020 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-06-11T13:22:44.020 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-06-11T13:22:44.026 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-06-11T13:22:44.027 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout inactive 2024-06-11T13:22:44.034 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout enabled 2024-06-11T13:22:44.040 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout active 2024-06-11T13:22:44.041 INFO:teuthology.orchestra.run.smithi057.stdout:Unit chronyd.service is enabled and running 2024-06-11T13:22:44.041 INFO:teuthology.orchestra.run.smithi057.stdout:Repeating the final host check... 2024-06-11T13:22:44.066 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stdout 5.0.2 2024-06-11T13:22:44.066 INFO:teuthology.orchestra.run.smithi057.stdout:podman (/bin/podman) version 5.0.2 is present 2024-06-11T13:22:44.066 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl is present 2024-06-11T13:22:44.066 INFO:teuthology.orchestra.run.smithi057.stdout:lvcreate is present 2024-06-11T13:22:44.073 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-06-11T13:22:44.073 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-06-11T13:22:44.079 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-06-11T13:22:44.079 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout inactive 2024-06-11T13:22:44.086 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout enabled 2024-06-11T13:22:44.093 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout active 2024-06-11T13:22:44.093 INFO:teuthology.orchestra.run.smithi057.stdout:Unit chronyd.service is enabled and running 2024-06-11T13:22:44.093 INFO:teuthology.orchestra.run.smithi057.stdout:Host looks OK 2024-06-11T13:22:44.093 INFO:teuthology.orchestra.run.smithi057.stdout:Cluster fsid: 98a6635e-27f5-11ef-bc9b-c7b262605968 2024-06-11T13:22:44.093 INFO:teuthology.orchestra.run.smithi057.stdout:Acquiring lock 140012805637504 on /run/cephadm/98a6635e-27f5-11ef-bc9b-c7b262605968.lock 2024-06-11T13:22:44.093 INFO:teuthology.orchestra.run.smithi057.stdout:Lock 140012805637504 acquired on /run/cephadm/98a6635e-27f5-11ef-bc9b-c7b262605968.lock 2024-06-11T13:22:44.094 INFO:teuthology.orchestra.run.smithi057.stdout:Verifying IP 172.21.15.57 port 6789 ... 2024-06-11T13:22:44.094 INFO:teuthology.orchestra.run.smithi057.stdout:Base mon IP(s) is [172.21.15.57:6789], mon addrv is [v1:172.21.15.57:6789] 2024-06-11T13:22:44.097 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.57 metric 100 2024-06-11T13:22:44.097 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.57 metric 100 2024-06-11T13:22:44.100 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-06-11T13:22:44.100 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-06-11T13:22:44.100 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-06-11T13:22:44.102 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-06-11T13:22:44.103 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-06-11T13:22:44.103 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-06-11T13:22:44.103 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-06-11T13:22:44.103 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1562/64 scope link noprefixroute 2024-06-11T13:22:44.103 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-06-11T13:22:44.104 INFO:teuthology.orchestra.run.smithi057.stdout:Mon IP `172.21.15.57` is in CIDR network `172.21.0.0/20` 2024-06-11T13:22:44.104 INFO:teuthology.orchestra.run.smithi057.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20'] 2024-06-11T13:22:44.104 INFO:teuthology.orchestra.run.smithi057.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-06-11T13:22:44.104 INFO:teuthology.orchestra.run.smithi057.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d... 2024-06-11T13:22:44.600 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stdout 2d7459ec65a098c35c774154617f2893b0ad051f56ce89b72ab795f65a97fc22 2024-06-11T13:22:44.601 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d... 2024-06-11T13:22:44.601 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stderr Getting image source signatures 2024-06-11T13:22:44.601 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stderr Copying blob sha256:1f67a7f98cdd57da92a966750bf1067cbf4164b90018f0203cbd8f2acee278bd 2024-06-11T13:22:44.601 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stderr Copying blob sha256:36850128dbaa55a7fd38bf12640a70ab57c121fdc76af02f72b423e059979691 2024-06-11T13:22:44.601 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stderr Copying config sha256:2d7459ec65a098c35c774154617f2893b0ad051f56ce89b72ab795f65a97fc22 2024-06-11T13:22:44.601 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stderr Writing manifest to image destination 2024-06-11T13:22:45.488 INFO:teuthology.orchestra.run.smithi057.stdout:ceph: stdout ceph version 19.0.0-2976-g3f51ea39 (3f51ea3944d39ca13aa40153c9a4ee298306679d) squid (rc) 2024-06-11T13:22:45.488 INFO:teuthology.orchestra.run.smithi057.stdout:Ceph version: ceph version 19.0.0-2976-g3f51ea39 (3f51ea3944d39ca13aa40153c9a4ee298306679d) squid (rc) 2024-06-11T13:22:45.488 INFO:teuthology.orchestra.run.smithi057.stdout:Extracting ceph user uid/gid from container image... 2024-06-11T13:22:46.376 INFO:teuthology.orchestra.run.smithi057.stdout:stat: stdout 167 167 2024-06-11T13:22:46.376 INFO:teuthology.orchestra.run.smithi057.stdout:Creating initial keys... 2024-06-11T13:22:47.289 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph-authtool: stdout AQCmT2hmhqG1KRAATjlY5yZtZu+BCqIzGf4/vw== 2024-06-11T13:22:48.094 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph-authtool: stdout AQCnT2hmXpMbJBAAlOdAkIFJo/rUk4pA90qYmw== 2024-06-11T13:22:49.033 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph-authtool: stdout AQCoT2hm23P1GhAAg5yJPpK6hNc3pnbzqMwSew== 2024-06-11T13:22:49.034 INFO:teuthology.orchestra.run.smithi057.stdout:Creating initial monmap... 2024-06-11T13:22:49.907 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-06-11T13:22:49.907 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-06-11T13:22:49.907 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 98a6635e-27f5-11ef-bc9b-c7b262605968 2024-06-11T13:22:49.907 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-06-11T13:22:49.907 INFO:teuthology.orchestra.run.smithi057.stdout:monmaptool for a [v1:172.21.15.57:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-06-11T13:22:49.907 INFO:teuthology.orchestra.run.smithi057.stdout:setting min_mon_release = quincy 2024-06-11T13:22:49.907 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/monmaptool: set fsid to 98a6635e-27f5-11ef-bc9b-c7b262605968 2024-06-11T13:22:49.908 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-06-11T13:22:49.908 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:22:49.908 INFO:teuthology.orchestra.run.smithi057.stdout:Creating mon... 2024-06-11T13:22:50.845 INFO:teuthology.orchestra.run.smithi057.stdout:create mon.a on 2024-06-11T13:22:51.076 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-06-11T13:22:51.244 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-06-11T13:22:51.438 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-98a6635e-27f5-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-98a6635e-27f5-11ef-bc9b-c7b262605968.target. 2024-06-11T13:22:51.438 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-98a6635e-27f5-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-98a6635e-27f5-11ef-bc9b-c7b262605968.target. 2024-06-11T13:22:51.687 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@mon.a 2024-06-11T13:22:51.687 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Failed to reset failed state of unit ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@mon.a.service: Unit ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@mon.a.service not loaded. 2024-06-11T13:22:51.859 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-98a6635e-27f5-11ef-bc9b-c7b262605968.target.wants/ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@mon.a.service → /etc/systemd/system/ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@.service. 2024-06-11T13:22:52.412 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-11T13:22:52.413 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout disabled 2024-06-11T13:22:52.420 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-11T13:22:52.420 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout inactive 2024-06-11T13:22:52.420 INFO:teuthology.orchestra.run.smithi057.stdout:firewalld.service is not enabled 2024-06-11T13:22:52.420 INFO:teuthology.orchestra.run.smithi057.stdout:Not possible to enable service . firewalld.service is not available 2024-06-11T13:22:52.420 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for mon to start... 2024-06-11T13:22:52.420 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for mon... 2024-06-11T13:22:53.528 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:53 smithi057 ceph-mon[29438]: from='client.? v1:172.21.15.57:0/2097639800' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-06-11T13:22:53.532 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout cluster: 2024-06-11T13:22:53.532 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout id: 98a6635e-27f5-11ef-bc9b-c7b262605968 2024-06-11T13:22:53.532 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-06-11T13:22:53.532 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-06-11T13:22:53.532 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout services: 2024-06-11T13:22:53.533 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.601194s) 2024-06-11T13:22:53.533 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-06-11T13:22:53.533 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-06-11T13:22:53.533 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-06-11T13:22:53.533 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout data: 2024-06-11T13:22:53.537 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-06-11T13:22:53.538 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-06-11T13:22:53.538 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-06-11T13:22:53.538 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout pgs: 2024-06-11T13:22:53.538 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-06-11T13:22:53.538 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:52.885+0000 7fbe62c59640 1 Processor -- start 2024-06-11T13:22:53.538 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:52.886+0000 7fbe62c59640 1 -- start start 2024-06-11T13:22:53.538 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:52.886+0000 7fbe62c59640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fbe5c078e30 con 0x7fbe5c106360 2024-06-11T13:22:53.538 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:52.886+0000 7fbe609f5640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fbe5c106360 0x7fbe5c108750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:57712/0 (socket says 172.21.15.57:57712) 2024-06-11T13:22:53.538 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:52.886+0000 7fbe609f5640 1 -- 172.21.15.57:0/3316127686 learned_addr learned my addr 172.21.15.57:0/3316127686 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:22:53.538 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:52.886+0000 7fbe5b7fe640 1 -- 172.21.15.57:0/3316127686 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1307013112 0 0) 0x7fbe5c078e30 con 0x7fbe5c106360 2024-06-11T13:22:53.538 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:52.886+0000 7fbe5b7fe640 1 -- 172.21.15.57:0/3316127686 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe48003520 con 0x7fbe5c106360 2024-06-11T13:22:53.538 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:52.887+0000 7fbe5b7fe640 1 -- 172.21.15.57:0/3316127686 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1450044988 0 0) 0x7fbe48003520 con 0x7fbe5c106360 2024-06-11T13:22:53.539 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:52.887+0000 7fbe5b7fe640 1 -- 172.21.15.57:0/3316127686 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe5c10ae80 con 0x7fbe5c106360 2024-06-11T13:22:53.539 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:52.887+0000 7fbe5b7fe640 1 -- 172.21.15.57:0/3316127686 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7fbe4c001580 con 0x7fbe5c106360 2024-06-11T13:22:53.539 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:52.887+0000 7fbe5b7fe640 1 -- 172.21.15.57:0/3316127686 <== mon.0 v1:172.21.15.57:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fbe4c001920 con 0x7fbe5c106360 2024-06-11T13:22:53.539 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:52.887+0000 7fbe5b7fe640 1 -- 172.21.15.57:0/3316127686 <== mon.0 v1:172.21.15.57:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7fbe4c001ca0 con 0x7fbe5c106360 2024-06-11T13:22:53.539 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:52.887+0000 7fbe62c59640 1 -- 172.21.15.57:0/3316127686 >> v1:172.21.15.57:6789/0 conn(0x7fbe5c106360 legacy=0x7fbe5c108750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:22:53.539 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:52.887+0000 7fbe62c59640 1 -- 172.21.15.57:0/3316127686 shutdown_connections 2024-06-11T13:22:53.539 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:52.887+0000 7fbe62c59640 1 -- 172.21.15.57:0/3316127686 >> 172.21.15.57:0/3316127686 conn(0x7fbe5c0ffcb0 msgr2=0x7fbe5c102130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:22:53.539 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:52.887+0000 7fbe62c59640 1 -- 172.21.15.57:0/3316127686 shutdown_connections 2024-06-11T13:22:53.539 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:52.887+0000 7fbe62c59640 1 -- 172.21.15.57:0/3316127686 wait complete. 2024-06-11T13:22:53.539 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:52.888+0000 7fbe62c59640 1 Processor -- start 2024-06-11T13:22:53.539 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:52.888+0000 7fbe62c59640 1 -- start start 2024-06-11T13:22:53.539 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:52.888+0000 7fbe62c59640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fbe5c19b520 con 0x7fbe5c106360 2024-06-11T13:22:53.539 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:52.888+0000 7fbe609f5640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fbe5c106360 0x7fbe5c19ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:57726/0 (socket says 172.21.15.57:57726) 2024-06-11T13:22:53.540 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:52.888+0000 7fbe609f5640 1 -- 172.21.15.57:0/2097639800 learned_addr learned my addr 172.21.15.57:0/2097639800 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:22:53.540 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:52.889+0000 7fbe59ffb640 1 -- 172.21.15.57:0/2097639800 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 209189856 0 0) 0x7fbe5c19b520 con 0x7fbe5c106360 2024-06-11T13:22:53.540 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:52.889+0000 7fbe59ffb640 1 -- 172.21.15.57:0/2097639800 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe48003160 con 0x7fbe5c106360 2024-06-11T13:22:53.540 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:52.889+0000 7fbe59ffb640 1 -- 172.21.15.57:0/2097639800 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1472115044 0 0) 0x7fbe48003160 con 0x7fbe5c106360 2024-06-11T13:22:53.540 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:52.889+0000 7fbe59ffb640 1 -- 172.21.15.57:0/2097639800 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fbe5c19b520 con 0x7fbe5c106360 2024-06-11T13:22:53.540 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:52.889+0000 7fbe59ffb640 1 -- 172.21.15.57:0/2097639800 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7fbe4c001480 con 0x7fbe5c106360 2024-06-11T13:22:53.540 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:52.889+0000 7fbe59ffb640 1 -- 172.21.15.57:0/2097639800 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3915174360 0 0) 0x7fbe5c19b520 con 0x7fbe5c106360 2024-06-11T13:22:53.540 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:52.889+0000 7fbe59ffb640 1 -- 172.21.15.57:0/2097639800 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe5c19b6f0 con 0x7fbe5c106360 2024-06-11T13:22:53.540 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:52.889+0000 7fbe62c59640 1 -- 172.21.15.57:0/2097639800 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fbe5c19ba00 con 0x7fbe5c106360 2024-06-11T13:22:53.540 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:52.889+0000 7fbe59ffb640 1 -- 172.21.15.57:0/2097639800 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fbe4c004c40 con 0x7fbe5c106360 2024-06-11T13:22:53.540 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:52.889+0000 7fbe59ffb640 1 -- 172.21.15.57:0/2097639800 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7fbe4c004fc0 con 0x7fbe5c106360 2024-06-11T13:22:53.540 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:52.889+0000 7fbe59ffb640 1 -- 172.21.15.57:0/2097639800 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 1) ==== 807+0+0 (unknown 3189685207 0 0) 0x7fbe4c0055f0 con 0x7fbe5c106360 2024-06-11T13:22:53.541 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:52.889+0000 7fbe62c59640 1 -- 172.21.15.57:0/2097639800 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fbe5c19f590 con 0x7fbe5c106360 2024-06-11T13:22:53.541 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:52.889+0000 7fbe59ffb640 1 -- 172.21.15.57:0/2097639800 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (unknown 3554970955 0 0) 0x7fbe4c005e50 con 0x7fbe5c106360 2024-06-11T13:22:53.541 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:52.890+0000 7fbe62c59640 1 -- 172.21.15.57:0/2097639800 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe5c10a000 con 0x7fbe5c106360 2024-06-11T13:22:53.541 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:52.893+0000 7fbe59ffb640 1 -- 172.21.15.57:0/2097639800 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (unknown 1092875540 0 2031569661) 0x7fbe4c0060d0 con 0x7fbe5c106360 2024-06-11T13:22:53.541 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:52.957+0000 7fbe62c59640 1 -- 172.21.15.57:0/2097639800 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "status"} v 0) -- 0x7fbe5c19fb60 con 0x7fbe5c106360 2024-06-11T13:22:53.541 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:52.958+0000 7fbe59ffb640 1 -- 172.21.15.57:0/2097639800 <== mon.0 v1:172.21.15.57:6789/0 10 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (unknown 1155462804 0 1489284701) 0x7fbe4c0062b0 con 0x7fbe5c106360 2024-06-11T13:22:53.541 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:52.959+0000 7fbe62c59640 1 -- 172.21.15.57:0/2097639800 >> v1:172.21.15.57:6789/0 conn(0x7fbe5c106360 legacy=0x7fbe5c19ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:22:53.541 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:52.959+0000 7fbe62c59640 1 -- 172.21.15.57:0/2097639800 shutdown_connections 2024-06-11T13:22:53.541 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:52.959+0000 7fbe62c59640 1 -- 172.21.15.57:0/2097639800 >> 172.21.15.57:0/2097639800 conn(0x7fbe5c0ffcb0 msgr2=0x7fbe5c105260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:22:53.541 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:52.959+0000 7fbe62c59640 1 -- 172.21.15.57:0/2097639800 shutdown_connections 2024-06-11T13:22:53.541 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:52.959+0000 7fbe62c59640 1 -- 172.21.15.57:0/2097639800 wait complete. 2024-06-11T13:22:53.541 INFO:teuthology.orchestra.run.smithi057.stdout:mon is available 2024-06-11T13:22:53.541 INFO:teuthology.orchestra.run.smithi057.stdout:Assimilating anything we can from ceph.conf... 2024-06-11T13:22:54.568 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:54 smithi057 ceph-mon[29438]: from='client.? v1:172.21.15.57:0/2334097636' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-06-11T13:22:54.568 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:54 smithi057 ceph-mon[29438]: from='client.? v1:172.21.15.57:0/2334097636' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-06-11T13:22:54.577 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-06-11T13:22:54.577 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout [global] 2024-06-11T13:22:54.577 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout fsid = 98a6635e-27f5-11ef-bc9b-c7b262605968 2024-06-11T13:22:54.577 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_host = [v1:172.21.15.57:6789] 2024-06-11T13:22:54.577 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-06-11T13:22:54.577 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-06-11T13:22:54.577 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-06-11T13:22:54.577 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-06-11T13:22:54.577 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-06-11T13:22:54.577 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout [mgr] 2024-06-11T13:22:54.578 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-06-11T13:22:54.578 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-06-11T13:22:54.578 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout [osd] 2024-06-11T13:22:54.578 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-06-11T13:22:54.578 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-06-11T13:22:54.578 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:53.929+0000 7f0f1a968640 1 Processor -- start 2024-06-11T13:22:54.578 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:53.929+0000 7f0f1a968640 1 -- start start 2024-06-11T13:22:54.578 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:53.930+0000 7f0f1a968640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f0f14108b80 con 0x7f0f14104750 2024-06-11T13:22:54.578 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:53.930+0000 7f0f13fff640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f0f14104750 0x7f0f14104b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:57732/0 (socket says 172.21.15.57:57732) 2024-06-11T13:22:54.578 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:53.930+0000 7f0f13fff640 1 -- 172.21.15.57:0/2733363826 learned_addr learned my addr 172.21.15.57:0/2733363826 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:22:54.578 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:53.930+0000 7f0f12ffd640 1 -- 172.21.15.57:0/2733363826 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2054529605 0 0) 0x7f0f14108b80 con 0x7f0f14104750 2024-06-11T13:22:54.578 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:53.930+0000 7f0f12ffd640 1 -- 172.21.15.57:0/2733363826 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0efc003520 con 0x7f0f14104750 2024-06-11T13:22:54.579 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:53.931+0000 7f0f12ffd640 1 -- 172.21.15.57:0/2733363826 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3441528028 0 0) 0x7f0efc003520 con 0x7f0f14104750 2024-06-11T13:22:54.579 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:53.931+0000 7f0f12ffd640 1 -- 172.21.15.57:0/2733363826 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f14109d60 con 0x7f0f14104750 2024-06-11T13:22:54.579 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:53.931+0000 7f0f12ffd640 1 -- 172.21.15.57:0/2733363826 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f0f04001580 con 0x7f0f14104750 2024-06-11T13:22:54.579 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:53.931+0000 7f0f12ffd640 1 -- 172.21.15.57:0/2733363826 <== mon.0 v1:172.21.15.57:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0f040018b0 con 0x7f0f14104750 2024-06-11T13:22:54.579 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:53.931+0000 7f0f12ffd640 1 -- 172.21.15.57:0/2733363826 <== mon.0 v1:172.21.15.57:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f0f04001c30 con 0x7f0f14104750 2024-06-11T13:22:54.579 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:53.931+0000 7f0f1a968640 1 -- 172.21.15.57:0/2733363826 >> v1:172.21.15.57:6789/0 conn(0x7f0f14104750 legacy=0x7f0f14104b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:22:54.579 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:53.931+0000 7f0f1a968640 1 -- 172.21.15.57:0/2733363826 shutdown_connections 2024-06-11T13:22:54.579 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:53.931+0000 7f0f1a968640 1 -- 172.21.15.57:0/2733363826 >> 172.21.15.57:0/2733363826 conn(0x7f0f140ffcb0 msgr2=0x7f0f14102130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:22:54.579 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:53.931+0000 7f0f1a968640 1 -- 172.21.15.57:0/2733363826 shutdown_connections 2024-06-11T13:22:54.579 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:53.931+0000 7f0f1a968640 1 -- 172.21.15.57:0/2733363826 wait complete. 2024-06-11T13:22:54.579 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:53.932+0000 7f0f1a968640 1 Processor -- start 2024-06-11T13:22:54.579 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:53.932+0000 7f0f1a968640 1 -- start start 2024-06-11T13:22:54.579 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:53.932+0000 7f0f1a968640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f0f141992a0 con 0x7f0f14104750 2024-06-11T13:22:54.580 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:53.933+0000 7f0f13fff640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f0f14104750 0x7f0f14198b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:57738/0 (socket says 172.21.15.57:57738) 2024-06-11T13:22:54.580 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:53.933+0000 7f0f13fff640 1 -- 172.21.15.57:0/2334097636 learned_addr learned my addr 172.21.15.57:0/2334097636 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:22:54.580 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:53.933+0000 7f0f117fa640 1 -- 172.21.15.57:0/2334097636 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 230061029 0 0) 0x7f0f141992a0 con 0x7f0f14104750 2024-06-11T13:22:54.580 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:53.933+0000 7f0f117fa640 1 -- 172.21.15.57:0/2334097636 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0efc003160 con 0x7f0f14104750 2024-06-11T13:22:54.580 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:53.933+0000 7f0f117fa640 1 -- 172.21.15.57:0/2334097636 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3727798366 0 0) 0x7f0efc003160 con 0x7f0f14104750 2024-06-11T13:22:54.580 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:53.933+0000 7f0f117fa640 1 -- 172.21.15.57:0/2334097636 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0f141992a0 con 0x7f0f14104750 2024-06-11T13:22:54.580 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:53.933+0000 7f0f117fa640 1 -- 172.21.15.57:0/2334097636 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f0f04001480 con 0x7f0f14104750 2024-06-11T13:22:54.580 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:53.933+0000 7f0f117fa640 1 -- 172.21.15.57:0/2334097636 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3209678868 0 0) 0x7f0f141992a0 con 0x7f0f14104750 2024-06-11T13:22:54.580 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:53.933+0000 7f0f117fa640 1 -- 172.21.15.57:0/2334097636 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f14199470 con 0x7f0f14104750 2024-06-11T13:22:54.580 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:53.933+0000 7f0f1a968640 1 -- 172.21.15.57:0/2334097636 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f0f14199780 con 0x7f0f14104750 2024-06-11T13:22:54.580 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:53.933+0000 7f0f1a968640 1 -- 172.21.15.57:0/2334097636 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f0f1419d310 con 0x7f0f14104750 2024-06-11T13:22:54.580 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:53.933+0000 7f0f117fa640 1 -- 172.21.15.57:0/2334097636 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0f04004bf0 con 0x7f0f14104750 2024-06-11T13:22:54.580 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:53.933+0000 7f0f117fa640 1 -- 172.21.15.57:0/2334097636 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f0f04004e20 con 0x7f0f14104750 2024-06-11T13:22:54.581 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:53.933+0000 7f0f117fa640 1 -- 172.21.15.57:0/2334097636 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 1) ==== 807+0+0 (unknown 3189685207 0 0) 0x7f0f040053d0 con 0x7f0f14104750 2024-06-11T13:22:54.581 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:53.933+0000 7f0f117fa640 1 -- 172.21.15.57:0/2334097636 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (unknown 3554970955 0 0) 0x7f0f04006750 con 0x7f0f14104750 2024-06-11T13:22:54.581 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:53.934+0000 7f0f1a968640 1 -- 172.21.15.57:0/2334097636 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f141923c0 con 0x7f0f14104750 2024-06-11T13:22:54.581 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:53.936+0000 7f0f117fa640 1 -- 172.21.15.57:0/2334097636 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (unknown 1092875540 0 2031569661) 0x7f0f040058a0 con 0x7f0f14104750 2024-06-11T13:22:54.581 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:53.999+0000 7f0f1a968640 1 -- 172.21.15.57:0/2334097636 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f0f14108ee0 con 0x7f0f14104750 2024-06-11T13:22:54.581 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:54.002+0000 7f0f117fa640 1 -- 172.21.15.57:0/2334097636 <== mon.0 v1:172.21.15.57:6789/0 10 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+317 (unknown 1213389831 0 1434572610) 0x7f0f04005a80 con 0x7f0f14104750 2024-06-11T13:22:54.581 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:54.002+0000 7f0f117fa640 1 -- 172.21.15.57:0/2334097636 <== mon.0 v1:172.21.15.57:6789/0 11 ==== config(25 keys) ==== 1037+0+0 (unknown 2947037021 0 0) 0x7f0f04006190 con 0x7f0f14104750 2024-06-11T13:22:54.581 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:54.003+0000 7f0f1a968640 1 -- 172.21.15.57:0/2334097636 >> v1:172.21.15.57:6789/0 conn(0x7f0f14104750 legacy=0x7f0f14198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:22:54.581 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:54.003+0000 7f0f1a968640 1 -- 172.21.15.57:0/2334097636 shutdown_connections 2024-06-11T13:22:54.581 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:54.003+0000 7f0f1a968640 1 -- 172.21.15.57:0/2334097636 >> 172.21.15.57:0/2334097636 conn(0x7f0f140ffcb0 msgr2=0x7f0f14078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:22:54.581 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:54.003+0000 7f0f1a968640 1 -- 172.21.15.57:0/2334097636 shutdown_connections 2024-06-11T13:22:54.581 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:54.003+0000 7f0f1a968640 1 -- 172.21.15.57:0/2334097636 wait complete. 2024-06-11T13:22:54.582 INFO:teuthology.orchestra.run.smithi057.stdout:Generating new minimal ceph.conf... 2024-06-11T13:22:55.648 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:55 smithi057 ceph-mon[29438]: from='client.? v1:172.21.15.57:0/3365424889' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:22:55.651 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:54.985+0000 7f547392d640 1 Processor -- start 2024-06-11T13:22:55.655 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:54.985+0000 7f547392d640 1 -- start start 2024-06-11T13:22:55.655 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:54.985+0000 7f547392d640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f546c07a390 con 0x7f546c07b790 2024-06-11T13:22:55.656 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:54.985+0000 7f54716c9640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f546c07b790 0x7f546c079bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:57740/0 (socket says 172.21.15.57:57740) 2024-06-11T13:22:55.656 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:54.985+0000 7f54716c9640 1 -- 172.21.15.57:0/3243488946 learned_addr learned my addr 172.21.15.57:0/3243488946 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:22:55.656 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:54.986+0000 7f545ffff640 1 -- 172.21.15.57:0/3243488946 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3313141732 0 0) 0x7f546c07a390 con 0x7f546c07b790 2024-06-11T13:22:55.656 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:54.986+0000 7f545ffff640 1 -- 172.21.15.57:0/3243488946 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5458003520 con 0x7f546c07b790 2024-06-11T13:22:55.656 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:54.986+0000 7f545ffff640 1 -- 172.21.15.57:0/3243488946 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1176589699 0 0) 0x7f5458003520 con 0x7f546c07b790 2024-06-11T13:22:55.656 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:54.986+0000 7f545ffff640 1 -- 172.21.15.57:0/3243488946 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f546c07a560 con 0x7f546c07b790 2024-06-11T13:22:55.656 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:54.986+0000 7f545ffff640 1 -- 172.21.15.57:0/3243488946 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f5468001580 con 0x7f546c07b790 2024-06-11T13:22:55.656 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:54.986+0000 7f545ffff640 1 -- 172.21.15.57:0/3243488946 <== mon.0 v1:172.21.15.57:6789/0 4 ==== config(25 keys) ==== 1037+0+0 (unknown 2947037021 0 0) 0x7f5468001c50 con 0x7f546c07b790 2024-06-11T13:22:55.656 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:54.986+0000 7f545ffff640 1 -- 172.21.15.57:0/3243488946 <== mon.0 v1:172.21.15.57:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f54680058d0 con 0x7f546c07b790 2024-06-11T13:22:55.656 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:54.987+0000 7f547392d640 1 -- 172.21.15.57:0/3243488946 >> v1:172.21.15.57:6789/0 conn(0x7f546c07b790 legacy=0x7f546c079bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:22:55.656 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:54.987+0000 7f547392d640 1 -- 172.21.15.57:0/3243488946 shutdown_connections 2024-06-11T13:22:55.657 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:54.987+0000 7f547392d640 1 -- 172.21.15.57:0/3243488946 >> 172.21.15.57:0/3243488946 conn(0x7f546c100e00 msgr2=0x7f546c103240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:22:55.657 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:54.987+0000 7f547392d640 1 -- 172.21.15.57:0/3243488946 shutdown_connections 2024-06-11T13:22:55.657 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:54.987+0000 7f547392d640 1 -- 172.21.15.57:0/3243488946 wait complete. 2024-06-11T13:22:55.657 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:54.987+0000 7f547392d640 1 Processor -- start 2024-06-11T13:22:55.657 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:54.987+0000 7f547392d640 1 -- start start 2024-06-11T13:22:55.657 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:54.988+0000 7f547392d640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f546c19bdb0 con 0x7f546c07b790 2024-06-11T13:22:55.657 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:54.988+0000 7f54716c9640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f546c07b790 0x7f546c19b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:57748/0 (socket says 172.21.15.57:57748) 2024-06-11T13:22:55.657 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:54.988+0000 7f54716c9640 1 -- 172.21.15.57:0/3365424889 learned_addr learned my addr 172.21.15.57:0/3365424889 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:22:55.657 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:54.988+0000 7f545e7fc640 1 -- 172.21.15.57:0/3365424889 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 79198030 0 0) 0x7f546c19bdb0 con 0x7f546c07b790 2024-06-11T13:22:55.657 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:54.988+0000 7f545e7fc640 1 -- 172.21.15.57:0/3365424889 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5458005460 con 0x7f546c07b790 2024-06-11T13:22:55.657 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:54.988+0000 7f545e7fc640 1 -- 172.21.15.57:0/3365424889 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1635661419 0 0) 0x7f5458005460 con 0x7f546c07b790 2024-06-11T13:22:55.657 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:54.988+0000 7f545e7fc640 1 -- 172.21.15.57:0/3365424889 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f546c19bdb0 con 0x7f546c07b790 2024-06-11T13:22:55.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:54.988+0000 7f545e7fc640 1 -- 172.21.15.57:0/3365424889 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f5468001080 con 0x7f546c07b790 2024-06-11T13:22:55.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:54.988+0000 7f545e7fc640 1 -- 172.21.15.57:0/3365424889 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 748864890 0 0) 0x7f546c19bdb0 con 0x7f546c07b790 2024-06-11T13:22:55.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:54.988+0000 7f545e7fc640 1 -- 172.21.15.57:0/3365424889 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f546c19bf80 con 0x7f546c07b790 2024-06-11T13:22:55.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:54.988+0000 7f547392d640 1 -- 172.21.15.57:0/3365424889 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f546c19c290 con 0x7f546c07b790 2024-06-11T13:22:55.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:54.989+0000 7f547392d640 1 -- 172.21.15.57:0/3365424889 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f546c19fe20 con 0x7f546c07b790 2024-06-11T13:22:55.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:54.989+0000 7f545e7fc640 1 -- 172.21.15.57:0/3365424889 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1037+0+0 (unknown 2947037021 0 0) 0x7f5468001b70 con 0x7f546c07b790 2024-06-11T13:22:55.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:54.989+0000 7f545e7fc640 1 -- 172.21.15.57:0/3365424889 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f5468005f20 con 0x7f546c07b790 2024-06-11T13:22:55.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:54.989+0000 7f545e7fc640 1 -- 172.21.15.57:0/3365424889 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 1) ==== 807+0+0 (unknown 3189685207 0 0) 0x7f54680064d0 con 0x7f546c07b790 2024-06-11T13:22:55.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:54.989+0000 7f545e7fc640 1 -- 172.21.15.57:0/3365424889 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (unknown 3554970955 0 0) 0x7f5468007c20 con 0x7f546c07b790 2024-06-11T13:22:55.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:54.989+0000 7f547392d640 1 -- 172.21.15.57:0/3365424889 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54500050b0 con 0x7f546c07b790 2024-06-11T13:22:55.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:54.992+0000 7f545e7fc640 1 -- 172.21.15.57:0/3365424889 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (unknown 1092875540 0 2031569661) 0x7f5468004ce0 con 0x7f546c07b790 2024-06-11T13:22:55.659 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:55.054+0000 7f547392d640 1 -- 172.21.15.57:0/3365424889 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f54500053a0 con 0x7f546c07b790 2024-06-11T13:22:55.659 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:55.055+0000 7f545e7fc640 1 -- 172.21.15.57:0/3365424889 <== mon.0 v1:172.21.15.57:6789/0 10 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+147 (unknown 1452402520 0 348010903) 0x7f5468004940 con 0x7f546c07b790 2024-06-11T13:22:55.659 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:55.055+0000 7f547392d640 1 -- 172.21.15.57:0/3365424889 >> v1:172.21.15.57:6789/0 conn(0x7f546c07b790 legacy=0x7f546c19b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:22:55.659 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:55.055+0000 7f547392d640 1 -- 172.21.15.57:0/3365424889 shutdown_connections 2024-06-11T13:22:55.659 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:55.055+0000 7f547392d640 1 -- 172.21.15.57:0/3365424889 >> 172.21.15.57:0/3365424889 conn(0x7f546c100e00 msgr2=0x7f546c109cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:22:55.659 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:55.055+0000 7f547392d640 1 -- 172.21.15.57:0/3365424889 shutdown_connections 2024-06-11T13:22:55.659 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:55.055+0000 7f547392d640 1 -- 172.21.15.57:0/3365424889 wait complete. 2024-06-11T13:22:55.659 INFO:teuthology.orchestra.run.smithi057.stdout:Restarting the monitor... 2024-06-11T13:22:55.935 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:55 smithi057 systemd[1]: Stopping Ceph mon.a for 98a6635e-27f5-11ef-bc9b-c7b262605968... 2024-06-11T13:22:56.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:55 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mon-a[29434]: 2024-06-11T13:22:55.940+0000 7f4587a88640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-11T13:22:56.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:55 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mon-a[29434]: 2024-06-11T13:22:55.940+0000 7f4587a88640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-06-11T13:22:56.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:56 smithi057 podman[29680]: 2024-06-11 13:22:56.054246189 +0000 UTC m=+0.221713494 container died b48fd1940574cf9e4b2d97b7de0b9c41b650eabc78483bc121366515ba54e1f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.35.3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-3f51ea3) 2024-06-11T13:22:56.552 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:56 smithi057 podman[29680]: 2024-06-11 13:22:56.382547627 +0000 UTC m=+0.550014922 container cleanup b48fd1940574cf9e4b2d97b7de0b9c41b650eabc78483bc121366515ba54e1f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.35.3, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-3f51ea3, GIT_CLEAN=True, org.label-schema.build-date=20240528) 2024-06-11T13:22:56.552 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:56 smithi057 bash[29680]: ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mon-a 2024-06-11T13:22:56.814 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:56 smithi057 podman[29694]: 2024-06-11 13:22:56.55850288 +0000 UTC m=+0.507954867 container remove b48fd1940574cf9e4b2d97b7de0b9c41b650eabc78483bc121366515ba54e1f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mon-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.35.3, RELEASE=squid-3f51ea3, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240528, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-06-11T13:22:57.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:56 smithi057 systemd[1]: ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@mon.a.service: Deactivated successfully. 2024-06-11T13:22:57.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:56 smithi057 systemd[1]: Stopped Ceph mon.a for 98a6635e-27f5-11ef-bc9b-c7b262605968. 2024-06-11T13:22:57.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:56 smithi057 systemd[1]: Starting Ceph mon.a for 98a6635e-27f5-11ef-bc9b-c7b262605968... 2024-06-11T13:22:57.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 podman[29786]: 2024-06-11 13:22:57.034118156 +0000 UTC m=+0.110024799 container create 8ca4f226c1e690507897c41903d6ff1518ff2f5d0599ee12072d3940837d4a21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mon-a, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.35.3, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-3f51ea3, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-06-11T13:22:57.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 podman[29786]: 2024-06-11 13:22:56.941863652 +0000 UTC m=+0.017770295 image pull 2d7459ec65a098c35c774154617f2893b0ad051f56ce89b72ab795f65a97fc22 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d 2024-06-11T13:22:57.292 INFO:teuthology.orchestra.run.smithi057.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-06-11T13:22:57.406 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 podman[29786]: 2024-06-11 13:22:57.236517455 +0000 UTC m=+0.312424093 container init 8ca4f226c1e690507897c41903d6ff1518ff2f5d0599ee12072d3940837d4a21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mon-a, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-3f51ea3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.35.3, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-06-11T13:22:57.407 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 podman[29786]: 2024-06-11 13:22:57.239728862 +0000 UTC m=+0.315635497 container start 8ca4f226c1e690507897c41903d6ff1518ff2f5d0599ee12072d3940837d4a21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mon-a, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.35.3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-3f51ea3, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-06-11T13:22:57.407 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: set uid:gid to 167:167 (ceph:ceph) 2024-06-11T13:22:57.407 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: ceph version 19.0.0-2976-g3f51ea39 (3f51ea3944d39ca13aa40153c9a4ee298306679d) squid (rc), process ceph-mon, pid 2 2024-06-11T13:22:57.407 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: pidfile_write: ignore empty --pid-file 2024-06-11T13:22:57.407 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: load: jerasure load: lrc 2024-06-11T13:22:57.407 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: RocksDB version: 7.9.2 2024-06-11T13:22:57.407 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Git sha 0 2024-06-11T13:22:57.408 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Compile date 2024-06-10 15:42:49 2024-06-11T13:22:57.408 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: DB SUMMARY 2024-06-11T13:22:57.408 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: DB Session ID: M9Q8ZY99S9071CEUOLHE 2024-06-11T13:22:57.408 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: CURRENT file: CURRENT 2024-06-11T13:22:57.408 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: IDENTITY file: IDENTITY 2024-06-11T13:22:57.408 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-06-11T13:22:57.408 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-06-11T13:22:57.408 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 88849 ; 2024-06-11T13:22:57.408 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.error_if_exists: 0 2024-06-11T13:22:57.408 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.create_if_missing: 0 2024-06-11T13:22:57.408 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.paranoid_checks: 1 2024-06-11T13:22:57.408 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.flush_verify_memtable_count: 1 2024-06-11T13:22:57.408 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-06-11T13:22:57.408 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-06-11T13:22:57.409 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.env: 0x559cba4a5740 2024-06-11T13:22:57.409 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.fs: PosixFileSystem 2024-06-11T13:22:57.409 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.info_log: 0x559cbcbeafc0 2024-06-11T13:22:57.409 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.max_file_opening_threads: 16 2024-06-11T13:22:57.409 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.statistics: (nil) 2024-06-11T13:22:57.409 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.use_fsync: 0 2024-06-11T13:22:57.409 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.max_log_file_size: 0 2024-06-11T13:22:57.409 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-11T13:22:57.409 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.log_file_time_to_roll: 0 2024-06-11T13:22:57.409 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.keep_log_file_num: 1000 2024-06-11T13:22:57.409 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.recycle_log_file_num: 0 2024-06-11T13:22:57.409 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.allow_fallocate: 1 2024-06-11T13:22:57.409 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.allow_mmap_reads: 0 2024-06-11T13:22:57.409 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.allow_mmap_writes: 0 2024-06-11T13:22:57.410 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.use_direct_reads: 0 2024-06-11T13:22:57.410 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-11T13:22:57.410 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.create_missing_column_families: 0 2024-06-11T13:22:57.410 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.db_log_dir: 2024-06-11T13:22:57.410 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.wal_dir: 2024-06-11T13:22:57.410 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.table_cache_numshardbits: 6 2024-06-11T13:22:57.410 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.WAL_ttl_seconds: 0 2024-06-11T13:22:57.410 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.WAL_size_limit_MB: 0 2024-06-11T13:22:57.410 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-11T13:22:57.410 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-11T13:22:57.410 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.is_fd_close_on_exec: 1 2024-06-11T13:22:57.410 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.advise_random_on_open: 1 2024-06-11T13:22:57.410 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.db_write_buffer_size: 0 2024-06-11T13:22:57.410 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.write_buffer_manager: 0x559cbcbfa1e0 2024-06-11T13:22:57.411 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-11T13:22:57.411 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-11T13:22:57.411 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.use_adaptive_mutex: 0 2024-06-11T13:22:57.411 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.rate_limiter: (nil) 2024-06-11T13:22:57.411 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-11T13:22:57.411 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.wal_recovery_mode: 2 2024-06-11T13:22:57.411 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.enable_thread_tracking: 0 2024-06-11T13:22:57.411 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.enable_pipelined_write: 0 2024-06-11T13:22:57.411 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.unordered_write: 0 2024-06-11T13:22:57.411 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-11T13:22:57.411 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-11T13:22:57.411 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-11T13:22:57.411 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-11T13:22:57.411 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.row_cache: None 2024-06-11T13:22:57.411 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.wal_filter: None 2024-06-11T13:22:57.412 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-11T13:22:57.413 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.allow_ingest_behind: 0 2024-06-11T13:22:57.413 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.two_write_queues: 0 2024-06-11T13:22:57.413 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.manual_wal_flush: 0 2024-06-11T13:22:57.413 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.wal_compression: 0 2024-06-11T13:22:57.413 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.atomic_flush: 0 2024-06-11T13:22:57.413 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-11T13:22:57.413 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.persist_stats_to_disk: 0 2024-06-11T13:22:57.413 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.write_dbid_to_manifest: 0 2024-06-11T13:22:57.413 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.log_readahead_size: 0 2024-06-11T13:22:57.413 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-06-11T13:22:57.414 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.best_efforts_recovery: 0 2024-06-11T13:22:57.414 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-06-11T13:22:57.414 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-06-11T13:22:57.414 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.allow_data_in_errors: 0 2024-06-11T13:22:57.414 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.db_host_id: __hostname__ 2024-06-11T13:22:57.414 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.enforce_single_del_contracts: true 2024-06-11T13:22:57.414 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.max_background_jobs: 2 2024-06-11T13:22:57.414 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.max_background_compactions: -1 2024-06-11T13:22:57.414 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.max_subcompactions: 1 2024-06-11T13:22:57.414 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-11T13:22:57.414 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-11T13:22:57.415 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.delayed_write_rate : 16777216 2024-06-11T13:22:57.415 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.max_total_wal_size: 0 2024-06-11T13:22:57.415 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-11T13:22:57.415 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.stats_dump_period_sec: 600 2024-06-11T13:22:57.415 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.stats_persist_period_sec: 600 2024-06-11T13:22:57.415 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-11T13:22:57.415 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.max_open_files: -1 2024-06-11T13:22:57.415 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.bytes_per_sync: 0 2024-06-11T13:22:57.415 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.wal_bytes_per_sync: 0 2024-06-11T13:22:57.416 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.strict_bytes_per_sync: 0 2024-06-11T13:22:57.416 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.compaction_readahead_size: 0 2024-06-11T13:22:57.416 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.max_background_flushes: -1 2024-06-11T13:22:57.416 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Compression algorithms supported: 2024-06-11T13:22:57.416 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: kZSTD supported: 0 2024-06-11T13:22:57.416 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: kXpressCompression supported: 0 2024-06-11T13:22:57.416 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: kBZip2Compression supported: 0 2024-06-11T13:22:57.416 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-11T13:22:57.416 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: kLZ4Compression supported: 1 2024-06-11T13:22:57.416 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: kZlibCompression supported: 1 2024-06-11T13:22:57.417 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: kLZ4HCCompression supported: 1 2024-06-11T13:22:57.417 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: kSnappyCompression supported: 1 2024-06-11T13:22:57.417 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Fast CRC32 supported: Supported on x86 2024-06-11T13:22:57.417 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: DMutex implementation: pthread_mutex_t 2024-06-11T13:22:57.417 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-06-11T13:22:57.417 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-06-11T13:22:57.417 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-11T13:22:57.417 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.merge_operator: 2024-06-11T13:22:57.417 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.compaction_filter: None 2024-06-11T13:22:57.417 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.compaction_filter_factory: None 2024-06-11T13:22:57.417 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.sst_partitioner_factory: None 2024-06-11T13:22:57.417 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.memtable_factory: SkipListFactory 2024-06-11T13:22:57.417 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.table_factory: BlockBasedTable 2024-06-11T13:22:57.417 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x559cbcbeac00) 2024-06-11T13:22:57.418 INFO:journalctl@ceph.mon.a.smithi057.stdout: cache_index_and_filter_blocks: 1 2024-06-11T13:22:57.418 INFO:journalctl@ceph.mon.a.smithi057.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-11T13:22:57.418 INFO:journalctl@ceph.mon.a.smithi057.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-11T13:22:57.418 INFO:journalctl@ceph.mon.a.smithi057.stdout: pin_top_level_index_and_filter: 1 2024-06-11T13:22:57.418 INFO:journalctl@ceph.mon.a.smithi057.stdout: index_type: 0 2024-06-11T13:22:57.418 INFO:journalctl@ceph.mon.a.smithi057.stdout: data_block_index_type: 0 2024-06-11T13:22:57.418 INFO:journalctl@ceph.mon.a.smithi057.stdout: index_shortening: 1 2024-06-11T13:22:57.418 INFO:journalctl@ceph.mon.a.smithi057.stdout: data_block_hash_table_util_ratio: 0.750000 2024-06-11T13:22:57.418 INFO:journalctl@ceph.mon.a.smithi057.stdout: checksum: 4 2024-06-11T13:22:57.418 INFO:journalctl@ceph.mon.a.smithi057.stdout: no_block_cache: 0 2024-06-11T13:22:57.418 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_cache: 0x559cbcbe1090 2024-06-11T13:22:57.418 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_cache_name: BinnedLRUCache 2024-06-11T13:22:57.418 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_cache_options: 2024-06-11T13:22:57.418 INFO:journalctl@ceph.mon.a.smithi057.stdout: capacity : 536870912 2024-06-11T13:22:57.419 INFO:journalctl@ceph.mon.a.smithi057.stdout: num_shard_bits : 4 2024-06-11T13:22:57.419 INFO:journalctl@ceph.mon.a.smithi057.stdout: strict_capacity_limit : 0 2024-06-11T13:22:57.419 INFO:journalctl@ceph.mon.a.smithi057.stdout: high_pri_pool_ratio: 0.000 2024-06-11T13:22:57.419 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_cache_compressed: (nil) 2024-06-11T13:22:57.419 INFO:journalctl@ceph.mon.a.smithi057.stdout: persistent_cache: (nil) 2024-06-11T13:22:57.419 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_size: 4096 2024-06-11T13:22:57.419 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_size_deviation: 10 2024-06-11T13:22:57.419 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_restart_interval: 16 2024-06-11T13:22:57.419 INFO:journalctl@ceph.mon.a.smithi057.stdout: index_block_restart_interval: 1 2024-06-11T13:22:57.419 INFO:journalctl@ceph.mon.a.smithi057.stdout: metadata_block_size: 4096 2024-06-11T13:22:57.419 INFO:journalctl@ceph.mon.a.smithi057.stdout: partition_filters: 0 2024-06-11T13:22:57.419 INFO:journalctl@ceph.mon.a.smithi057.stdout: use_delta_encoding: 1 2024-06-11T13:22:57.419 INFO:journalctl@ceph.mon.a.smithi057.stdout: filter_policy: bloomfilter 2024-06-11T13:22:57.419 INFO:journalctl@ceph.mon.a.smithi057.stdout: whole_key_filtering: 1 2024-06-11T13:22:57.419 INFO:journalctl@ceph.mon.a.smithi057.stdout: verify_compression: 0 2024-06-11T13:22:57.420 INFO:journalctl@ceph.mon.a.smithi057.stdout: read_amp_bytes_per_bit: 0 2024-06-11T13:22:57.420 INFO:journalctl@ceph.mon.a.smithi057.stdout: format_version: 5 2024-06-11T13:22:57.420 INFO:journalctl@ceph.mon.a.smithi057.stdout: enable_index_compression: 1 2024-06-11T13:22:57.420 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_align: 0 2024-06-11T13:22:57.420 INFO:journalctl@ceph.mon.a.smithi057.stdout: max_auto_readahead_size: 262144 2024-06-11T13:22:57.420 INFO:journalctl@ceph.mon.a.smithi057.stdout: prepopulate_block_cache: 0 2024-06-11T13:22:57.420 INFO:journalctl@ceph.mon.a.smithi057.stdout: initial_auto_readahead_size: 8192 2024-06-11T13:22:57.420 INFO:journalctl@ceph.mon.a.smithi057.stdout: num_file_reads_for_auto_readahead: 2 2024-06-11T13:22:57.420 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.write_buffer_size: 33554432 2024-06-11T13:22:57.420 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.max_write_buffer_number: 2 2024-06-11T13:22:57.420 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.compression: NoCompression 2024-06-11T13:22:57.420 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.bottommost_compression: Disabled 2024-06-11T13:22:57.420 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.prefix_extractor: nullptr 2024-06-11T13:22:57.420 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-11T13:22:57.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.num_levels: 7 2024-06-11T13:22:57.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-11T13:22:57.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-11T13:22:57.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-11T13:22:57.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-11T13:22:57.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-11T13:22:57.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-11T13:22:57.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-11T13:22:57.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-11T13:22:57.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-06-11T13:22:57.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-11T13:22:57.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-06-11T13:22:57.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-06-11T13:22:57.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.compression_opts.window_bits: -14 2024-06-11T13:22:57.421 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.compression_opts.level: 32767 2024-06-11T13:22:57.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.compression_opts.strategy: 0 2024-06-11T13:22:57.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-11T13:22:57.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-11T13:22:57.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-06-11T13:22:57.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-06-11T13:22:57.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.compression_opts.enabled: false 2024-06-11T13:22:57.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-06-11T13:22:57.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-11T13:22:57.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-11T13:22:57.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-11T13:22:57.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.target_file_size_base: 67108864 2024-06-11T13:22:57.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.target_file_size_multiplier: 1 2024-06-11T13:22:57.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-11T13:22:57.422 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-11T13:22:57.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-11T13:22:57.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-11T13:22:57.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-11T13:22:57.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-11T13:22:57.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-11T13:22:57.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-11T13:22:57.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-11T13:22:57.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-11T13:22:57.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-11T13:22:57.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-11T13:22:57.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-06-11T13:22:57.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.arena_block_size: 1048576 2024-06-11T13:22:57.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-11T13:22:57.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-11T13:22:57.423 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.disable_auto_compactions: 0 2024-06-11T13:22:57.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-11T13:22:57.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-11T13:22:57.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-11T13:22:57.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-11T13:22:57.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-11T13:22:57.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-11T13:22:57.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-11T13:22:57.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-11T13:22:57.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-11T13:22:57.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-11T13:22:57.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.table_properties_collectors: 2024-06-11T13:22:57.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.inplace_update_support: 0 2024-06-11T13:22:57.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.inplace_update_num_locks: 10000 2024-06-11T13:22:57.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-11T13:22:57.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-11T13:22:57.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.memtable_huge_page_size: 0 2024-06-11T13:22:57.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.bloom_locality: 0 2024-06-11T13:22:57.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.max_successive_merges: 0 2024-06-11T13:22:57.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.optimize_filters_for_hits: 0 2024-06-11T13:22:57.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.paranoid_file_checks: 0 2024-06-11T13:22:57.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.force_consistency_checks: 1 2024-06-11T13:22:57.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.report_bg_io_stats: 0 2024-06-11T13:22:57.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.ttl: 2592000 2024-06-11T13:22:57.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.periodic_compaction_seconds: 0 2024-06-11T13:22:57.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-06-11T13:22:57.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-06-11T13:22:57.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.enable_blob_files: false 2024-06-11T13:22:57.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.min_blob_size: 0 2024-06-11T13:22:57.425 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.blob_file_size: 268435456 2024-06-11T13:22:57.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.blob_compression_type: NoCompression 2024-06-11T13:22:57.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.enable_blob_garbage_collection: false 2024-06-11T13:22:57.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-06-11T13:22:57.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-06-11T13:22:57.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-06-11T13:22:57.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.blob_file_starting_level: 0 2024-06-11T13:22:57.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-06-11T13:22:57.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-06-11T13:22:57.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-06-11T13:22:57.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 09e71afd-2128-4881-897c-9781bb067201 2024-06-11T13:22:57.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: EVENT_LOG_v1 {"time_micros": 1718112177266258, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-06-11T13:22:57.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-06-11T13:22:57.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: EVENT_LOG_v1 {"time_micros": 1718112177267240, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 85132, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 270, "table_properties": {"data_size": 83311, "index_size": 231, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 10877, "raw_average_key_size": 47, "raw_value_size": 77210, "raw_average_value_size": 337, "num_data_blocks": 10, "num_entries": 229, "num_filter_entries": 229, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1718112177, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "09e71afd-2128-4881-897c-9781bb067201", "db_session_id": "M9Q8ZY99S9071CEUOLHE", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-06-11T13:22:57.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: EVENT_LOG_v1 {"time_micros": 1718112177267337, "job": 1, "event": "recovery_finished"} 2024-06-11T13:22:57.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-06-11T13:22:57.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-06-11T13:22:57.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x559cbcc0ee00 2024-06-11T13:22:57.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: DB pointer 0x559cbcd04000 2024-06-11T13:22:57.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-06-11T13:22:57.427 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-06-11T13:22:57.427 INFO:journalctl@ceph.mon.a.smithi057.stdout: ** DB Stats ** 2024-06-11T13:22:57.427 INFO:journalctl@ceph.mon.a.smithi057.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-11T13:22:57.427 INFO:journalctl@ceph.mon.a.smithi057.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-11T13:22:57.427 INFO:journalctl@ceph.mon.a.smithi057.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-11T13:22:57.427 INFO:journalctl@ceph.mon.a.smithi057.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-11T13:22:57.427 INFO:journalctl@ceph.mon.a.smithi057.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-11T13:22:57.427 INFO:journalctl@ceph.mon.a.smithi057.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-11T13:22:57.427 INFO:journalctl@ceph.mon.a.smithi057.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-11T13:22:57.427 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2024-06-11T13:22:57.428 INFO:journalctl@ceph.mon.a.smithi057.stdout: ** Compaction Stats [default] ** 2024-06-11T13:22:57.428 INFO:journalctl@ceph.mon.a.smithi057.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-11T13:22:57.428 INFO:journalctl@ceph.mon.a.smithi057.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-06-11T13:22:57.428 INFO:journalctl@ceph.mon.a.smithi057.stdout: L0 2/0 84.94 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 129.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-11T13:22:57.428 INFO:journalctl@ceph.mon.a.smithi057.stdout: Sum 2/0 84.94 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 129.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-11T13:22:57.428 INFO:journalctl@ceph.mon.a.smithi057.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 129.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-11T13:22:57.428 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2024-06-11T13:22:57.428 INFO:journalctl@ceph.mon.a.smithi057.stdout: ** Compaction Stats [default] ** 2024-06-11T13:22:57.428 INFO:journalctl@ceph.mon.a.smithi057.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-11T13:22:57.428 INFO:journalctl@ceph.mon.a.smithi057.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-11T13:22:57.428 INFO:journalctl@ceph.mon.a.smithi057.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 129.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-11T13:22:57.428 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2024-06-11T13:22:57.428 INFO:journalctl@ceph.mon.a.smithi057.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-06-11T13:22:57.428 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2024-06-11T13:22:57.428 INFO:journalctl@ceph.mon.a.smithi057.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-11T13:22:57.429 INFO:journalctl@ceph.mon.a.smithi057.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-06-11T13:22:57.429 INFO:journalctl@ceph.mon.a.smithi057.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-11T13:22:57.429 INFO:journalctl@ceph.mon.a.smithi057.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-11T13:22:57.429 INFO:journalctl@ceph.mon.a.smithi057.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-11T13:22:57.429 INFO:journalctl@ceph.mon.a.smithi057.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-11T13:22:57.429 INFO:journalctl@ceph.mon.a.smithi057.stdout: Cumulative compaction: 0.00 GB write, 23.30 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-11T13:22:57.429 INFO:journalctl@ceph.mon.a.smithi057.stdout: Interval compaction: 0.00 GB write, 23.30 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-11T13:22:57.429 INFO:journalctl@ceph.mon.a.smithi057.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-11T13:22:57.429 INFO:journalctl@ceph.mon.a.smithi057.stdout: Block cache BinnedLRUCache@0x559cbcbe1090#2 capacity: 512.00 MB usage: 1.19 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-06-11T13:22:57.429 INFO:journalctl@ceph.mon.a.smithi057.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.77 KB,0.000146031%) IndexBlock(2,0.42 KB,8.04663e-05%) Misc(1,0.00 KB,0%) 2024-06-11T13:22:57.429 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2024-06-11T13:22:57.429 INFO:journalctl@ceph.mon.a.smithi057.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-11T13:22:57.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: starting mon.a rank 0 at public addrs v1:172.21.15.57:6789/0 at bind addrs v1:172.21.15.57:6789/0 mon_data /var/lib/ceph/mon/ceph-a fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 2024-06-11T13:22:57.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: mon.a@-1(???) e1 preinit fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 2024-06-11T13:22:57.429 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: mon.a@-1(???).mds e1 new map 2024-06-11T13:22:57.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: mon.a@-1(???).mds e1 print_map 2024-06-11T13:22:57.430 INFO:journalctl@ceph.mon.a.smithi057.stdout: e1 2024-06-11T13:22:57.430 INFO:journalctl@ceph.mon.a.smithi057.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-11T13:22:57.430 INFO:journalctl@ceph.mon.a.smithi057.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-11T13:22:57.430 INFO:journalctl@ceph.mon.a.smithi057.stdout: legacy client fscid: -1 2024-06-11T13:22:57.430 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2024-06-11T13:22:57.430 INFO:journalctl@ceph.mon.a.smithi057.stdout: No filesystems configured 2024-06-11T13:22:57.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-06-11T13:22:57.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-11T13:22:57.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-11T13:22:57.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-11T13:22:57.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-06-11T13:22:57.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 bash[29786]: 8ca4f226c1e690507897c41903d6ff1518ff2f5d0599ee12072d3940837d4a21 2024-06-11T13:22:57.430 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 systemd[1]: Started Ceph mon.a for 98a6635e-27f5-11ef-bc9b-c7b262605968. 2024-06-11T13:22:57.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: mon.a is new leader, mons a in quorum (ranks 0) 2024-06-11T13:22:57.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: monmap epoch 1 2024-06-11T13:22:57.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 2024-06-11T13:22:57.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: last_changed 2024-06-11T13:22:49.372656+0000 2024-06-11T13:22:57.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: created 2024-06-11T13:22:49.372656+0000 2024-06-11T13:22:57.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: min_mon_release 19 (squid) 2024-06-11T13:22:57.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: election_strategy: 1 2024-06-11T13:22:57.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: 0: v1:172.21.15.57:6789/0 mon.a 2024-06-11T13:22:57.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: fsmap 2024-06-11T13:22:57.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: osdmap e1: 0 total, 0 up, 0 in 2024-06-11T13:22:57.431 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:57 smithi057 ceph-mon[29805]: mgrmap e1: no daemons active 2024-06-11T13:22:58.381 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:57.756+0000 7f482b7cb640 1 Processor -- start 2024-06-11T13:22:58.384 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:57.756+0000 7f482b7cb640 1 -- start start 2024-06-11T13:22:58.384 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:57.756+0000 7f482b7cb640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f482407a390 con 0x7f482407b790 2024-06-11T13:22:58.384 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:57.757+0000 7f4829567640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f482407b790 0x7f4824079bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:57762/0 (socket says 172.21.15.57:57762) 2024-06-11T13:22:58.384 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:57.757+0000 7f4829567640 1 -- 172.21.15.57:0/3174907589 learned_addr learned my addr 172.21.15.57:0/3174907589 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:22:58.384 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:57.757+0000 7f481bfff640 1 -- 172.21.15.57:0/3174907589 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 982436363 0 0) 0x7f482407a390 con 0x7f482407b790 2024-06-11T13:22:58.384 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:57.757+0000 7f481bfff640 1 -- 172.21.15.57:0/3174907589 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4810003520 con 0x7f482407b790 2024-06-11T13:22:58.384 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:57.757+0000 7f481bfff640 1 -- 172.21.15.57:0/3174907589 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1568551067 0 0) 0x7f4810003520 con 0x7f482407b790 2024-06-11T13:22:58.384 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:57.757+0000 7f481bfff640 1 -- 172.21.15.57:0/3174907589 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f482407a560 con 0x7f482407b790 2024-06-11T13:22:58.384 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:57.757+0000 7f481bfff640 1 -- 172.21.15.57:0/3174907589 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f4820001580 con 0x7f482407b790 2024-06-11T13:22:58.384 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:57.758+0000 7f481bfff640 1 -- 172.21.15.57:0/3174907589 <== mon.0 v1:172.21.15.57:6789/0 4 ==== config(25 keys) ==== 1037+0+0 (unknown 2947037021 0 0) 0x7f4820001c50 con 0x7f482407b790 2024-06-11T13:22:58.384 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:57.758+0000 7f481bfff640 1 -- 172.21.15.57:0/3174907589 <== mon.0 v1:172.21.15.57:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f48200058d0 con 0x7f482407b790 2024-06-11T13:22:58.384 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:57.758+0000 7f482b7cb640 1 -- 172.21.15.57:0/3174907589 >> v1:172.21.15.57:6789/0 conn(0x7f482407b790 legacy=0x7f4824079bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:22:58.385 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:57.758+0000 7f482b7cb640 1 -- 172.21.15.57:0/3174907589 shutdown_connections 2024-06-11T13:22:58.385 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:57.758+0000 7f482b7cb640 1 -- 172.21.15.57:0/3174907589 >> 172.21.15.57:0/3174907589 conn(0x7f4824100e00 msgr2=0x7f4824103240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:22:58.385 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:57.758+0000 7f482b7cb640 1 -- 172.21.15.57:0/3174907589 shutdown_connections 2024-06-11T13:22:58.385 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:57.758+0000 7f482b7cb640 1 -- 172.21.15.57:0/3174907589 wait complete. 2024-06-11T13:22:58.385 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:57.759+0000 7f482b7cb640 1 Processor -- start 2024-06-11T13:22:58.385 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:57.759+0000 7f482b7cb640 1 -- start start 2024-06-11T13:22:58.385 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:57.759+0000 7f482b7cb640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f482419bdc0 con 0x7f482407b790 2024-06-11T13:22:58.385 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:57.759+0000 7f4829567640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f482407b790 0x7f482419b6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:57776/0 (socket says 172.21.15.57:57776) 2024-06-11T13:22:58.385 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:57.759+0000 7f4829567640 1 -- 172.21.15.57:0/3133146785 learned_addr learned my addr 172.21.15.57:0/3133146785 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:22:58.387 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:57.759+0000 7f481a7fc640 1 -- 172.21.15.57:0/3133146785 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3738363296 0 0) 0x7f482419bdc0 con 0x7f482407b790 2024-06-11T13:22:58.387 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:57.759+0000 7f481a7fc640 1 -- 172.21.15.57:0/3133146785 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4810005460 con 0x7f482407b790 2024-06-11T13:22:58.387 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:57.759+0000 7f481a7fc640 1 -- 172.21.15.57:0/3133146785 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3859096927 0 0) 0x7f4810005460 con 0x7f482407b790 2024-06-11T13:22:58.388 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:57.759+0000 7f481a7fc640 1 -- 172.21.15.57:0/3133146785 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f482419bdc0 con 0x7f482407b790 2024-06-11T13:22:58.388 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:57.759+0000 7f481a7fc640 1 -- 172.21.15.57:0/3133146785 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f4820001080 con 0x7f482407b790 2024-06-11T13:22:58.388 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:57.760+0000 7f481a7fc640 1 -- 172.21.15.57:0/3133146785 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 215191287 0 0) 0x7f482419bdc0 con 0x7f482407b790 2024-06-11T13:22:58.388 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:57.760+0000 7f481a7fc640 1 -- 172.21.15.57:0/3133146785 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f482419bf90 con 0x7f482407b790 2024-06-11T13:22:58.388 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:57.760+0000 7f482b7cb640 1 -- 172.21.15.57:0/3133146785 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f482419c2a0 con 0x7f482407b790 2024-06-11T13:22:58.388 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:57.760+0000 7f482b7cb640 1 -- 172.21.15.57:0/3133146785 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f482419fe30 con 0x7f482407b790 2024-06-11T13:22:58.388 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:57.760+0000 7f481a7fc640 1 -- 172.21.15.57:0/3133146785 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1037+0+0 (unknown 2947037021 0 0) 0x7f4820001bf0 con 0x7f482407b790 2024-06-11T13:22:58.388 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:57.760+0000 7f481a7fc640 1 -- 172.21.15.57:0/3133146785 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f4820005f90 con 0x7f482407b790 2024-06-11T13:22:58.388 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:57.760+0000 7f481a7fc640 1 -- 172.21.15.57:0/3133146785 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 1) ==== 807+0+0 (unknown 3189685207 0 0) 0x7f48200065c0 con 0x7f482407b790 2024-06-11T13:22:58.388 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:57.760+0000 7f481a7fc640 1 -- 172.21.15.57:0/3133146785 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (unknown 3554970955 0 0) 0x7f4820007d00 con 0x7f482407b790 2024-06-11T13:22:58.388 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:57.761+0000 7f482b7cb640 1 -- 172.21.15.57:0/3133146785 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48080050b0 con 0x7f482407b790 2024-06-11T13:22:58.388 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:57.763+0000 7f481a7fc640 1 -- 172.21.15.57:0/3133146785 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (unknown 1092875540 0 2031569661) 0x7f4820004d50 con 0x7f482407b790 2024-06-11T13:22:58.388 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:57.825+0000 7f482b7cb640 1 -- 172.21.15.57:0/3133146785 --> v1:172.21.15.57:6789/0 -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f48080053a0 con 0x7f482407b790 2024-06-11T13:22:58.388 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:57.829+0000 7f481a7fc640 1 -- 172.21.15.57:0/3133146785 <== mon.0 v1:172.21.15.57:6789/0 10 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (unknown 534127980 0 0) 0x7f4820004940 con 0x7f482407b790 2024-06-11T13:22:58.389 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:57.830+0000 7f482b7cb640 1 -- 172.21.15.57:0/3133146785 >> v1:172.21.15.57:6789/0 conn(0x7f482407b790 legacy=0x7f482419b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:22:58.389 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:57.830+0000 7f482b7cb640 1 -- 172.21.15.57:0/3133146785 shutdown_connections 2024-06-11T13:22:58.389 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:57.830+0000 7f482b7cb640 1 -- 172.21.15.57:0/3133146785 >> 172.21.15.57:0/3133146785 conn(0x7f4824100e00 msgr2=0x7f4824192360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:22:58.389 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:57.830+0000 7f482b7cb640 1 -- 172.21.15.57:0/3133146785 shutdown_connections 2024-06-11T13:22:58.389 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:57.830+0000 7f482b7cb640 1 -- 172.21.15.57:0/3133146785 wait complete. 2024-06-11T13:22:58.389 INFO:teuthology.orchestra.run.smithi057.stdout:Wrote config to /etc/ceph/ceph.conf 2024-06-11T13:22:58.410 INFO:teuthology.orchestra.run.smithi057.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-06-11T13:22:58.411 INFO:teuthology.orchestra.run.smithi057.stdout:Creating mgr... 2024-06-11T13:22:58.412 INFO:teuthology.orchestra.run.smithi057.stdout:Verifying port 0.0.0.0:9283 ... 2024-06-11T13:22:58.412 INFO:teuthology.orchestra.run.smithi057.stdout:Verifying port 0.0.0.0:8765 ... 2024-06-11T13:22:58.677 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@mgr.y 2024-06-11T13:22:58.677 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Failed to reset failed state of unit ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@mgr.y.service: Unit ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@mgr.y.service not loaded. 2024-06-11T13:22:58.831 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-98a6635e-27f5-11ef-bc9b-c7b262605968.target.wants/ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@mgr.y.service → /etc/systemd/system/ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@.service. 2024-06-11T13:22:59.159 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:58 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3133146785' entity='client.admin' 2024-06-11T13:22:59.363 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-11T13:22:59.366 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout disabled 2024-06-11T13:22:59.371 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-11T13:22:59.371 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout inactive 2024-06-11T13:22:59.371 INFO:teuthology.orchestra.run.smithi057.stdout:firewalld.service is not enabled 2024-06-11T13:22:59.371 INFO:teuthology.orchestra.run.smithi057.stdout:Not possible to enable service . firewalld.service is not available 2024-06-11T13:22:59.381 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-11T13:22:59.381 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout disabled 2024-06-11T13:22:59.390 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-11T13:22:59.390 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout inactive 2024-06-11T13:22:59.390 INFO:teuthology.orchestra.run.smithi057.stdout:firewalld.service is not enabled 2024-06-11T13:22:59.390 INFO:teuthology.orchestra.run.smithi057.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-06-11T13:22:59.390 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for mgr to start... 2024-06-11T13:22:59.390 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for mgr... 2024-06-11T13:22:59.491 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:22:59 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:22:59.411+0000 7fb3dac41040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-11T13:22:59.781 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:22:59 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:22:59.571+0000 7fb3dac41040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-11T13:22:59.781 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:22:59 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:22:59.646+0000 7fb3dac41040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-11T13:23:00.267 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:22:59 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2579997371' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-11T13:23:00.267 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:22:59 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:22:59.913+0000 7fb3dac41040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-11T13:23:00.267 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:22:59 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:22:59.976+0000 7fb3dac41040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-11T13:23:00.441 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-06-11T13:23:00.444 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout { 2024-06-11T13:23:00.444 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "fsid": "98a6635e-27f5-11ef-bc9b-c7b262605968", 2024-06-11T13:23:00.445 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "health": { 2024-06-11T13:23:00.445 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-11T13:23:00.445 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-11T13:23:00.445 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-11T13:23:00.445 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-06-11T13:23:00.445 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-11T13:23:00.445 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-11T13:23:00.445 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 0 2024-06-11T13:23:00.445 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2024-06-11T13:23:00.445 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-11T13:23:00.445 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "a" 2024-06-11T13:23:00.446 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2024-06-11T13:23:00.446 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-06-11T13:23:00.446 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-11T13:23:00.446 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-11T13:23:00.446 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-11T13:23:00.446 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-11T13:23:00.446 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-06-11T13:23:00.446 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-11T13:23:00.446 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-11T13:23:00.446 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-11T13:23:00.446 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-11T13:23:00.446 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-11T13:23:00.446 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-11T13:23:00.446 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-11T13:23:00.447 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-11T13:23:00.454 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-06-11T13:23:00.454 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-11T13:23:00.454 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-11T13:23:00.454 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-11T13:23:00.454 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-11T13:23:00.454 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-11T13:23:00.454 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-11T13:23:00.454 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-11T13:23:00.454 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-11T13:23:00.454 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-11T13:23:00.454 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-06-11T13:23:00.455 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-11T13:23:00.455 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-11T13:23:00.455 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-11T13:23:00.455 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-11T13:23:00.455 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-06-11T13:23:00.455 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-11T13:23:00.455 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-11T13:23:00.455 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-11T13:23:00.455 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-11T13:23:00.455 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-11T13:23:00.455 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-11T13:23:00.455 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "restful" 2024-06-11T13:23:00.456 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2024-06-11T13:23:00.456 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-11T13:23:00.456 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-06-11T13:23:00.456 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-11T13:23:00.456 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-11T13:23:00.456 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "modified": "2024-06-11T13:22:52.359125+0000", 2024-06-11T13:23:00.456 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-11T13:23:00.456 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-06-11T13:23:00.456 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-11T13:23:00.456 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout } 2024-06-11T13:23:00.456 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:59.797+0000 7ff9855c3640 1 Processor -- start 2024-06-11T13:23:00.456 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:59.798+0000 7ff9855c3640 1 -- start start 2024-06-11T13:23:00.457 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:59.798+0000 7ff9855c3640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ff980079720 con 0x7ff9801062c0 2024-06-11T13:23:00.457 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:59.798+0000 7ff97effd640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7ff9801062c0 0x7ff9801086b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:55158/0 (socket says 172.21.15.57:55158) 2024-06-11T13:23:00.457 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:59.798+0000 7ff97effd640 1 -- 172.21.15.57:0/1353186307 learned_addr learned my addr 172.21.15.57:0/1353186307 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:00.457 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:59.798+0000 7ff97dffb640 1 -- 172.21.15.57:0/1353186307 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 933846876 0 0) 0x7ff980079720 con 0x7ff9801062c0 2024-06-11T13:23:00.457 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:59.799+0000 7ff97dffb640 1 -- 172.21.15.57:0/1353186307 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff968003520 con 0x7ff9801062c0 2024-06-11T13:23:00.457 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:59.799+0000 7ff97dffb640 1 -- 172.21.15.57:0/1353186307 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1623881016 0 0) 0x7ff968003520 con 0x7ff9801062c0 2024-06-11T13:23:00.457 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:59.799+0000 7ff97dffb640 1 -- 172.21.15.57:0/1353186307 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff98010ade0 con 0x7ff9801062c0 2024-06-11T13:23:00.457 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:59.799+0000 7ff97dffb640 1 -- 172.21.15.57:0/1353186307 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7ff970001580 con 0x7ff9801062c0 2024-06-11T13:23:00.457 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:59.799+0000 7ff97dffb640 1 -- 172.21.15.57:0/1353186307 <== mon.0 v1:172.21.15.57:6789/0 4 ==== config(25 keys) ==== 1037+0+0 (unknown 2947037021 0 0) 0x7ff970001d20 con 0x7ff9801062c0 2024-06-11T13:23:00.457 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:59.799+0000 7ff97dffb640 1 -- 172.21.15.57:0/1353186307 <== mon.0 v1:172.21.15.57:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7ff9700058d0 con 0x7ff9801062c0 2024-06-11T13:23:00.457 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:59.800+0000 7ff9855c3640 1 -- 172.21.15.57:0/1353186307 >> v1:172.21.15.57:6789/0 conn(0x7ff9801062c0 legacy=0x7ff9801086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:00.457 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:59.800+0000 7ff9855c3640 1 -- 172.21.15.57:0/1353186307 shutdown_connections 2024-06-11T13:23:00.457 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:59.800+0000 7ff9855c3640 1 -- 172.21.15.57:0/1353186307 >> 172.21.15.57:0/1353186307 conn(0x7ff9800ffc70 msgr2=0x7ff9801020f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:00.458 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:59.800+0000 7ff9855c3640 1 -- 172.21.15.57:0/1353186307 shutdown_connections 2024-06-11T13:23:00.458 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:59.800+0000 7ff9855c3640 1 -- 172.21.15.57:0/1353186307 wait complete. 2024-06-11T13:23:00.458 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:59.800+0000 7ff9855c3640 1 Processor -- start 2024-06-11T13:23:00.458 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:59.801+0000 7ff9855c3640 1 -- start start 2024-06-11T13:23:00.458 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:59.801+0000 7ff9855c3640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ff98019f4f0 con 0x7ff9801062c0 2024-06-11T13:23:00.458 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:59.801+0000 7ff97effd640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7ff9801062c0 0x7ff980105b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:55172/0 (socket says 172.21.15.57:55172) 2024-06-11T13:23:00.458 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:59.801+0000 7ff97effd640 1 -- 172.21.15.57:0/2579997371 learned_addr learned my addr 172.21.15.57:0/2579997371 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:00.458 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:59.801+0000 7ff967fff640 1 -- 172.21.15.57:0/2579997371 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1924630094 0 0) 0x7ff98019f4f0 con 0x7ff9801062c0 2024-06-11T13:23:00.458 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:59.801+0000 7ff967fff640 1 -- 172.21.15.57:0/2579997371 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff968005410 con 0x7ff9801062c0 2024-06-11T13:23:00.458 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:59.801+0000 7ff967fff640 1 -- 172.21.15.57:0/2579997371 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4043947790 0 0) 0x7ff968005410 con 0x7ff9801062c0 2024-06-11T13:23:00.458 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:59.801+0000 7ff967fff640 1 -- 172.21.15.57:0/2579997371 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff98019f4f0 con 0x7ff9801062c0 2024-06-11T13:23:00.458 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:59.801+0000 7ff967fff640 1 -- 172.21.15.57:0/2579997371 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7ff970001080 con 0x7ff9801062c0 2024-06-11T13:23:00.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:59.802+0000 7ff967fff640 1 -- 172.21.15.57:0/2579997371 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 626889319 0 0) 0x7ff98019f4f0 con 0x7ff9801062c0 2024-06-11T13:23:00.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:59.802+0000 7ff967fff640 1 -- 172.21.15.57:0/2579997371 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9801a06d0 con 0x7ff9801062c0 2024-06-11T13:23:00.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:59.802+0000 7ff9855c3640 1 -- 172.21.15.57:0/2579997371 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7ff98019f6c0 con 0x7ff9801062c0 2024-06-11T13:23:00.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:59.802+0000 7ff9855c3640 1 -- 172.21.15.57:0/2579997371 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7ff98019fc00 con 0x7ff9801062c0 2024-06-11T13:23:00.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:59.802+0000 7ff967fff640 1 -- 172.21.15.57:0/2579997371 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1037+0+0 (unknown 2947037021 0 0) 0x7ff970001bc0 con 0x7ff9801062c0 2024-06-11T13:23:00.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:59.802+0000 7ff967fff640 1 -- 172.21.15.57:0/2579997371 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7ff970005e20 con 0x7ff9801062c0 2024-06-11T13:23:00.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:59.802+0000 7ff967fff640 1 -- 172.21.15.57:0/2579997371 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 1) ==== 807+0+0 (unknown 3189685207 0 0) 0x7ff970006450 con 0x7ff9801062c0 2024-06-11T13:23:00.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:59.802+0000 7ff967fff640 1 -- 172.21.15.57:0/2579997371 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (unknown 3554970955 0 0) 0x7ff970007bb0 con 0x7ff9801062c0 2024-06-11T13:23:00.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:59.803+0000 7ff9855c3640 1 -- 172.21.15.57:0/2579997371 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff96c005080 con 0x7ff9801062c0 2024-06-11T13:23:00.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:59.805+0000 7ff967fff640 1 -- 172.21.15.57:0/2579997371 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (unknown 1092875540 0 2031569661) 0x7ff970004a20 con 0x7ff9801062c0 2024-06-11T13:23:00.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:59.870+0000 7ff9855c3640 1 -- 172.21.15.57:0/2579997371 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ff96c005370 con 0x7ff9801062c0 2024-06-11T13:23:00.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:59.870+0000 7ff967fff640 1 -- 172.21.15.57:0/2579997371 <== mon.0 v1:172.21.15.57:6789/0 10 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (unknown 4201413639 0 2246936324) 0x7ff970005640 con 0x7ff9801062c0 2024-06-11T13:23:00.460 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:59.871+0000 7ff9855c3640 1 -- 172.21.15.57:0/2579997371 >> v1:172.21.15.57:6789/0 conn(0x7ff9801062c0 legacy=0x7ff980105b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:00.460 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:59.871+0000 7ff9855c3640 1 -- 172.21.15.57:0/2579997371 shutdown_connections 2024-06-11T13:23:00.460 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:59.871+0000 7ff9855c3640 1 -- 172.21.15.57:0/2579997371 >> 172.21.15.57:0/2579997371 conn(0x7ff9800ffc70 msgr2=0x7ff980190100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:00.460 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:59.871+0000 7ff9855c3640 1 -- 172.21.15.57:0/2579997371 shutdown_connections 2024-06-11T13:23:00.460 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:22:59.871+0000 7ff9855c3640 1 -- 172.21.15.57:0/2579997371 wait complete. 2024-06-11T13:23:00.460 INFO:teuthology.orchestra.run.smithi057.stdout:mgr not available, waiting (1/15)... 2024-06-11T13:23:01.280 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:00 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:00.980+0000 7fb3dac41040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-11T13:23:01.281 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:01 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:01.103+0000 7fb3dac41040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-11T13:23:01.281 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:01 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:01.163+0000 7fb3dac41040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-11T13:23:01.677 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:01 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:01.391+0000 7fb3dac41040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-11T13:23:01.677 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:01 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:01.456+0000 7fb3dac41040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-11T13:23:01.677 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:01 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:01.527+0000 7fb3dac41040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-11T13:23:01.677 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:01 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:01.683+0000 7fb3dac41040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-11T13:23:02.280 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:02 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:02.132+0000 7fb3dac41040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-11T13:23:02.780 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:02 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:02.473+0000 7fb3dac41040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-11T13:23:02.781 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:02 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:02.548+0000 7fb3dac41040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-11T13:23:02.781 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:02 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:02.711+0000 7fb3dac41040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-11T13:23:03.281 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:03 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:03.049+0000 7fb3dac41040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-11T13:23:03.281 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:03 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:03.115+0000 7fb3dac41040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-11T13:23:03.628 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:03 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:03.313+0000 7fb3dac41040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-11T13:23:03.629 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:03 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:03.473+0000 7fb3dac41040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-11T13:23:03.629 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:03 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:03.541+0000 7fb3dac41040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-11T13:23:03.629 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:03 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:03.630+0000 7fb3dac41040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-11T13:23:04.001 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:03 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:03.694+0000 7fb3dac41040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-11T13:23:04.270 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:04 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1707063593' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-11T13:23:04.531 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:04 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:04.276+0000 7fb3dac41040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-11T13:23:04.531 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:04 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:04.447+0000 7fb3dac41040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-11T13:23:05.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:05 smithi057 ceph-mon[29805]: Activating manager daemon y 2024-06-11T13:23:05.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:05 smithi057 ceph-mon[29805]: mgrmap e2: y(active, starting, since 0.00347756s) 2024-06-11T13:23:05.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:05 smithi057 ceph-mon[29805]: from='mgr.14100 v1:172.21.15.57:0/2399755768' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-11T13:23:05.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:05 smithi057 ceph-mon[29805]: from='mgr.14100 v1:172.21.15.57:0/2399755768' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-11T13:23:05.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:05 smithi057 ceph-mon[29805]: from='mgr.14100 v1:172.21.15.57:0/2399755768' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-11T13:23:05.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:05 smithi057 ceph-mon[29805]: from='mgr.14100 v1:172.21.15.57:0/2399755768' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-11T13:23:05.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:05 smithi057 ceph-mon[29805]: from='mgr.14100 v1:172.21.15.57:0/2399755768' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-11T13:23:05.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:05 smithi057 ceph-mon[29805]: from='mgr.14100 v1:172.21.15.57:0/2399755768' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-11T13:23:05.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:05 smithi057 ceph-mon[29805]: from='mgr.14100 v1:172.21.15.57:0/2399755768' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-11T13:23:05.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:05 smithi057 ceph-mon[29805]: from='mgr.14100 v1:172.21.15.57:0/2399755768' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-11T13:23:05.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:05 smithi057 ceph-mon[29805]: Manager daemon y is now available 2024-06-11T13:23:05.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:05 smithi057 ceph-mon[29805]: from='mgr.14100 v1:172.21.15.57:0/2399755768' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-11T13:23:05.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:05 smithi057 ceph-mon[29805]: from='mgr.14100 v1:172.21.15.57:0/2399755768' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-11T13:23:05.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:05 smithi057 ceph-mon[29805]: from='mgr.14100 v1:172.21.15.57:0/2399755768' entity='mgr.y' 2024-06-11T13:23:05.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:05 smithi057 ceph-mon[29805]: from='mgr.14100 v1:172.21.15.57:0/2399755768' entity='mgr.y' 2024-06-11T13:23:05.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:05 smithi057 ceph-mon[29805]: from='mgr.14100 v1:172.21.15.57:0/2399755768' entity='mgr.y' 2024-06-11T13:23:06.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:06 smithi057 ceph-mon[29805]: mgrmap e3: y(active, since 1.00726s) 2024-06-11T13:23:07.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:07 smithi057 ceph-mon[29805]: mgrmap e4: y(active, since 2s) 2024-06-11T13:23:10.284 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-06-11T13:23:10.284 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout { 2024-06-11T13:23:10.284 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "fsid": "98a6635e-27f5-11ef-bc9b-c7b262605968", 2024-06-11T13:23:10.285 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "health": { 2024-06-11T13:23:10.285 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-11T13:23:10.285 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-11T13:23:10.285 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-11T13:23:10.285 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-06-11T13:23:10.285 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-11T13:23:10.285 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-11T13:23:10.285 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 0 2024-06-11T13:23:10.285 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2024-06-11T13:23:10.286 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-11T13:23:10.286 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "a" 2024-06-11T13:23:10.286 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2024-06-11T13:23:10.286 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-06-11T13:23:10.286 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-11T13:23:10.286 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-11T13:23:10.286 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-11T13:23:10.286 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-11T13:23:10.286 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-06-11T13:23:10.286 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-11T13:23:10.287 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-11T13:23:10.287 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-11T13:23:10.287 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-11T13:23:10.287 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-11T13:23:10.287 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-11T13:23:10.287 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-11T13:23:10.287 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-11T13:23:10.287 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-06-11T13:23:10.287 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-11T13:23:10.287 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-11T13:23:10.287 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-11T13:23:10.287 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-11T13:23:10.287 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-11T13:23:10.287 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-11T13:23:10.287 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-11T13:23:10.288 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-11T13:23:10.288 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-11T13:23:10.288 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-06-11T13:23:10.288 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-11T13:23:10.288 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-11T13:23:10.288 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-11T13:23:10.288 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-11T13:23:10.288 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-06-11T13:23:10.288 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-11T13:23:10.289 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-11T13:23:10.289 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-11T13:23:10.289 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-11T13:23:10.289 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-11T13:23:10.289 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-11T13:23:10.289 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "restful" 2024-06-11T13:23:10.289 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2024-06-11T13:23:10.289 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-11T13:23:10.289 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-06-11T13:23:10.289 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-11T13:23:10.289 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-11T13:23:10.289 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "modified": "2024-06-11T13:22:52.359125+0000", 2024-06-11T13:23:10.289 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-11T13:23:10.290 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-06-11T13:23:10.290 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-11T13:23:10.290 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout } 2024-06-11T13:23:10.290 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:04.087+0000 7ff02f1fb640 1 Processor -- start 2024-06-11T13:23:10.290 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:04.087+0000 7ff02f1fb640 1 -- start start 2024-06-11T13:23:10.290 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:04.087+0000 7ff02f1fb640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ff028108b80 con 0x7ff028104750 2024-06-11T13:23:10.290 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:04.088+0000 7ff02cf97640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7ff028104750 0x7ff028104b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:55176/0 (socket says 172.21.15.57:55176) 2024-06-11T13:23:10.290 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:04.088+0000 7ff02cf97640 1 -- 172.21.15.57:0/1716795666 learned_addr learned my addr 172.21.15.57:0/1716795666 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:10.291 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:04.088+0000 7ff0277fe640 1 -- 172.21.15.57:0/1716795666 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2133324158 0 0) 0x7ff028108b80 con 0x7ff028104750 2024-06-11T13:23:10.291 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:04.088+0000 7ff0277fe640 1 -- 172.21.15.57:0/1716795666 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff018005730 con 0x7ff028104750 2024-06-11T13:23:10.291 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:04.088+0000 7ff0277fe640 1 -- 172.21.15.57:0/1716795666 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2536111752 0 0) 0x7ff018005730 con 0x7ff028104750 2024-06-11T13:23:10.291 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:04.088+0000 7ff0277fe640 1 -- 172.21.15.57:0/1716795666 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff028109d60 con 0x7ff028104750 2024-06-11T13:23:10.291 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:04.088+0000 7ff0277fe640 1 -- 172.21.15.57:0/1716795666 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7ff01c001580 con 0x7ff028104750 2024-06-11T13:23:10.291 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:04.088+0000 7ff0277fe640 1 -- 172.21.15.57:0/1716795666 <== mon.0 v1:172.21.15.57:6789/0 4 ==== config(25 keys) ==== 1037+0+0 (unknown 2947037021 0 0) 0x7ff01c001c50 con 0x7ff028104750 2024-06-11T13:23:10.291 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:04.088+0000 7ff0277fe640 1 -- 172.21.15.57:0/1716795666 <== mon.0 v1:172.21.15.57:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7ff01c0058d0 con 0x7ff028104750 2024-06-11T13:23:10.291 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:04.089+0000 7ff02f1fb640 1 -- 172.21.15.57:0/1716795666 >> v1:172.21.15.57:6789/0 conn(0x7ff028104750 legacy=0x7ff028104b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:10.291 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:04.089+0000 7ff02f1fb640 1 -- 172.21.15.57:0/1716795666 shutdown_connections 2024-06-11T13:23:10.292 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:04.089+0000 7ff02f1fb640 1 -- 172.21.15.57:0/1716795666 >> 172.21.15.57:0/1716795666 conn(0x7ff0280ffcb0 msgr2=0x7ff028102130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:10.292 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:04.089+0000 7ff02f1fb640 1 -- 172.21.15.57:0/1716795666 shutdown_connections 2024-06-11T13:23:10.292 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:04.089+0000 7ff02f1fb640 1 -- 172.21.15.57:0/1716795666 wait complete. 2024-06-11T13:23:10.292 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:04.090+0000 7ff02f1fb640 1 Processor -- start 2024-06-11T13:23:10.292 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:04.090+0000 7ff02f1fb640 1 -- start start 2024-06-11T13:23:10.292 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:04.090+0000 7ff02f1fb640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ff0281979a0 con 0x7ff028104750 2024-06-11T13:23:10.292 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:04.090+0000 7ff02cf97640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7ff028104750 0x7ff028197290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:55182/0 (socket says 172.21.15.57:55182) 2024-06-11T13:23:10.292 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:04.090+0000 7ff02cf97640 1 -- 172.21.15.57:0/1707063593 learned_addr learned my addr 172.21.15.57:0/1707063593 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:10.292 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:04.090+0000 7ff025ffb640 1 -- 172.21.15.57:0/1707063593 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4063742491 0 0) 0x7ff0281979a0 con 0x7ff028104750 2024-06-11T13:23:10.292 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:04.090+0000 7ff025ffb640 1 -- 172.21.15.57:0/1707063593 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff018007670 con 0x7ff028104750 2024-06-11T13:23:10.292 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:04.091+0000 7ff025ffb640 1 -- 172.21.15.57:0/1707063593 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2498625664 0 0) 0x7ff018007670 con 0x7ff028104750 2024-06-11T13:23:10.293 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:04.091+0000 7ff025ffb640 1 -- 172.21.15.57:0/1707063593 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff0281979a0 con 0x7ff028104750 2024-06-11T13:23:10.293 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:04.091+0000 7ff025ffb640 1 -- 172.21.15.57:0/1707063593 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7ff01c001080 con 0x7ff028104750 2024-06-11T13:23:10.293 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:04.091+0000 7ff025ffb640 1 -- 172.21.15.57:0/1707063593 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3511085121 0 0) 0x7ff0281979a0 con 0x7ff028104750 2024-06-11T13:23:10.293 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:04.091+0000 7ff025ffb640 1 -- 172.21.15.57:0/1707063593 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff028197b70 con 0x7ff028104750 2024-06-11T13:23:10.293 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:04.091+0000 7ff02f1fb640 1 -- 172.21.15.57:0/1707063593 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7ff028197e80 con 0x7ff028104750 2024-06-11T13:23:10.293 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:04.091+0000 7ff025ffb640 1 -- 172.21.15.57:0/1707063593 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1037+0+0 (unknown 2947037021 0 0) 0x7ff01c005860 con 0x7ff028104750 2024-06-11T13:23:10.293 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:04.091+0000 7ff025ffb640 1 -- 172.21.15.57:0/1707063593 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7ff01c006170 con 0x7ff028104750 2024-06-11T13:23:10.293 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:04.091+0000 7ff025ffb640 1 -- 172.21.15.57:0/1707063593 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 1) ==== 807+0+0 (unknown 3189685207 0 0) 0x7ff01c006760 con 0x7ff028104750 2024-06-11T13:23:10.293 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:04.092+0000 7ff02f1fb640 1 -- 172.21.15.57:0/1707063593 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7ff02819b9c0 con 0x7ff028104750 2024-06-11T13:23:10.293 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:04.092+0000 7ff025ffb640 1 -- 172.21.15.57:0/1707063593 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (unknown 3554970955 0 0) 0x7ff01c006c80 con 0x7ff028104750 2024-06-11T13:23:10.293 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:04.092+0000 7ff02f1fb640 1 -- 172.21.15.57:0/1707063593 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff028063000 con 0x7ff028104750 2024-06-11T13:23:10.294 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:04.094+0000 7ff025ffb640 1 -- 172.21.15.57:0/1707063593 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (unknown 1092875540 0 2031569661) 0x7ff01c0054f0 con 0x7ff028104750 2024-06-11T13:23:10.294 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:04.159+0000 7ff02f1fb640 1 -- 172.21.15.57:0/1707063593 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ff02819beb0 con 0x7ff028104750 2024-06-11T13:23:10.294 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:04.160+0000 7ff025ffb640 1 -- 172.21.15.57:0/1707063593 <== mon.0 v1:172.21.15.57:6789/0 10 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (unknown 4201413639 0 3147973383) 0x7ff01c005b10 con 0x7ff028104750 2024-06-11T13:23:10.294 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:04.161+0000 7ff02f1fb640 1 -- 172.21.15.57:0/1707063593 >> v1:172.21.15.57:6789/0 conn(0x7ff028104750 legacy=0x7ff028197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:10.294 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:04.161+0000 7ff02f1fb640 1 -- 172.21.15.57:0/1707063593 shutdown_connections 2024-06-11T13:23:10.294 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:04.161+0000 7ff02f1fb640 1 -- 172.21.15.57:0/1707063593 >> 172.21.15.57:0/1707063593 conn(0x7ff0280ffcb0 msgr2=0x7ff028102130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:10.294 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:04.161+0000 7ff02f1fb640 1 -- 172.21.15.57:0/1707063593 shutdown_connections 2024-06-11T13:23:10.294 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:04.161+0000 7ff02f1fb640 1 -- 172.21.15.57:0/1707063593 wait complete. 2024-06-11T13:23:10.294 INFO:teuthology.orchestra.run.smithi057.stdout:mgr not available, waiting (2/15)... 2024-06-11T13:23:13.243 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:13 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3430190351' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-11T13:23:13.556 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-06-11T13:23:13.556 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout { 2024-06-11T13:23:13.556 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "fsid": "98a6635e-27f5-11ef-bc9b-c7b262605968", 2024-06-11T13:23:13.556 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "health": { 2024-06-11T13:23:13.557 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-11T13:23:13.557 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-11T13:23:13.557 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-11T13:23:13.557 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-06-11T13:23:13.557 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-11T13:23:13.557 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-11T13:23:13.557 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 0 2024-06-11T13:23:13.557 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2024-06-11T13:23:13.557 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-11T13:23:13.557 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "a" 2024-06-11T13:23:13.557 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2024-06-11T13:23:13.557 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum_age": 15, 2024-06-11T13:23:13.557 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-11T13:23:13.558 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-11T13:23:13.558 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-11T13:23:13.558 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-11T13:23:13.558 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-06-11T13:23:13.558 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-11T13:23:13.558 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-11T13:23:13.558 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-11T13:23:13.558 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-11T13:23:13.558 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-11T13:23:13.558 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-11T13:23:13.558 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-11T13:23:13.565 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-11T13:23:13.565 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-06-11T13:23:13.565 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-11T13:23:13.565 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-11T13:23:13.566 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-11T13:23:13.566 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-11T13:23:13.566 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-11T13:23:13.566 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-11T13:23:13.566 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-11T13:23:13.566 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-11T13:23:13.566 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-11T13:23:13.566 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-06-11T13:23:13.566 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-11T13:23:13.566 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-11T13:23:13.566 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-11T13:23:13.606 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-11T13:23:13.607 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-06-11T13:23:13.607 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-11T13:23:13.607 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-11T13:23:13.607 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-11T13:23:13.607 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-11T13:23:13.607 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-11T13:23:13.607 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-11T13:23:13.607 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "restful" 2024-06-11T13:23:13.607 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2024-06-11T13:23:13.607 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-11T13:23:13.607 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-06-11T13:23:13.607 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-11T13:23:13.607 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-11T13:23:13.608 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "modified": "2024-06-11T13:22:52.359125+0000", 2024-06-11T13:23:13.608 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-11T13:23:13.608 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-06-11T13:23:13.608 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-11T13:23:13.608 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout } 2024-06-11T13:23:13.608 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:12.701+0000 7fb4132f3640 1 Processor -- start 2024-06-11T13:23:13.608 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:12.701+0000 7fb4132f3640 1 -- start start 2024-06-11T13:23:13.608 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:12.701+0000 7fb4132f3640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fb40c07a310 con 0x7fb40c07b710 2024-06-11T13:23:13.608 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:12.702+0000 7fb41108f640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fb40c07b710 0x7fb40c079b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:36440/0 (socket says 172.21.15.57:36440) 2024-06-11T13:23:13.608 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:12.702+0000 7fb41108f640 1 -- 172.21.15.57:0/3319869663 learned_addr learned my addr 172.21.15.57:0/3319869663 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:13.608 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:12.702+0000 7fb403fff640 1 -- 172.21.15.57:0/3319869663 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3751053971 0 0) 0x7fb40c07a310 con 0x7fb40c07b710 2024-06-11T13:23:13.608 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:12.702+0000 7fb403fff640 1 -- 172.21.15.57:0/3319869663 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb3f8003520 con 0x7fb40c07b710 2024-06-11T13:23:13.608 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:12.702+0000 7fb403fff640 1 -- 172.21.15.57:0/3319869663 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1087505401 0 0) 0x7fb3f8003520 con 0x7fb40c07b710 2024-06-11T13:23:13.608 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:12.702+0000 7fb403fff640 1 -- 172.21.15.57:0/3319869663 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb40c07a4e0 con 0x7fb40c07b710 2024-06-11T13:23:13.609 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:12.702+0000 7fb403fff640 1 -- 172.21.15.57:0/3319869663 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7fb408001580 con 0x7fb40c07b710 2024-06-11T13:23:13.609 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:12.702+0000 7fb403fff640 1 -- 172.21.15.57:0/3319869663 <== mon.0 v1:172.21.15.57:6789/0 4 ==== config(25 keys) ==== 1037+0+0 (unknown 2947037021 0 0) 0x7fb408001c50 con 0x7fb40c07b710 2024-06-11T13:23:13.609 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:12.702+0000 7fb403fff640 1 -- 172.21.15.57:0/3319869663 <== mon.0 v1:172.21.15.57:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7fb4080058d0 con 0x7fb40c07b710 2024-06-11T13:23:13.609 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:12.703+0000 7fb4132f3640 1 -- 172.21.15.57:0/3319869663 >> v1:172.21.15.57:6789/0 conn(0x7fb40c07b710 legacy=0x7fb40c079b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:13.609 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:12.703+0000 7fb4132f3640 1 -- 172.21.15.57:0/3319869663 shutdown_connections 2024-06-11T13:23:13.609 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:12.703+0000 7fb4132f3640 1 -- 172.21.15.57:0/3319869663 >> 172.21.15.57:0/3319869663 conn(0x7fb40c100e00 msgr2=0x7fb40c103240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:13.609 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:12.703+0000 7fb4132f3640 1 -- 172.21.15.57:0/3319869663 shutdown_connections 2024-06-11T13:23:13.609 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:12.703+0000 7fb4132f3640 1 -- 172.21.15.57:0/3319869663 wait complete. 2024-06-11T13:23:13.609 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:12.704+0000 7fb4132f3640 1 Processor -- start 2024-06-11T13:23:13.609 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:12.704+0000 7fb4132f3640 1 -- start start 2024-06-11T13:23:13.609 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:12.704+0000 7fb4132f3640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fb40c19bd80 con 0x7fb40c07b710 2024-06-11T13:23:13.609 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:12.704+0000 7fb41108f640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fb40c07b710 0x7fb40c19b670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:36454/0 (socket says 172.21.15.57:36454) 2024-06-11T13:23:13.609 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:12.704+0000 7fb41108f640 1 -- 172.21.15.57:0/3430190351 learned_addr learned my addr 172.21.15.57:0/3430190351 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:13.610 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:12.704+0000 7fb4027fc640 1 -- 172.21.15.57:0/3430190351 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1337844174 0 0) 0x7fb40c19bd80 con 0x7fb40c07b710 2024-06-11T13:23:13.610 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:12.704+0000 7fb4027fc640 1 -- 172.21.15.57:0/3430190351 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb3f8005460 con 0x7fb40c07b710 2024-06-11T13:23:13.610 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:12.705+0000 7fb4027fc640 1 -- 172.21.15.57:0/3430190351 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1420788116 0 0) 0x7fb3f8005460 con 0x7fb40c07b710 2024-06-11T13:23:13.610 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:12.705+0000 7fb4027fc640 1 -- 172.21.15.57:0/3430190351 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb40c19bd80 con 0x7fb40c07b710 2024-06-11T13:23:13.610 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:12.705+0000 7fb4027fc640 1 -- 172.21.15.57:0/3430190351 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7fb408001080 con 0x7fb40c07b710 2024-06-11T13:23:13.610 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:12.705+0000 7fb4027fc640 1 -- 172.21.15.57:0/3430190351 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2380898322 0 0) 0x7fb40c19bd80 con 0x7fb40c07b710 2024-06-11T13:23:13.610 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:12.705+0000 7fb4027fc640 1 -- 172.21.15.57:0/3430190351 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb40c19bf50 con 0x7fb40c07b710 2024-06-11T13:23:13.610 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:12.705+0000 7fb4132f3640 1 -- 172.21.15.57:0/3430190351 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fb40c19c260 con 0x7fb40c07b710 2024-06-11T13:23:13.610 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:12.705+0000 7fb4132f3640 1 -- 172.21.15.57:0/3430190351 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fb40c19fdf0 con 0x7fb40c07b710 2024-06-11T13:23:13.610 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:12.705+0000 7fb4027fc640 1 -- 172.21.15.57:0/3430190351 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1037+0+0 (unknown 2947037021 0 0) 0x7fb408001c10 con 0x7fb40c07b710 2024-06-11T13:23:13.610 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:12.705+0000 7fb4027fc640 1 -- 172.21.15.57:0/3430190351 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7fb408005fb0 con 0x7fb40c07b710 2024-06-11T13:23:13.610 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:12.706+0000 7fb4027fc640 1 -- 172.21.15.57:0/3430190351 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 4) ==== 46241+0+0 (unknown 3247358592 0 0) 0x7fb408007280 con 0x7fb40c07b710 2024-06-11T13:23:13.610 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:12.706+0000 7fb4132f3640 1 -- 172.21.15.57:0/3430190351 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3f00050b0 con 0x7fb40c07b710 2024-06-11T13:23:13.611 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:12.706+0000 7fb4027fc640 1 -- 172.21.15.57:0/3430190351 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (unknown 3554970955 0 0) 0x7fb408047750 con 0x7fb40c07b710 2024-06-11T13:23:13.611 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:12.711+0000 7fb4027fc640 1 -- 172.21.15.57:0/3430190351 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7fb4080174d0 con 0x7fb40c07b710 2024-06-11T13:23:13.611 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:12.967+0000 7fb4132f3640 1 -- 172.21.15.57:0/3430190351 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb3f00053a0 con 0x7fb40c07b710 2024-06-11T13:23:13.611 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:12.967+0000 7fb4027fc640 1 -- 172.21.15.57:0/3430190351 <== mon.0 v1:172.21.15.57:6789/0 10 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (unknown 4201413639 0 2741066483) 0x7fb408016dd0 con 0x7fb40c07b710 2024-06-11T13:23:13.611 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:12.970+0000 7fb4132f3640 1 -- 172.21.15.57:0/3430190351 >> v1:172.21.15.57:6800/262061228 conn(0x7fb3f803a3f0 legacy=0x7fb3f803c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:13.611 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:12.970+0000 7fb4132f3640 1 -- 172.21.15.57:0/3430190351 >> v1:172.21.15.57:6789/0 conn(0x7fb40c07b710 legacy=0x7fb40c19b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:13.611 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:12.970+0000 7fb4132f3640 1 -- 172.21.15.57:0/3430190351 shutdown_connections 2024-06-11T13:23:13.611 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:12.970+0000 7fb4132f3640 1 -- 172.21.15.57:0/3430190351 >> 172.21.15.57:0/3430190351 conn(0x7fb40c100e00 msgr2=0x7fb40c109d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:13.611 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:12.970+0000 7fb4132f3640 1 -- 172.21.15.57:0/3430190351 shutdown_connections 2024-06-11T13:23:13.611 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:12.970+0000 7fb4132f3640 1 -- 172.21.15.57:0/3430190351 wait complete. 2024-06-11T13:23:13.611 INFO:teuthology.orchestra.run.smithi057.stdout:mgr is available 2024-06-11T13:23:14.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:14 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3826266855' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-06-11T13:23:14.736 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-06-11T13:23:14.742 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout [global] 2024-06-11T13:23:14.742 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout fsid = 98a6635e-27f5-11ef-bc9b-c7b262605968 2024-06-11T13:23:14.742 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-06-11T13:23:14.742 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-06-11T13:23:14.742 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-06-11T13:23:14.742 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-06-11T13:23:14.742 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-06-11T13:23:14.742 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout [mgr] 2024-06-11T13:23:14.742 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-06-11T13:23:14.742 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-06-11T13:23:14.742 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout [osd] 2024-06-11T13:23:14.743 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-06-11T13:23:14.743 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-06-11T13:23:14.743 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:13.966+0000 7fc0af88f640 1 Processor -- start 2024-06-11T13:23:14.743 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:13.966+0000 7fc0af88f640 1 -- start start 2024-06-11T13:23:14.743 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:13.966+0000 7fc0af88f640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fc0a8078ed0 con 0x7fc0a81062c0 2024-06-11T13:23:14.743 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:13.967+0000 7fc0ad62b640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fc0a81062c0 0x7fc0a81086b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:36466/0 (socket says 172.21.15.57:36466) 2024-06-11T13:23:14.743 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:13.967+0000 7fc0ad62b640 1 -- 172.21.15.57:0/1645272702 learned_addr learned my addr 172.21.15.57:0/1645272702 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:14.743 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:13.967+0000 7fc09bfff640 1 -- 172.21.15.57:0/1645272702 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2777675416 0 0) 0x7fc0a8078ed0 con 0x7fc0a81062c0 2024-06-11T13:23:14.743 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:13.967+0000 7fc09bfff640 1 -- 172.21.15.57:0/1645272702 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc094003520 con 0x7fc0a81062c0 2024-06-11T13:23:14.743 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:13.967+0000 7fc09bfff640 1 -- 172.21.15.57:0/1645272702 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1262188770 0 0) 0x7fc094003520 con 0x7fc0a81062c0 2024-06-11T13:23:14.743 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:13.967+0000 7fc09bfff640 1 -- 172.21.15.57:0/1645272702 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0a810ade0 con 0x7fc0a81062c0 2024-06-11T13:23:14.743 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:13.967+0000 7fc09bfff640 1 -- 172.21.15.57:0/1645272702 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7fc0a4001580 con 0x7fc0a81062c0 2024-06-11T13:23:14.743 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:13.967+0000 7fc09bfff640 1 -- 172.21.15.57:0/1645272702 <== mon.0 v1:172.21.15.57:6789/0 4 ==== config(25 keys) ==== 1037+0+0 (unknown 2947037021 0 0) 0x7fc0a4001d20 con 0x7fc0a81062c0 2024-06-11T13:23:14.744 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:13.967+0000 7fc09bfff640 1 -- 172.21.15.57:0/1645272702 <== mon.0 v1:172.21.15.57:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7fc0a40058d0 con 0x7fc0a81062c0 2024-06-11T13:23:14.744 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:13.968+0000 7fc0af88f640 1 -- 172.21.15.57:0/1645272702 >> v1:172.21.15.57:6789/0 conn(0x7fc0a81062c0 legacy=0x7fc0a81086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:14.744 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:13.968+0000 7fc0af88f640 1 -- 172.21.15.57:0/1645272702 shutdown_connections 2024-06-11T13:23:14.744 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:13.968+0000 7fc0af88f640 1 -- 172.21.15.57:0/1645272702 >> 172.21.15.57:0/1645272702 conn(0x7fc0a80ffcb0 msgr2=0x7fc0a8102130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:14.744 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:13.968+0000 7fc0af88f640 1 -- 172.21.15.57:0/1645272702 shutdown_connections 2024-06-11T13:23:14.744 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:13.968+0000 7fc0af88f640 1 -- 172.21.15.57:0/1645272702 wait complete. 2024-06-11T13:23:14.744 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:13.969+0000 7fc0af88f640 1 Processor -- start 2024-06-11T13:23:14.744 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:13.969+0000 7fc0af88f640 1 -- start start 2024-06-11T13:23:14.744 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:13.969+0000 7fc0af88f640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fc0a8199b20 con 0x7fc0a81062c0 2024-06-11T13:23:14.744 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:13.969+0000 7fc0ad62b640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fc0a81062c0 0x7fc0a8199410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:36478/0 (socket says 172.21.15.57:36478) 2024-06-11T13:23:14.744 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:13.969+0000 7fc0ad62b640 1 -- 172.21.15.57:0/3826266855 learned_addr learned my addr 172.21.15.57:0/3826266855 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:14.744 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:13.969+0000 7fc09a7fc640 1 -- 172.21.15.57:0/3826266855 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 640483372 0 0) 0x7fc0a8199b20 con 0x7fc0a81062c0 2024-06-11T13:23:14.744 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:13.969+0000 7fc09a7fc640 1 -- 172.21.15.57:0/3826266855 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc094005410 con 0x7fc0a81062c0 2024-06-11T13:23:14.744 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:13.969+0000 7fc09a7fc640 1 -- 172.21.15.57:0/3826266855 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4014357697 0 0) 0x7fc094005410 con 0x7fc0a81062c0 2024-06-11T13:23:14.745 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:13.969+0000 7fc09a7fc640 1 -- 172.21.15.57:0/3826266855 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc0a8199b20 con 0x7fc0a81062c0 2024-06-11T13:23:14.745 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:13.970+0000 7fc09a7fc640 1 -- 172.21.15.57:0/3826266855 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7fc0a4001080 con 0x7fc0a81062c0 2024-06-11T13:23:14.745 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:13.970+0000 7fc09a7fc640 1 -- 172.21.15.57:0/3826266855 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 698118982 0 0) 0x7fc0a8199b20 con 0x7fc0a81062c0 2024-06-11T13:23:14.745 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:13.970+0000 7fc09a7fc640 1 -- 172.21.15.57:0/3826266855 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0a8199cf0 con 0x7fc0a81062c0 2024-06-11T13:23:14.745 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:13.970+0000 7fc0af88f640 1 -- 172.21.15.57:0/3826266855 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fc0a819a000 con 0x7fc0a81062c0 2024-06-11T13:23:14.745 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:13.970+0000 7fc0af88f640 1 -- 172.21.15.57:0/3826266855 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fc0a819db90 con 0x7fc0a81062c0 2024-06-11T13:23:14.745 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:13.970+0000 7fc09a7fc640 1 -- 172.21.15.57:0/3826266855 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1037+0+0 (unknown 2947037021 0 0) 0x7fc0a4004a20 con 0x7fc0a81062c0 2024-06-11T13:23:14.745 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:13.970+0000 7fc09a7fc640 1 -- 172.21.15.57:0/3826266855 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7fc0a4006200 con 0x7fc0a81062c0 2024-06-11T13:23:14.745 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:13.971+0000 7fc09a7fc640 1 -- 172.21.15.57:0/3826266855 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 4) ==== 46241+0+0 (unknown 3247358592 0 0) 0x7fc0a4011870 con 0x7fc0a81062c0 2024-06-11T13:23:14.745 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:13.971+0000 7fc0af88f640 1 -- 172.21.15.57:0/3826266855 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc08c0050b0 con 0x7fc0a81062c0 2024-06-11T13:23:14.745 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:13.971+0000 7fc09a7fc640 1 -- 172.21.15.57:0/3826266855 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (unknown 3554970955 0 0) 0x7fc0a40476e0 con 0x7fc0a81062c0 2024-06-11T13:23:14.745 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:13.979+0000 7fc09a7fc640 1 -- 172.21.15.57:0/3826266855 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7fc0a4017460 con 0x7fc0a81062c0 2024-06-11T13:23:14.745 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:14.166+0000 7fc0af88f640 1 -- 172.21.15.57:0/3826266855 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fc08c002420 con 0x7fc0a81062c0 2024-06-11T13:23:14.746 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:14.167+0000 7fc09a7fc640 1 -- 172.21.15.57:0/3826266855 <== mon.0 v1:172.21.15.57:6789/0 10 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+282 (unknown 1187553405 0 2042242438) 0x7fc0a4016d60 con 0x7fc0a81062c0 2024-06-11T13:23:14.746 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:14.170+0000 7fc0af88f640 1 -- 172.21.15.57:0/3826266855 >> v1:172.21.15.57:6800/262061228 conn(0x7fc09403a450 legacy=0x7fc09403c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:14.746 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:14.170+0000 7fc0af88f640 1 -- 172.21.15.57:0/3826266855 >> v1:172.21.15.57:6789/0 conn(0x7fc0a81062c0 legacy=0x7fc0a8199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:14.746 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:14.170+0000 7fc0af88f640 1 -- 172.21.15.57:0/3826266855 shutdown_connections 2024-06-11T13:23:14.746 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:14.170+0000 7fc0af88f640 1 -- 172.21.15.57:0/3826266855 >> 172.21.15.57:0/3826266855 conn(0x7fc0a80ffcb0 msgr2=0x7fc0a8190160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:14.746 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:14.170+0000 7fc0af88f640 1 -- 172.21.15.57:0/3826266855 shutdown_connections 2024-06-11T13:23:14.746 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:14.170+0000 7fc0af88f640 1 -- 172.21.15.57:0/3826266855 wait complete. 2024-06-11T13:23:14.746 INFO:teuthology.orchestra.run.smithi057.stdout:Enabling cephadm module... 2024-06-11T13:23:15.715 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:15 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3193983949' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-06-11T13:23:15.716 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:15 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ignoring --setuser ceph since I am not root 2024-06-11T13:23:15.716 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:15 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ignoring --setgroup ceph since I am not root 2024-06-11T13:23:15.716 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:15 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:15.565+0000 7fdc26ac2040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-11T13:23:16.030 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:15 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:15.728+0000 7fdc26ac2040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-11T13:23:16.031 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:15 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:15.800+0000 7fdc26ac2040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-11T13:23:16.305 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:15.121+0000 7f76a7ab2640 1 Processor -- start 2024-06-11T13:23:16.305 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:15.121+0000 7f76a7ab2640 1 -- start start 2024-06-11T13:23:16.305 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:15.121+0000 7f76a7ab2640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f76a007a390 con 0x7f76a007b790 2024-06-11T13:23:16.305 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:15.122+0000 7f76a584e640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f76a007b790 0x7f76a0079bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:36490/0 (socket says 172.21.15.57:36490) 2024-06-11T13:23:16.305 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:15.122+0000 7f76a584e640 1 -- 172.21.15.57:0/3678742325 learned_addr learned my addr 172.21.15.57:0/3678742325 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:16.305 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:15.122+0000 7f76a484c640 1 -- 172.21.15.57:0/3678742325 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 172105229 0 0) 0x7f76a007a390 con 0x7f76a007b790 2024-06-11T13:23:16.305 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:15.122+0000 7f76a484c640 1 -- 172.21.15.57:0/3678742325 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f768c003520 con 0x7f76a007b790 2024-06-11T13:23:16.305 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:15.122+0000 7f76a484c640 1 -- 172.21.15.57:0/3678742325 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1803743864 0 0) 0x7f768c003520 con 0x7f76a007b790 2024-06-11T13:23:16.305 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:15.122+0000 7f76a484c640 1 -- 172.21.15.57:0/3678742325 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a007a560 con 0x7f76a007b790 2024-06-11T13:23:16.305 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:15.122+0000 7f76a484c640 1 -- 172.21.15.57:0/3678742325 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f769c001580 con 0x7f76a007b790 2024-06-11T13:23:16.305 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:15.122+0000 7f76a484c640 1 -- 172.21.15.57:0/3678742325 <== mon.0 v1:172.21.15.57:6789/0 4 ==== config(25 keys) ==== 1037+0+0 (unknown 2947037021 0 0) 0x7f769c001c50 con 0x7f76a007b790 2024-06-11T13:23:16.306 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:15.122+0000 7f76a484c640 1 -- 172.21.15.57:0/3678742325 <== mon.0 v1:172.21.15.57:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f769c0058d0 con 0x7f76a007b790 2024-06-11T13:23:16.306 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:15.123+0000 7f76a7ab2640 1 -- 172.21.15.57:0/3678742325 >> v1:172.21.15.57:6789/0 conn(0x7f76a007b790 legacy=0x7f76a0079bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:16.306 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:15.123+0000 7f76a7ab2640 1 -- 172.21.15.57:0/3678742325 shutdown_connections 2024-06-11T13:23:16.306 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:15.123+0000 7f76a7ab2640 1 -- 172.21.15.57:0/3678742325 >> 172.21.15.57:0/3678742325 conn(0x7f76a0100e00 msgr2=0x7f76a0103240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:16.306 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:15.123+0000 7f76a7ab2640 1 -- 172.21.15.57:0/3678742325 shutdown_connections 2024-06-11T13:23:16.306 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:15.123+0000 7f76a7ab2640 1 -- 172.21.15.57:0/3678742325 wait complete. 2024-06-11T13:23:16.306 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:15.124+0000 7f76a7ab2640 1 Processor -- start 2024-06-11T13:23:16.306 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:15.124+0000 7f76a7ab2640 1 -- start start 2024-06-11T13:23:16.306 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:15.124+0000 7f76a7ab2640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f76a019bdc0 con 0x7f76a007b790 2024-06-11T13:23:16.307 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:15.124+0000 7f76a584e640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f76a007b790 0x7f76a019b6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:36504/0 (socket says 172.21.15.57:36504) 2024-06-11T13:23:16.307 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:15.124+0000 7f76a584e640 1 -- 172.21.15.57:0/3193983949 learned_addr learned my addr 172.21.15.57:0/3193983949 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:16.307 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:15.124+0000 7f7696ffd640 1 -- 172.21.15.57:0/3193983949 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1840901971 0 0) 0x7f76a019bdc0 con 0x7f76a007b790 2024-06-11T13:23:16.307 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:15.124+0000 7f7696ffd640 1 -- 172.21.15.57:0/3193983949 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f768c005460 con 0x7f76a007b790 2024-06-11T13:23:16.307 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:15.124+0000 7f7696ffd640 1 -- 172.21.15.57:0/3193983949 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3098972687 0 0) 0x7f768c005460 con 0x7f76a007b790 2024-06-11T13:23:16.307 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:15.124+0000 7f7696ffd640 1 -- 172.21.15.57:0/3193983949 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f76a019bdc0 con 0x7f76a007b790 2024-06-11T13:23:16.307 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:15.124+0000 7f7696ffd640 1 -- 172.21.15.57:0/3193983949 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f769c001080 con 0x7f76a007b790 2024-06-11T13:23:16.307 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:15.124+0000 7f7696ffd640 1 -- 172.21.15.57:0/3193983949 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3541863991 0 0) 0x7f76a019bdc0 con 0x7f76a007b790 2024-06-11T13:23:16.307 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:15.124+0000 7f7696ffd640 1 -- 172.21.15.57:0/3193983949 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a019bf90 con 0x7f76a007b790 2024-06-11T13:23:16.307 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:15.125+0000 7f76a7ab2640 1 -- 172.21.15.57:0/3193983949 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f76a019c2a0 con 0x7f76a007b790 2024-06-11T13:23:16.307 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:15.125+0000 7f76a7ab2640 1 -- 172.21.15.57:0/3193983949 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f76a019fe30 con 0x7f76a007b790 2024-06-11T13:23:16.307 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:15.125+0000 7f7696ffd640 1 -- 172.21.15.57:0/3193983949 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1037+0+0 (unknown 2947037021 0 0) 0x7f769c001b70 con 0x7f76a007b790 2024-06-11T13:23:16.307 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:15.125+0000 7f7696ffd640 1 -- 172.21.15.57:0/3193983949 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f769c005f20 con 0x7f76a007b790 2024-06-11T13:23:16.307 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:15.125+0000 7f7696ffd640 1 -- 172.21.15.57:0/3193983949 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 4) ==== 46241+0+0 (unknown 3247358592 0 0) 0x7f769c0115f0 con 0x7f76a007b790 2024-06-11T13:23:16.308 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:15.126+0000 7f76a7ab2640 1 -- 172.21.15.57:0/3193983949 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76840050b0 con 0x7f76a007b790 2024-06-11T13:23:16.308 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:15.126+0000 7f7696ffd640 1 -- 172.21.15.57:0/3193983949 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (unknown 3554970955 0 0) 0x7f769c0476f0 con 0x7f76a007b790 2024-06-11T13:23:16.308 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:15.130+0000 7f7696ffd640 1 -- 172.21.15.57:0/3193983949 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f769c017490 con 0x7f76a007b790 2024-06-11T13:23:16.308 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:15.364+0000 7f76a7ab2640 1 -- 172.21.15.57:0/3193983949 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f76840053a0 con 0x7f76a007b790 2024-06-11T13:23:16.308 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:15.416+0000 7f7696ffd640 1 -- 172.21.15.57:0/3193983949 <== mon.0 v1:172.21.15.57:6789/0 10 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (unknown 2263024820 0 0) 0x7f769c016d90 con 0x7f76a007b790 2024-06-11T13:23:16.308 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:15.417+0000 7f7696ffd640 1 -- 172.21.15.57:0/3193983949 <== mon.0 v1:172.21.15.57:6789/0 11 ==== mgrmap(e 5) ==== 46252+0+0 (unknown 2352721741 0 0) 0x7f769c0460e0 con 0x7f76a007b790 2024-06-11T13:23:16.308 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:15.419+0000 7f76a7ab2640 1 -- 172.21.15.57:0/3193983949 >> v1:172.21.15.57:6800/262061228 conn(0x7f768c03a3a0 legacy=0x7f768c03c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:16.308 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:15.419+0000 7f76a7ab2640 1 -- 172.21.15.57:0/3193983949 >> v1:172.21.15.57:6789/0 conn(0x7f76a007b790 legacy=0x7f76a019b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:16.308 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:15.419+0000 7f76a7ab2640 1 -- 172.21.15.57:0/3193983949 shutdown_connections 2024-06-11T13:23:16.308 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:15.419+0000 7f76a7ab2640 1 -- 172.21.15.57:0/3193983949 >> 172.21.15.57:0/3193983949 conn(0x7f76a0100e00 msgr2=0x7f76a0192360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:16.308 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:15.419+0000 7f76a7ab2640 1 -- 172.21.15.57:0/3193983949 shutdown_connections 2024-06-11T13:23:16.308 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:15.419+0000 7f76a7ab2640 1 -- 172.21.15.57:0/3193983949 wait complete. 2024-06-11T13:23:16.416 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:16 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:16.058+0000 7fdc26ac2040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-11T13:23:16.417 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:16 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:16.121+0000 7fdc26ac2040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-11T13:23:16.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:16 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3193983949' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-06-11T13:23:16.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:16 smithi057 ceph-mon[29805]: mgrmap e5: y(active, since 10s) 2024-06-11T13:23:17.531 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:17 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:17.124+0000 7fdc26ac2040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-11T13:23:17.531 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:17 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:17.250+0000 7fdc26ac2040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-11T13:23:17.531 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:17 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:17.311+0000 7fdc26ac2040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-11T13:23:17.827 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:17 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:17.540+0000 7fdc26ac2040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-11T13:23:17.827 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:17 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:17.605+0000 7fdc26ac2040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-11T13:23:17.827 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:17 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:17.676+0000 7fdc26ac2040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-11T13:23:17.827 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:17 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:17.832+0000 7fdc26ac2040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-11T13:23:18.531 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:18 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:18.282+0000 7fdc26ac2040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-11T13:23:18.849 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:18 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:18.623+0000 7fdc26ac2040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-11T13:23:18.849 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:18 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:18.697+0000 7fdc26ac2040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-11T13:23:19.174 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:18 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2992453745' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-06-11T13:23:19.174 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:18 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:18.859+0000 7fdc26ac2040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-11T13:23:19.449 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:19 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:19.208+0000 7fdc26ac2040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-11T13:23:19.449 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:19 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:19.274+0000 7fdc26ac2040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-11T13:23:19.458 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout { 2024-06-11T13:23:19.458 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-06-11T13:23:19.458 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-11T13:23:19.458 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-06-11T13:23:19.458 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-06-11T13:23:19.458 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout } 2024-06-11T13:23:19.458 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:18.649+0000 7fe7fa1b5640 1 Processor -- start 2024-06-11T13:23:19.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:18.649+0000 7fe7fa1b5640 1 -- start start 2024-06-11T13:23:19.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:18.649+0000 7fe7fa1b5640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fe7f4108b80 con 0x7fe7f4104750 2024-06-11T13:23:19.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:18.649+0000 7fe7f37fe640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fe7f4104750 0x7fe7f4104b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:36542/0 (socket says 172.21.15.57:36542) 2024-06-11T13:23:19.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:18.649+0000 7fe7f37fe640 1 -- 172.21.15.57:0/2045796033 learned_addr learned my addr 172.21.15.57:0/2045796033 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:19.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:18.650+0000 7fe7f27fc640 1 -- 172.21.15.57:0/2045796033 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3140216721 0 0) 0x7fe7f4108b80 con 0x7fe7f4104750 2024-06-11T13:23:19.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:18.650+0000 7fe7f27fc640 1 -- 172.21.15.57:0/2045796033 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe7dc003520 con 0x7fe7f4104750 2024-06-11T13:23:19.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:18.650+0000 7fe7f27fc640 1 -- 172.21.15.57:0/2045796033 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3355100616 0 0) 0x7fe7dc003520 con 0x7fe7f4104750 2024-06-11T13:23:19.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:18.650+0000 7fe7f27fc640 1 -- 172.21.15.57:0/2045796033 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f4109d60 con 0x7fe7f4104750 2024-06-11T13:23:19.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:18.650+0000 7fe7f27fc640 1 -- 172.21.15.57:0/2045796033 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7fe7e4001580 con 0x7fe7f4104750 2024-06-11T13:23:19.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:18.650+0000 7fe7f27fc640 1 -- 172.21.15.57:0/2045796033 <== mon.0 v1:172.21.15.57:6789/0 4 ==== config(25 keys) ==== 1037+0+0 (unknown 2947037021 0 0) 0x7fe7e4001d20 con 0x7fe7f4104750 2024-06-11T13:23:19.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:18.650+0000 7fe7f27fc640 1 -- 172.21.15.57:0/2045796033 <== mon.0 v1:172.21.15.57:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7fe7e40058d0 con 0x7fe7f4104750 2024-06-11T13:23:19.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:18.651+0000 7fe7fa1b5640 1 -- 172.21.15.57:0/2045796033 >> v1:172.21.15.57:6789/0 conn(0x7fe7f4104750 legacy=0x7fe7f4104b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:19.460 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:18.651+0000 7fe7fa1b5640 1 -- 172.21.15.57:0/2045796033 shutdown_connections 2024-06-11T13:23:19.460 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:18.651+0000 7fe7fa1b5640 1 -- 172.21.15.57:0/2045796033 >> 172.21.15.57:0/2045796033 conn(0x7fe7f40ffcb0 msgr2=0x7fe7f4102130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:19.460 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:18.651+0000 7fe7fa1b5640 1 -- 172.21.15.57:0/2045796033 shutdown_connections 2024-06-11T13:23:19.460 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:18.651+0000 7fe7fa1b5640 1 -- 172.21.15.57:0/2045796033 wait complete. 2024-06-11T13:23:19.460 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:18.651+0000 7fe7fa1b5640 1 Processor -- start 2024-06-11T13:23:19.460 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:18.652+0000 7fe7fa1b5640 1 -- start start 2024-06-11T13:23:19.460 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:18.652+0000 7fe7fa1b5640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fe7f419bd40 con 0x7fe7f4104750 2024-06-11T13:23:19.460 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:18.652+0000 7fe7f37fe640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fe7f4104750 0x7fe7f419b630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:36558/0 (socket says 172.21.15.57:36558) 2024-06-11T13:23:19.460 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:18.652+0000 7fe7f37fe640 1 -- 172.21.15.57:0/2992453745 learned_addr learned my addr 172.21.15.57:0/2992453745 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:19.460 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:18.652+0000 7fe7f0ff9640 1 -- 172.21.15.57:0/2992453745 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2215485370 0 0) 0x7fe7f419bd40 con 0x7fe7f4104750 2024-06-11T13:23:19.460 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:18.652+0000 7fe7f0ff9640 1 -- 172.21.15.57:0/2992453745 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe7dc005410 con 0x7fe7f4104750 2024-06-11T13:23:19.460 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:18.652+0000 7fe7f0ff9640 1 -- 172.21.15.57:0/2992453745 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 566436586 0 0) 0x7fe7dc005410 con 0x7fe7f4104750 2024-06-11T13:23:19.461 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:18.652+0000 7fe7f0ff9640 1 -- 172.21.15.57:0/2992453745 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe7f419bd40 con 0x7fe7f4104750 2024-06-11T13:23:19.461 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:18.652+0000 7fe7f0ff9640 1 -- 172.21.15.57:0/2992453745 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7fe7e40047e0 con 0x7fe7f4104750 2024-06-11T13:23:19.461 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:18.653+0000 7fe7f0ff9640 1 -- 172.21.15.57:0/2992453745 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4007110007 0 0) 0x7fe7f419bd40 con 0x7fe7f4104750 2024-06-11T13:23:19.461 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:18.653+0000 7fe7f0ff9640 1 -- 172.21.15.57:0/2992453745 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f419bf10 con 0x7fe7f4104750 2024-06-11T13:23:19.461 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:18.653+0000 7fe7fa1b5640 1 -- 172.21.15.57:0/2992453745 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fe7f419c220 con 0x7fe7f4104750 2024-06-11T13:23:19.461 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:18.653+0000 7fe7fa1b5640 1 -- 172.21.15.57:0/2992453745 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fe7f419fdb0 con 0x7fe7f4104750 2024-06-11T13:23:19.461 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:18.653+0000 7fe7f0ff9640 1 -- 172.21.15.57:0/2992453745 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1037+0+0 (unknown 2947037021 0 0) 0x7fe7e4001c00 con 0x7fe7f4104750 2024-06-11T13:23:19.461 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:18.653+0000 7fe7f0ff9640 1 -- 172.21.15.57:0/2992453745 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7fe7e4005e90 con 0x7fe7f4104750 2024-06-11T13:23:19.461 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:18.654+0000 7fe7f0ff9640 1 -- 172.21.15.57:0/2992453745 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 5) ==== 46252+0+0 (unknown 2352721741 0 0) 0x7fe7e40114f0 con 0x7fe7f4104750 2024-06-11T13:23:19.462 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:18.654+0000 7fe7fa1b5640 1 -- 172.21.15.57:0/2992453745 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7e0005080 con 0x7fe7f4104750 2024-06-11T13:23:19.462 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:18.654+0000 7fe7f2ffd640 1 -- 172.21.15.57:0/2992453745 >> v1:172.21.15.57:6800/262061228 conn(0x7fe7dc03a300 legacy=0x7fe7dc03c7c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.57:6800/262061228 2024-06-11T13:23:19.462 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:18.654+0000 7fe7f0ff9640 1 -- 172.21.15.57:0/2992453745 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (unknown 3554970955 0 0) 0x7fe7e4048560 con 0x7fe7f4104750 2024-06-11T13:23:19.462 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:18.658+0000 7fe7f0ff9640 1 -- 172.21.15.57:0/2992453745 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7fe7e4017400 con 0x7fe7f4104750 2024-06-11T13:23:19.462 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:18.855+0000 7fe7f2ffd640 1 -- 172.21.15.57:0/2992453745 >> v1:172.21.15.57:6800/262061228 conn(0x7fe7dc03a300 legacy=0x7fe7dc03c7c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.57:6800/262061228 2024-06-11T13:23:19.462 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:18.893+0000 7fe7fa1b5640 1 -- 172.21.15.57:0/2992453745 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fe7e0005b80 con 0x7fe7f4104750 2024-06-11T13:23:19.462 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:18.894+0000 7fe7f0ff9640 1 -- 172.21.15.57:0/2992453745 <== mon.0 v1:172.21.15.57:6789/0 10 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (unknown 3768197548 0 15966916) 0x7fe7e4016d00 con 0x7fe7f4104750 2024-06-11T13:23:19.462 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:18.896+0000 7fe7fa1b5640 1 -- 172.21.15.57:0/2992453745 >> v1:172.21.15.57:6800/262061228 conn(0x7fe7dc03a300 legacy=0x7fe7dc03c7c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-11T13:23:19.462 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:18.896+0000 7fe7fa1b5640 1 -- 172.21.15.57:0/2992453745 >> v1:172.21.15.57:6789/0 conn(0x7fe7f4104750 legacy=0x7fe7f419b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:19.462 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:18.896+0000 7fe7fa1b5640 1 -- 172.21.15.57:0/2992453745 shutdown_connections 2024-06-11T13:23:19.462 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:18.896+0000 7fe7fa1b5640 1 -- 172.21.15.57:0/2992453745 >> 172.21.15.57:0/2992453745 conn(0x7fe7f40ffcb0 msgr2=0x7fe7f4102130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:19.462 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:18.896+0000 7fe7fa1b5640 1 -- 172.21.15.57:0/2992453745 shutdown_connections 2024-06-11T13:23:19.462 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:18.896+0000 7fe7fa1b5640 1 -- 172.21.15.57:0/2992453745 wait complete. 2024-06-11T13:23:19.462 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for the mgr to restart... 2024-06-11T13:23:19.463 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for mgr epoch 5... 2024-06-11T13:23:19.703 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:19 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:19.477+0000 7fdc26ac2040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-11T13:23:19.703 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:19 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:19.638+0000 7fdc26ac2040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-11T13:23:19.703 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:19 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:19.708+0000 7fdc26ac2040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-11T13:23:20.031 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:19 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:19.800+0000 7fdc26ac2040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-11T13:23:20.031 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:19 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:19.865+0000 7fdc26ac2040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-11T13:23:20.781 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:20 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:20.440+0000 7fdc26ac2040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-11T13:23:20.781 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:20 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:20.611+0000 7fdc26ac2040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-11T13:23:20.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:20 smithi057 ceph-mon[29805]: Active manager daemon y restarted 2024-06-11T13:23:20.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:20 smithi057 ceph-mon[29805]: Activating manager daemon y 2024-06-11T13:23:20.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:20 smithi057 ceph-mon[29805]: osdmap e2: 0 total, 0 up, 0 in 2024-06-11T13:23:20.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:20 smithi057 ceph-mon[29805]: mgrmap e6: y(active, starting, since 0.00765555s) 2024-06-11T13:23:20.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:20 smithi057 ceph-mon[29805]: from='mgr.14118 v1:172.21.15.57:0/2109654694' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-11T13:23:20.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:20 smithi057 ceph-mon[29805]: from='mgr.14118 v1:172.21.15.57:0/2109654694' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-11T13:23:20.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:20 smithi057 ceph-mon[29805]: from='mgr.14118 v1:172.21.15.57:0/2109654694' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-11T13:23:20.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:20 smithi057 ceph-mon[29805]: from='mgr.14118 v1:172.21.15.57:0/2109654694' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-11T13:23:20.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:20 smithi057 ceph-mon[29805]: from='mgr.14118 v1:172.21.15.57:0/2109654694' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-11T13:23:20.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:20 smithi057 ceph-mon[29805]: Manager daemon y is now available 2024-06-11T13:23:20.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:20 smithi057 ceph-mon[29805]: from='mgr.14118 v1:172.21.15.57:0/2109654694' entity='mgr.y' 2024-06-11T13:23:20.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:20 smithi057 ceph-mon[29805]: from='mgr.14118 v1:172.21.15.57:0/2109654694' entity='mgr.y' 2024-06-11T13:23:20.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:20 smithi057 ceph-mon[29805]: from='mgr.14118 v1:172.21.15.57:0/2109654694' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:23:20.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:20 smithi057 ceph-mon[29805]: from='mgr.14118 v1:172.21.15.57:0/2109654694' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:23:20.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:20 smithi057 ceph-mon[29805]: from='mgr.14118 v1:172.21.15.57:0/2109654694' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-11T13:23:20.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:20 smithi057 ceph-mon[29805]: from='mgr.14118 v1:172.21.15.57:0/2109654694' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-11T13:23:22.718 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:22 smithi057 ceph-mon[29805]: Found migration_current of "None". Setting to last migration. 2024-06-11T13:23:22.718 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:22 smithi057 ceph-mon[29805]: from='mgr.14118 v1:172.21.15.57:0/2109654694' entity='mgr.y' 2024-06-11T13:23:22.718 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:22 smithi057 ceph-mon[29805]: from='mgr.14118 v1:172.21.15.57:0/2109654694' entity='mgr.y' 2024-06-11T13:23:22.718 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:22 smithi057 ceph-mon[29805]: mgrmap e7: y(active, since 1.014s) 2024-06-11T13:23:22.849 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout { 2024-06-11T13:23:22.849 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-06-11T13:23:22.850 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "initialized": true 2024-06-11T13:23:22.850 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout } 2024-06-11T13:23:22.850 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:19.914+0000 7f75c916c640 1 Processor -- start 2024-06-11T13:23:22.850 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:19.914+0000 7f75c916c640 1 -- start start 2024-06-11T13:23:22.850 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:19.914+0000 7f75c916c640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f75c4078ed0 con 0x7f75c41062d0 2024-06-11T13:23:22.850 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:19.914+0000 7f75c2d9d640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f75c41062d0 0x7f75c41086c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:44048/0 (socket says 172.21.15.57:44048) 2024-06-11T13:23:22.850 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:19.914+0000 7f75c2d9d640 1 -- 172.21.15.57:0/4204484306 learned_addr learned my addr 172.21.15.57:0/4204484306 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:22.850 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:19.914+0000 7f75c1d9b640 1 -- 172.21.15.57:0/4204484306 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4124526668 0 0) 0x7f75c4078ed0 con 0x7f75c41062d0 2024-06-11T13:23:22.850 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:19.914+0000 7f75c1d9b640 1 -- 172.21.15.57:0/4204484306 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f75ac003520 con 0x7f75c41062d0 2024-06-11T13:23:22.850 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:19.915+0000 7f75c1d9b640 1 -- 172.21.15.57:0/4204484306 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3772057761 0 0) 0x7f75ac003520 con 0x7f75c41062d0 2024-06-11T13:23:22.851 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:19.915+0000 7f75c1d9b640 1 -- 172.21.15.57:0/4204484306 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75c410adf0 con 0x7f75c41062d0 2024-06-11T13:23:22.851 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:19.915+0000 7f75c1d9b640 1 -- 172.21.15.57:0/4204484306 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f75b4001580 con 0x7f75c41062d0 2024-06-11T13:23:22.851 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:19.915+0000 7f75c1d9b640 1 -- 172.21.15.57:0/4204484306 <== mon.0 v1:172.21.15.57:6789/0 4 ==== config(25 keys) ==== 1037+0+0 (unknown 2947037021 0 0) 0x7f75b4001d20 con 0x7f75c41062d0 2024-06-11T13:23:22.851 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:19.915+0000 7f75c1d9b640 1 -- 172.21.15.57:0/4204484306 <== mon.0 v1:172.21.15.57:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f75b40058d0 con 0x7f75c41062d0 2024-06-11T13:23:22.851 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:19.916+0000 7f75c916c640 1 -- 172.21.15.57:0/4204484306 >> v1:172.21.15.57:6789/0 conn(0x7f75c41062d0 legacy=0x7f75c41086c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:22.851 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:19.916+0000 7f75c916c640 1 -- 172.21.15.57:0/4204484306 shutdown_connections 2024-06-11T13:23:22.851 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:19.916+0000 7f75c916c640 1 -- 172.21.15.57:0/4204484306 >> 172.21.15.57:0/4204484306 conn(0x7f75c40ffcc0 msgr2=0x7f75c4102140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:22.851 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:19.916+0000 7f75c916c640 1 -- 172.21.15.57:0/4204484306 shutdown_connections 2024-06-11T13:23:22.851 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:19.916+0000 7f75c916c640 1 -- 172.21.15.57:0/4204484306 wait complete. 2024-06-11T13:23:22.851 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:19.917+0000 7f75c916c640 1 Processor -- start 2024-06-11T13:23:22.852 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:19.917+0000 7f75c916c640 1 -- start start 2024-06-11T13:23:22.852 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:19.917+0000 7f75c916c640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f75c4199b50 con 0x7f75c41062d0 2024-06-11T13:23:22.852 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:19.917+0000 7f75c2d9d640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f75c41062d0 0x7f75c4199440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:44060/0 (socket says 172.21.15.57:44060) 2024-06-11T13:23:22.852 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:19.917+0000 7f75c2d9d640 1 -- 172.21.15.57:0/1404987104 learned_addr learned my addr 172.21.15.57:0/1404987104 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:22.852 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:19.918+0000 7f75abfff640 1 -- 172.21.15.57:0/1404987104 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1976985339 0 0) 0x7f75c4199b50 con 0x7f75c41062d0 2024-06-11T13:23:22.852 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:19.918+0000 7f75abfff640 1 -- 172.21.15.57:0/1404987104 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f75ac005410 con 0x7f75c41062d0 2024-06-11T13:23:22.852 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:19.918+0000 7f75abfff640 1 -- 172.21.15.57:0/1404987104 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3556892323 0 0) 0x7f75ac005410 con 0x7f75c41062d0 2024-06-11T13:23:22.852 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:19.918+0000 7f75abfff640 1 -- 172.21.15.57:0/1404987104 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f75c4199b50 con 0x7f75c41062d0 2024-06-11T13:23:22.852 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:19.918+0000 7f75abfff640 1 -- 172.21.15.57:0/1404987104 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f75b4001080 con 0x7f75c41062d0 2024-06-11T13:23:22.852 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:19.918+0000 7f75abfff640 1 -- 172.21.15.57:0/1404987104 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1390203561 0 0) 0x7f75c4199b50 con 0x7f75c41062d0 2024-06-11T13:23:22.852 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:19.918+0000 7f75abfff640 1 -- 172.21.15.57:0/1404987104 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75c4199d20 con 0x7f75c41062d0 2024-06-11T13:23:22.852 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:19.918+0000 7f75c916c640 1 -- 172.21.15.57:0/1404987104 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f75c419a030 con 0x7f75c41062d0 2024-06-11T13:23:22.853 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:19.918+0000 7f75c916c640 1 -- 172.21.15.57:0/1404987104 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f75c419dbc0 con 0x7f75c41062d0 2024-06-11T13:23:22.853 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:19.918+0000 7f75abfff640 1 -- 172.21.15.57:0/1404987104 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1037+0+0 (unknown 2947037021 0 0) 0x7f75b4004a20 con 0x7f75c41062d0 2024-06-11T13:23:22.853 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:19.918+0000 7f75abfff640 1 -- 172.21.15.57:0/1404987104 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f75b4006170 con 0x7f75c41062d0 2024-06-11T13:23:22.853 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:19.919+0000 7f75abfff640 1 -- 172.21.15.57:0/1404987104 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 5) ==== 46252+0+0 (unknown 2352721741 0 0) 0x7f75b4006330 con 0x7f75c41062d0 2024-06-11T13:23:22.853 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:19.919+0000 7f75abfff640 1 -- 172.21.15.57:0/1404987104 --> v1:172.21.15.57:6800/262061228 -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f75b4001080 con 0x7f75ac03a4f0 2024-06-11T13:23:22.853 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:19.919+0000 7f75c259c640 1 -- 172.21.15.57:0/1404987104 >> v1:172.21.15.57:6800/262061228 conn(0x7f75ac03a4f0 legacy=0x7f75ac03c9b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.57:6800/262061228 2024-06-11T13:23:22.853 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:19.919+0000 7f75abfff640 1 -- 172.21.15.57:0/1404987104 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (unknown 3554970955 0 0) 0x7f75b40484e0 con 0x7f75c41062d0 2024-06-11T13:23:22.853 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:20.120+0000 7f75c259c640 1 -- 172.21.15.57:0/1404987104 >> v1:172.21.15.57:6800/262061228 conn(0x7f75ac03a4f0 legacy=0x7f75ac03c9b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.57:6800/262061228 2024-06-11T13:23:22.853 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:20.521+0000 7f75c259c640 1 -- 172.21.15.57:0/1404987104 >> v1:172.21.15.57:6800/262061228 conn(0x7f75ac03a4f0 legacy=0x7f75ac03c9b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.57:6800/262061228 2024-06-11T13:23:22.853 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:20.619+0000 7f75abfff640 1 -- 172.21.15.57:0/1404987104 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mgrmap(e 6) ==== 46054+0+0 (unknown 945423654 0 0) 0x7f75b4046ed0 con 0x7f75c41062d0 2024-06-11T13:23:22.853 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:20.619+0000 7f75abfff640 1 -- 172.21.15.57:0/1404987104 >> v1:172.21.15.57:6800/262061228 conn(0x7f75ac03a4f0 legacy=0x7f75ac03c9b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-11T13:23:22.854 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:21.624+0000 7f75abfff640 1 -- 172.21.15.57:0/1404987104 <== mon.0 v1:172.21.15.57:6789/0 10 ==== mgrmap(e 7) ==== 46146+0+0 (unknown 2371625627 0 0) 0x7f75b4048eb0 con 0x7f75c41062d0 2024-06-11T13:23:22.854 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:21.624+0000 7f75abfff640 1 -- 172.21.15.57:0/1404987104 --> v1:172.21.15.57:6800/1064738157 -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f75b4001080 con 0x7f75ac03ed80 2024-06-11T13:23:22.854 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:21.625+0000 7f75abfff640 1 -- 172.21.15.57:0/1404987104 <== mgr.14118 v1:172.21.15.57:6800/1064738157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (unknown 0 0 4109177037) 0x7f75b4001080 con 0x7f75ac03ed80 2024-06-11T13:23:22.854 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:21.631+0000 7f75c916c640 1 -- 172.21.15.57:0/1404987104 --> v1:172.21.15.57:6800/1064738157 -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f75c4063000 con 0x7f75ac03ed80 2024-06-11T13:23:22.854 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:21.631+0000 7f75abfff640 1 -- 172.21.15.57:0/1404987104 <== mgr.14118 v1:172.21.15.57:6800/1064738157 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (unknown 0 0 96372106) 0x7f75c4063000 con 0x7f75ac03ed80 2024-06-11T13:23:22.854 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:21.632+0000 7f75c916c640 1 -- 172.21.15.57:0/1404987104 >> v1:172.21.15.57:6800/1064738157 conn(0x7f75ac03ed80 legacy=0x7f75ac041170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:22.854 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:21.632+0000 7f75c916c640 1 -- 172.21.15.57:0/1404987104 >> v1:172.21.15.57:6789/0 conn(0x7f75c41062d0 legacy=0x7f75c4199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:22.854 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:21.632+0000 7f75c916c640 1 -- 172.21.15.57:0/1404987104 shutdown_connections 2024-06-11T13:23:22.854 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:21.632+0000 7f75c916c640 1 -- 172.21.15.57:0/1404987104 >> 172.21.15.57:0/1404987104 conn(0x7f75c40ffcc0 msgr2=0x7f75c4190170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:22.854 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:21.632+0000 7f75c916c640 1 -- 172.21.15.57:0/1404987104 shutdown_connections 2024-06-11T13:23:22.854 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:21.632+0000 7f75c916c640 1 -- 172.21.15.57:0/1404987104 wait complete. 2024-06-11T13:23:22.855 INFO:teuthology.orchestra.run.smithi057.stdout:mgr epoch 5 is available 2024-06-11T13:23:22.855 INFO:teuthology.orchestra.run.smithi057.stdout:Setting orchestrator backend to cephadm... 2024-06-11T13:23:22.972 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:22 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:22.724+0000 7fdb94127640 -1 log_channel(cephadm) log [ERR] : [11/Jun/2024:13:23:22] ENGINE Error in HTTPServer.serve 2024-06-11T13:23:22.972 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:22 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: Traceback (most recent call last): 2024-06-11T13:23:22.973 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:22 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-06-11T13:23:22.973 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:22 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: self._connections.run(self.expiration_interval) 2024-06-11T13:23:22.973 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:22 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-06-11T13:23:22.973 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:22 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: self._run(expiration_interval) 2024-06-11T13:23:22.973 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:22 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-06-11T13:23:22.973 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:22 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: new_conn = self._from_server_socket(self.server.socket) 2024-06-11T13:23:22.973 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:22 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-06-11T13:23:22.973 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:22 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-06-11T13:23:22.973 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:22 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-06-11T13:23:22.973 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:22 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: s = self.context.wrap_socket( 2024-06-11T13:23:22.973 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:22 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-06-11T13:23:22.974 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:22 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: return self.sslsocket_class._create( 2024-06-11T13:23:22.974 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:22 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-06-11T13:23:22.974 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:22 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: self.do_handshake() 2024-06-11T13:23:22.974 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:22 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-06-11T13:23:22.974 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:22 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: self._sslobj.do_handshake() 2024-06-11T13:23:22.974 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:22 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1129) 2024-06-11T13:23:22.974 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:22 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:23.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:23 smithi057 ceph-mon[29805]: from='client.14122 v1:172.21.15.57:0/1404987104' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-06-11T13:23:23.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:23 smithi057 ceph-mon[29805]: from='client.14122 v1:172.21.15.57:0/1404987104' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-06-11T13:23:23.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:23 smithi057 ceph-mon[29805]: from='mgr.14118 v1:172.21.15.57:0/2109654694' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:23:24.212 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:23.430+0000 7f98d17ed640 1 Processor -- start 2024-06-11T13:23:24.212 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:23.430+0000 7f98d17ed640 1 -- start start 2024-06-11T13:23:24.212 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:23.430+0000 7f98d17ed640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f98cc07e5a0 con 0x7f98cc07c110 2024-06-11T13:23:24.213 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:23.430+0000 7f98caffd640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f98cc07c110 0x7f98cc07c530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:44136/0 (socket says 172.21.15.57:44136) 2024-06-11T13:23:24.213 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:23.430+0000 7f98caffd640 1 -- 172.21.15.57:0/674744877 learned_addr learned my addr 172.21.15.57:0/674744877 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:24.213 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:23.431+0000 7f98ca7fc640 1 -- 172.21.15.57:0/674744877 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2917287863 0 0) 0x7f98cc07e5a0 con 0x7f98cc07c110 2024-06-11T13:23:24.213 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:23.431+0000 7f98ca7fc640 1 -- 172.21.15.57:0/674744877 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f98b8003520 con 0x7f98cc07c110 2024-06-11T13:23:24.213 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:23.431+0000 7f98ca7fc640 1 -- 172.21.15.57:0/674744877 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 816395362 0 0) 0x7f98b8003520 con 0x7f98cc07c110 2024-06-11T13:23:24.213 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:23.431+0000 7f98ca7fc640 1 -- 172.21.15.57:0/674744877 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98cc07f780 con 0x7f98cc07c110 2024-06-11T13:23:24.213 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:23.431+0000 7f98ca7fc640 1 -- 172.21.15.57:0/674744877 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f98bc001580 con 0x7f98cc07c110 2024-06-11T13:23:24.213 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:23.431+0000 7f98ca7fc640 1 -- 172.21.15.57:0/674744877 <== mon.0 v1:172.21.15.57:6789/0 4 ==== config(25 keys) ==== 1037+0+0 (unknown 2947037021 0 0) 0x7f98bc001d20 con 0x7f98cc07c110 2024-06-11T13:23:24.213 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:23.431+0000 7f98ca7fc640 1 -- 172.21.15.57:0/674744877 <== mon.0 v1:172.21.15.57:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f98bc0058d0 con 0x7f98cc07c110 2024-06-11T13:23:24.213 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:23.431+0000 7f98d17ed640 1 -- 172.21.15.57:0/674744877 >> v1:172.21.15.57:6789/0 conn(0x7f98cc07c110 legacy=0x7f98cc07c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:24.213 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:23.432+0000 7f98d17ed640 1 -- 172.21.15.57:0/674744877 shutdown_connections 2024-06-11T13:23:24.213 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:23.432+0000 7f98d17ed640 1 -- 172.21.15.57:0/674744877 >> 172.21.15.57:0/674744877 conn(0x7f98cc07af20 msgr2=0x7f98cc07b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:24.214 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:23.432+0000 7f98d17ed640 1 -- 172.21.15.57:0/674744877 shutdown_connections 2024-06-11T13:23:24.214 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:23.432+0000 7f98d17ed640 1 -- 172.21.15.57:0/674744877 wait complete. 2024-06-11T13:23:24.214 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:23.432+0000 7f98d17ed640 1 Processor -- start 2024-06-11T13:23:24.214 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:23.432+0000 7f98d17ed640 1 -- start start 2024-06-11T13:23:24.214 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:23.433+0000 7f98d17ed640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f98cc1a0120 con 0x7f98cc07c110 2024-06-11T13:23:24.214 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:23.433+0000 7f98caffd640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f98cc07c110 0x7f98cc19fa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:44142/0 (socket says 172.21.15.57:44142) 2024-06-11T13:23:24.214 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:23.433+0000 7f98caffd640 1 -- 172.21.15.57:0/3974891804 learned_addr learned my addr 172.21.15.57:0/3974891804 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:24.214 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:23.433+0000 7f98c8ff9640 1 -- 172.21.15.57:0/3974891804 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1895398057 0 0) 0x7f98cc1a0120 con 0x7f98cc07c110 2024-06-11T13:23:24.214 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:23.433+0000 7f98c8ff9640 1 -- 172.21.15.57:0/3974891804 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f98b8005410 con 0x7f98cc07c110 2024-06-11T13:23:24.214 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:23.433+0000 7f98c8ff9640 1 -- 172.21.15.57:0/3974891804 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2512703867 0 0) 0x7f98b8005410 con 0x7f98cc07c110 2024-06-11T13:23:24.214 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:23.433+0000 7f98c8ff9640 1 -- 172.21.15.57:0/3974891804 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f98cc1a0120 con 0x7f98cc07c110 2024-06-11T13:23:24.214 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:23.433+0000 7f98c8ff9640 1 -- 172.21.15.57:0/3974891804 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f98bc001080 con 0x7f98cc07c110 2024-06-11T13:23:24.214 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:23.433+0000 7f98c8ff9640 1 -- 172.21.15.57:0/3974891804 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4032956297 0 0) 0x7f98cc1a0120 con 0x7f98cc07c110 2024-06-11T13:23:24.215 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:23.433+0000 7f98c8ff9640 1 -- 172.21.15.57:0/3974891804 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98cc1a02f0 con 0x7f98cc07c110 2024-06-11T13:23:24.215 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:23.433+0000 7f98d17ed640 1 -- 172.21.15.57:0/3974891804 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f98cc1a0600 con 0x7f98cc07c110 2024-06-11T13:23:24.215 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:23.433+0000 7f98d17ed640 1 -- 172.21.15.57:0/3974891804 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f98cc1a4190 con 0x7f98cc07c110 2024-06-11T13:23:24.215 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:23.434+0000 7f98c8ff9640 1 -- 172.21.15.57:0/3974891804 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1037+0+0 (unknown 2947037021 0 0) 0x7f98bc001c80 con 0x7f98cc07c110 2024-06-11T13:23:24.215 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:23.434+0000 7f98c8ff9640 1 -- 172.21.15.57:0/3974891804 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f98bc005f10 con 0x7f98cc07c110 2024-06-11T13:23:24.215 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:23.434+0000 7f98c8ff9640 1 -- 172.21.15.57:0/3974891804 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 8) ==== 46252+0+0 (unknown 1643827861 0 0) 0x7f98bc011570 con 0x7f98cc07c110 2024-06-11T13:23:24.215 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:23.434+0000 7f98d17ed640 1 -- 172.21.15.57:0/3974891804 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98ac0050b0 con 0x7f98cc07c110 2024-06-11T13:23:24.215 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:23.435+0000 7f98c8ff9640 1 -- 172.21.15.57:0/3974891804 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (unknown 4180383460 0 0) 0x7f98bc047780 con 0x7f98cc07c110 2024-06-11T13:23:24.215 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:23.440+0000 7f98c8ff9640 1 -- 172.21.15.57:0/3974891804 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f98bc0174a0 con 0x7f98cc07c110 2024-06-11T13:23:24.215 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:23.641+0000 7f98d17ed640 1 -- 172.21.15.57:0/3974891804 --> v1:172.21.15.57:6800/1064738157 -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f98ac000bc0 con 0x7f98b803a4a0 2024-06-11T13:23:24.215 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:23.645+0000 7f98c8ff9640 1 -- 172.21.15.57:0/3974891804 <== mgr.14118 v1:172.21.15.57:6800/1064738157 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (unknown 0 0 0) 0x7f98ac000bc0 con 0x7f98b803a4a0 2024-06-11T13:23:24.215 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:23.647+0000 7f98d17ed640 1 -- 172.21.15.57:0/3974891804 >> v1:172.21.15.57:6800/1064738157 conn(0x7f98b803a4a0 legacy=0x7f98b803c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:24.216 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:23.647+0000 7f98d17ed640 1 -- 172.21.15.57:0/3974891804 >> v1:172.21.15.57:6789/0 conn(0x7f98cc07c110 legacy=0x7f98cc19fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:24.216 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:23.647+0000 7f98d17ed640 1 -- 172.21.15.57:0/3974891804 shutdown_connections 2024-06-11T13:23:24.216 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:23.647+0000 7f98d17ed640 1 -- 172.21.15.57:0/3974891804 >> 172.21.15.57:0/3974891804 conn(0x7f98cc07af20 msgr2=0x7f98cc1058f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:24.216 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:23.648+0000 7f98d17ed640 1 -- 172.21.15.57:0/3974891804 shutdown_connections 2024-06-11T13:23:24.216 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:23.648+0000 7f98d17ed640 1 -- 172.21.15.57:0/3974891804 wait complete. 2024-06-11T13:23:24.629 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:24 smithi057 ceph-mon[29805]: [11/Jun/2024:13:23:22] ENGINE Bus STARTING 2024-06-11T13:23:24.630 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:24 smithi057 ceph-mon[29805]: [11/Jun/2024:13:23:22] ENGINE Serving on https://172.21.15.57:7150 2024-06-11T13:23:24.630 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:24 smithi057 ceph-mon[29805]: [11/Jun/2024:13:23:22] ENGINE Error in HTTPServer.serve 2024-06-11T13:23:24.630 INFO:journalctl@ceph.mon.a.smithi057.stdout: Traceback (most recent call last): 2024-06-11T13:23:24.630 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-06-11T13:23:24.630 INFO:journalctl@ceph.mon.a.smithi057.stdout: self._connections.run(self.expiration_interval) 2024-06-11T13:23:24.630 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-06-11T13:23:24.630 INFO:journalctl@ceph.mon.a.smithi057.stdout: self._run(expiration_interval) 2024-06-11T13:23:24.630 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-06-11T13:23:24.630 INFO:journalctl@ceph.mon.a.smithi057.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-06-11T13:23:24.630 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-06-11T13:23:24.631 INFO:journalctl@ceph.mon.a.smithi057.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-06-11T13:23:24.631 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-06-11T13:23:24.631 INFO:journalctl@ceph.mon.a.smithi057.stdout: s = self.context.wrap_socket( 2024-06-11T13:23:24.631 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-06-11T13:23:24.631 INFO:journalctl@ceph.mon.a.smithi057.stdout: return self.sslsocket_class._create( 2024-06-11T13:23:24.631 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-06-11T13:23:24.631 INFO:journalctl@ceph.mon.a.smithi057.stdout: self.do_handshake() 2024-06-11T13:23:24.631 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-06-11T13:23:24.631 INFO:journalctl@ceph.mon.a.smithi057.stdout: self._sslobj.do_handshake() 2024-06-11T13:23:24.631 INFO:journalctl@ceph.mon.a.smithi057.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1129) 2024-06-11T13:23:24.631 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:24 smithi057 ceph-mon[29805]: [11/Jun/2024:13:23:22] ENGINE Serving on http://172.21.15.57:8765 2024-06-11T13:23:24.631 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:24 smithi057 ceph-mon[29805]: [11/Jun/2024:13:23:22] ENGINE Bus STARTED 2024-06-11T13:23:24.631 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:24 smithi057 ceph-mon[29805]: mgrmap e8: y(active, since 2s) 2024-06-11T13:23:24.632 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:24 smithi057 ceph-mon[29805]: from='mgr.14118 v1:172.21.15.57:0/2109654694' entity='mgr.y' 2024-06-11T13:23:24.632 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:24 smithi057 ceph-mon[29805]: from='mgr.14118 v1:172.21.15.57:0/2109654694' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:23:25.403 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout value unchanged 2024-06-11T13:23:25.404 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:24.635+0000 7f440368e640 1 Processor -- start 2024-06-11T13:23:25.404 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:24.635+0000 7f440368e640 1 -- start start 2024-06-11T13:23:25.404 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:24.635+0000 7f440368e640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f43fc109d00 con 0x7f43fc103b40 2024-06-11T13:23:25.404 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:24.636+0000 7f440142a640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f43fc103b40 0x7f43fc103f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:44158/0 (socket says 172.21.15.57:44158) 2024-06-11T13:23:25.404 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:24.636+0000 7f440142a640 1 -- 172.21.15.57:0/2505729610 learned_addr learned my addr 172.21.15.57:0/2505729610 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:25.404 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:24.636+0000 7f43f3fff640 1 -- 172.21.15.57:0/2505729610 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1079552951 0 0) 0x7f43fc109d00 con 0x7f43fc103b40 2024-06-11T13:23:25.404 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:24.636+0000 7f43f3fff640 1 -- 172.21.15.57:0/2505729610 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43e8003520 con 0x7f43fc103b40 2024-06-11T13:23:25.404 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:24.636+0000 7f43f3fff640 1 -- 172.21.15.57:0/2505729610 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2619400798 0 0) 0x7f43e8003520 con 0x7f43fc103b40 2024-06-11T13:23:25.404 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:24.636+0000 7f43f3fff640 1 -- 172.21.15.57:0/2505729610 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43fc10aee0 con 0x7f43fc103b40 2024-06-11T13:23:25.404 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:24.636+0000 7f43f3fff640 1 -- 172.21.15.57:0/2505729610 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f43f8001580 con 0x7f43fc103b40 2024-06-11T13:23:25.405 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:24.636+0000 7f43f3fff640 1 -- 172.21.15.57:0/2505729610 <== mon.0 v1:172.21.15.57:6789/0 4 ==== config(25 keys) ==== 1037+0+0 (unknown 2947037021 0 0) 0x7f43f8001c50 con 0x7f43fc103b40 2024-06-11T13:23:25.405 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:24.636+0000 7f43f3fff640 1 -- 172.21.15.57:0/2505729610 <== mon.0 v1:172.21.15.57:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f43f80058d0 con 0x7f43fc103b40 2024-06-11T13:23:25.405 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:24.637+0000 7f440368e640 1 -- 172.21.15.57:0/2505729610 >> v1:172.21.15.57:6789/0 conn(0x7f43fc103b40 legacy=0x7f43fc103f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:25.405 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:24.637+0000 7f440368e640 1 -- 172.21.15.57:0/2505729610 shutdown_connections 2024-06-11T13:23:25.405 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:24.637+0000 7f440368e640 1 -- 172.21.15.57:0/2505729610 >> 172.21.15.57:0/2505729610 conn(0x7f43fc0ff430 msgr2=0x7f43fc1018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:25.405 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:24.637+0000 7f440368e640 1 -- 172.21.15.57:0/2505729610 shutdown_connections 2024-06-11T13:23:25.406 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:24.637+0000 7f440368e640 1 -- 172.21.15.57:0/2505729610 wait complete. 2024-06-11T13:23:25.406 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:24.638+0000 7f440368e640 1 Processor -- start 2024-06-11T13:23:25.406 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:24.638+0000 7f440368e640 1 -- start start 2024-06-11T13:23:25.407 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:24.638+0000 7f440368e640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f43fc199be0 con 0x7f43fc103b40 2024-06-11T13:23:25.407 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:24.638+0000 7f440142a640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f43fc103b40 0x7f43fc1994d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:44168/0 (socket says 172.21.15.57:44168) 2024-06-11T13:23:25.407 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:24.638+0000 7f440142a640 1 -- 172.21.15.57:0/4264959242 learned_addr learned my addr 172.21.15.57:0/4264959242 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:25.407 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:24.638+0000 7f43f27fc640 1 -- 172.21.15.57:0/4264959242 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1650446108 0 0) 0x7f43fc199be0 con 0x7f43fc103b40 2024-06-11T13:23:25.407 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:24.638+0000 7f43f27fc640 1 -- 172.21.15.57:0/4264959242 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43e8005460 con 0x7f43fc103b40 2024-06-11T13:23:25.407 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:24.638+0000 7f43f27fc640 1 -- 172.21.15.57:0/4264959242 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1981327527 0 0) 0x7f43e8005460 con 0x7f43fc103b40 2024-06-11T13:23:25.407 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:24.638+0000 7f43f27fc640 1 -- 172.21.15.57:0/4264959242 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f43fc199be0 con 0x7f43fc103b40 2024-06-11T13:23:25.407 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:24.638+0000 7f43f27fc640 1 -- 172.21.15.57:0/4264959242 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f43f8001080 con 0x7f43fc103b40 2024-06-11T13:23:25.407 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:24.639+0000 7f43f27fc640 1 -- 172.21.15.57:0/4264959242 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 415591727 0 0) 0x7f43fc199be0 con 0x7f43fc103b40 2024-06-11T13:23:25.407 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:24.639+0000 7f43f27fc640 1 -- 172.21.15.57:0/4264959242 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43fc199db0 con 0x7f43fc103b40 2024-06-11T13:23:25.407 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:24.639+0000 7f440368e640 1 -- 172.21.15.57:0/4264959242 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f43fc19a0c0 con 0x7f43fc103b40 2024-06-11T13:23:25.407 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:24.639+0000 7f440368e640 1 -- 172.21.15.57:0/4264959242 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f43fc19dc50 con 0x7f43fc103b40 2024-06-11T13:23:25.408 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:24.639+0000 7f43f27fc640 1 -- 172.21.15.57:0/4264959242 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1037+0+0 (unknown 2947037021 0 0) 0x7f43f8001b70 con 0x7f43fc103b40 2024-06-11T13:23:25.408 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:24.639+0000 7f43f27fc640 1 -- 172.21.15.57:0/4264959242 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f43f8005f20 con 0x7f43fc103b40 2024-06-11T13:23:25.408 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:24.640+0000 7f43f27fc640 1 -- 172.21.15.57:0/4264959242 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 8) ==== 46252+0+0 (unknown 1643827861 0 0) 0x7f43f8011580 con 0x7f43fc103b40 2024-06-11T13:23:25.408 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:24.640+0000 7f440368e640 1 -- 172.21.15.57:0/4264959242 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43e00050b0 con 0x7f43fc103b40 2024-06-11T13:23:25.408 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:24.640+0000 7f43f27fc640 1 -- 172.21.15.57:0/4264959242 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (unknown 4180383460 0 0) 0x7f43f8047790 con 0x7f43fc103b40 2024-06-11T13:23:25.408 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:24.644+0000 7f43f27fc640 1 -- 172.21.15.57:0/4264959242 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f43f8004ce0 con 0x7f43fc103b40 2024-06-11T13:23:25.408 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:24.832+0000 7f440368e640 1 -- 172.21.15.57:0/4264959242 --> v1:172.21.15.57:6800/1064738157 -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f43e0000bc0 con 0x7f43e803a440 2024-06-11T13:23:25.408 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:24.832+0000 7f43f27fc640 1 -- 172.21.15.57:0/4264959242 <== mgr.14118 v1:172.21.15.57:6800/1064738157 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (unknown 0 0 2070689548) 0x7f43e0000bc0 con 0x7f43e803a440 2024-06-11T13:23:25.408 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:24.835+0000 7f440368e640 1 -- 172.21.15.57:0/4264959242 >> v1:172.21.15.57:6800/1064738157 conn(0x7f43e803a440 legacy=0x7f43e803c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:25.408 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:24.835+0000 7f440368e640 1 -- 172.21.15.57:0/4264959242 >> v1:172.21.15.57:6789/0 conn(0x7f43fc103b40 legacy=0x7f43fc1994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:25.408 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:24.835+0000 7f440368e640 1 -- 172.21.15.57:0/4264959242 shutdown_connections 2024-06-11T13:23:25.408 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:24.835+0000 7f440368e640 1 -- 172.21.15.57:0/4264959242 >> 172.21.15.57:0/4264959242 conn(0x7f43fc0ff430 msgr2=0x7f43fc1018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:25.408 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:24.835+0000 7f440368e640 1 -- 172.21.15.57:0/4264959242 shutdown_connections 2024-06-11T13:23:25.409 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:24.835+0000 7f440368e640 1 -- 172.21.15.57:0/4264959242 wait complete. 2024-06-11T13:23:25.409 INFO:teuthology.orchestra.run.smithi057.stdout:Generating ssh key... 2024-06-11T13:23:25.647 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:25 smithi057 ceph-mon[29805]: from='client.14130 v1:172.21.15.57:0/3974891804' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:23:26.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:26 smithi057 ceph-mon[29805]: from='client.14132 v1:172.21.15.57:0/4264959242' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:23:26.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:26 smithi057 ceph-mon[29805]: from='mgr.14118 v1:172.21.15.57:0/2109654694' entity='mgr.y' 2024-06-11T13:23:26.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:26 smithi057 ceph-mon[29805]: from='mgr.14118 v1:172.21.15.57:0/2109654694' entity='mgr.y' 2024-06-11T13:23:26.620 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:26 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: Generating public/private rsa key pair. 2024-06-11T13:23:26.620 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:26 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: Your identification has been saved in /tmp/tmp4hrdzqx3/key 2024-06-11T13:23:26.620 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:26 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: Your public key has been saved in /tmp/tmp4hrdzqx3/key.pub 2024-06-11T13:23:26.620 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:26 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: The key fingerprint is: 2024-06-11T13:23:26.620 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:26 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: SHA256:b+0QJN8hh/YXeiYEvTxMNvS+3KCrQKdHmSYioV2GlFE ceph-98a6635e-27f5-11ef-bc9b-c7b262605968 2024-06-11T13:23:26.620 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:26 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: The key's randomart image is: 2024-06-11T13:23:26.620 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:26 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: +---[RSA 3072]----+ 2024-06-11T13:23:26.620 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:26 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: | o+E .o. | 2024-06-11T13:23:26.621 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:26 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: | ... o=. | 2024-06-11T13:23:26.621 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:26 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: | o o . ==+oo | 2024-06-11T13:23:26.621 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:26 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: | o + = O=+ . | 2024-06-11T13:23:26.621 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:26 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: | . o . S O =.* | 2024-06-11T13:23:26.621 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:26 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: | . o B o B + | 2024-06-11T13:23:26.621 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:26 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: | o = o o .| 2024-06-11T13:23:26.621 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:26 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: | + o . | 2024-06-11T13:23:26.621 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:26 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: | ..o | 2024-06-11T13:23:26.621 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:26 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: +----[SHA256]-----+ 2024-06-11T13:23:26.911 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:25.855+0000 7f0767f87640 1 Processor -- start 2024-06-11T13:23:26.911 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:25.855+0000 7f0767f87640 1 -- start start 2024-06-11T13:23:26.911 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:25.855+0000 7f0767f87640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f076007a310 con 0x7f076007b710 2024-06-11T13:23:26.911 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:25.855+0000 7f0765d23640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f076007b710 0x7f0760079b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:44172/0 (socket says 172.21.15.57:44172) 2024-06-11T13:23:26.911 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:25.855+0000 7f0765d23640 1 -- 172.21.15.57:0/3744198355 learned_addr learned my addr 172.21.15.57:0/3744198355 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:26.912 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:25.855+0000 7f0764d21640 1 -- 172.21.15.57:0/3744198355 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2942937662 0 0) 0x7f076007a310 con 0x7f076007b710 2024-06-11T13:23:26.912 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:25.855+0000 7f0764d21640 1 -- 172.21.15.57:0/3744198355 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f074c003520 con 0x7f076007b710 2024-06-11T13:23:26.912 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:25.856+0000 7f0764d21640 1 -- 172.21.15.57:0/3744198355 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1688305858 0 0) 0x7f074c003520 con 0x7f076007b710 2024-06-11T13:23:26.912 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:25.856+0000 7f0764d21640 1 -- 172.21.15.57:0/3744198355 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f076007a4e0 con 0x7f076007b710 2024-06-11T13:23:26.912 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:25.856+0000 7f0764d21640 1 -- 172.21.15.57:0/3744198355 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f075c001580 con 0x7f076007b710 2024-06-11T13:23:26.912 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:25.856+0000 7f0764d21640 1 -- 172.21.15.57:0/3744198355 <== mon.0 v1:172.21.15.57:6789/0 4 ==== config(25 keys) ==== 1037+0+0 (unknown 2947037021 0 0) 0x7f075c001d20 con 0x7f076007b710 2024-06-11T13:23:26.913 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:25.856+0000 7f0764d21640 1 -- 172.21.15.57:0/3744198355 <== mon.0 v1:172.21.15.57:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f075c0058d0 con 0x7f076007b710 2024-06-11T13:23:26.913 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:25.856+0000 7f0767f87640 1 -- 172.21.15.57:0/3744198355 >> v1:172.21.15.57:6789/0 conn(0x7f076007b710 legacy=0x7f0760079b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:26.913 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:25.856+0000 7f0767f87640 1 -- 172.21.15.57:0/3744198355 shutdown_connections 2024-06-11T13:23:26.913 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:25.857+0000 7f0767f87640 1 -- 172.21.15.57:0/3744198355 >> 172.21.15.57:0/3744198355 conn(0x7f0760100d80 msgr2=0x7f0760103200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:26.913 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:25.857+0000 7f0767f87640 1 -- 172.21.15.57:0/3744198355 shutdown_connections 2024-06-11T13:23:26.913 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:25.857+0000 7f0767f87640 1 -- 172.21.15.57:0/3744198355 wait complete. 2024-06-11T13:23:26.914 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:25.857+0000 7f0767f87640 1 Processor -- start 2024-06-11T13:23:26.914 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:25.857+0000 7f0767f87640 1 -- start start 2024-06-11T13:23:26.914 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:25.857+0000 7f0767f87640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f076019bd80 con 0x7f076007b710 2024-06-11T13:23:26.914 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:25.857+0000 7f0765d23640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f076007b710 0x7f076019b670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:44186/0 (socket says 172.21.15.57:44186) 2024-06-11T13:23:26.914 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:25.858+0000 7f0765d23640 1 -- 172.21.15.57:0/124142958 learned_addr learned my addr 172.21.15.57:0/124142958 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:26.914 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:25.858+0000 7f0756ffd640 1 -- 172.21.15.57:0/124142958 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3770095392 0 0) 0x7f076019bd80 con 0x7f076007b710 2024-06-11T13:23:26.915 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:25.858+0000 7f0756ffd640 1 -- 172.21.15.57:0/124142958 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f074c005410 con 0x7f076007b710 2024-06-11T13:23:26.915 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:25.858+0000 7f0756ffd640 1 -- 172.21.15.57:0/124142958 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2307071335 0 0) 0x7f074c005410 con 0x7f076007b710 2024-06-11T13:23:26.915 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:25.858+0000 7f0756ffd640 1 -- 172.21.15.57:0/124142958 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f076019bd80 con 0x7f076007b710 2024-06-11T13:23:26.915 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:25.858+0000 7f0756ffd640 1 -- 172.21.15.57:0/124142958 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f075c001080 con 0x7f076007b710 2024-06-11T13:23:26.915 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:25.858+0000 7f0756ffd640 1 -- 172.21.15.57:0/124142958 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 530161144 0 0) 0x7f076019bd80 con 0x7f076007b710 2024-06-11T13:23:26.915 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:25.858+0000 7f0756ffd640 1 -- 172.21.15.57:0/124142958 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f076019bf50 con 0x7f076007b710 2024-06-11T13:23:26.916 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:25.858+0000 7f0767f87640 1 -- 172.21.15.57:0/124142958 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f076019c260 con 0x7f076007b710 2024-06-11T13:23:26.916 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:25.858+0000 7f0767f87640 1 -- 172.21.15.57:0/124142958 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f076019fdf0 con 0x7f076007b710 2024-06-11T13:23:26.916 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:25.858+0000 7f0756ffd640 1 -- 172.21.15.57:0/124142958 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1037+0+0 (unknown 2947037021 0 0) 0x7f075c001bc0 con 0x7f076007b710 2024-06-11T13:23:26.916 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:25.858+0000 7f0756ffd640 1 -- 172.21.15.57:0/124142958 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f075c005e90 con 0x7f076007b710 2024-06-11T13:23:26.916 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:25.859+0000 7f0756ffd640 1 -- 172.21.15.57:0/124142958 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 8) ==== 46252+0+0 (unknown 1643827861 0 0) 0x7f075c0115b0 con 0x7f076007b710 2024-06-11T13:23:26.916 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:25.859+0000 7f0767f87640 1 -- 172.21.15.57:0/124142958 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07440050b0 con 0x7f076007b710 2024-06-11T13:23:26.917 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:25.859+0000 7f0756ffd640 1 -- 172.21.15.57:0/124142958 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (unknown 4180383460 0 0) 0x7f075c047840 con 0x7f076007b710 2024-06-11T13:23:26.917 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:25.864+0000 7f0756ffd640 1 -- 172.21.15.57:0/124142958 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f075c0174d0 con 0x7f076007b710 2024-06-11T13:23:26.917 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:26.053+0000 7f0767f87640 1 -- 172.21.15.57:0/124142958 --> v1:172.21.15.57:6800/1064738157 -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f0744000bc0 con 0x7f074c03a450 2024-06-11T13:23:26.917 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:26.339+0000 7f0756ffd640 1 -- 172.21.15.57:0/124142958 <== mgr.14118 v1:172.21.15.57:6800/1064738157 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (unknown 0 0 0) 0x7f0744000bc0 con 0x7f074c03a450 2024-06-11T13:23:26.917 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:26.341+0000 7f0767f87640 1 -- 172.21.15.57:0/124142958 >> v1:172.21.15.57:6800/1064738157 conn(0x7f074c03a450 legacy=0x7f074c03c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:26.917 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:26.341+0000 7f0767f87640 1 -- 172.21.15.57:0/124142958 >> v1:172.21.15.57:6789/0 conn(0x7f076007b710 legacy=0x7f076019b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:26.918 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:26.341+0000 7f0767f87640 1 -- 172.21.15.57:0/124142958 shutdown_connections 2024-06-11T13:23:26.918 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:26.341+0000 7f0767f87640 1 -- 172.21.15.57:0/124142958 >> 172.21.15.57:0/124142958 conn(0x7f0760100d80 msgr2=0x7f0760109d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:26.918 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:26.341+0000 7f0767f87640 1 -- 172.21.15.57:0/124142958 shutdown_connections 2024-06-11T13:23:26.918 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:26.341+0000 7f0767f87640 1 -- 172.21.15.57:0/124142958 wait complete. 2024-06-11T13:23:27.506 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:27 smithi057 ceph-mon[29805]: from='client.14134 v1:172.21.15.57:0/124142958' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:23:27.506 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:27 smithi057 ceph-mon[29805]: Generating ssh key... 2024-06-11T13:23:28.066 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCfXBP/feJoo2nBQu0Lrvey6bhe7zfLyuifJivQLPpxIfQW8iKSdDsj7T13YZCD1Lh8rX4ya0gRvvibdkjj3mAKd13VN4VIgEZmLZEhFjosfLSmKjviz8tbBMJr6QtxrYJOycrkSrdTiToLBi53HrrYumt0BkXg/Pr4brn+cyIKr3k3lRkj8S5602Yrh7WN3lThdcpZk0OUJF+VMWvMJeTUPypvkiaTtHV4+AQ7TVEtJpyAiZC3ofIU+rkmhZWkk+NE+oDoroZgPq9VDusu7UmEIylpbCE0NATJP8dgqLs+At5eFFfTwoRuTx3JYPTYtVLBGHpgyiifNKkbSccFHzr15OUQX4cbpsZ+yYbV3ZR5tkbKhHcSusGiU1bkKTRRZUteu8gk+elCeHfFY9sTDGTuBKCGhdHBn+dJLY4AW6uUB8HZTJVSktmuc/raFh0jVgcpxbPbpWTtlwNpIPMp19qJ9XNqJl0sESkKGZeejbGK/OZxyA6HHDV9iLiShLYu728= ceph-98a6635e-27f5-11ef-bc9b-c7b262605968 2024-06-11T13:23:28.066 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:27.314+0000 7f284f2e7640 1 Processor -- start 2024-06-11T13:23:28.067 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:27.314+0000 7f284f2e7640 1 -- start start 2024-06-11T13:23:28.067 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:27.314+0000 7f284f2e7640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f2848109c90 con 0x7f2848105860 2024-06-11T13:23:28.067 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:27.314+0000 7f284d083640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f2848105860 0x7f2848105c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:44200/0 (socket says 172.21.15.57:44200) 2024-06-11T13:23:28.067 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:27.314+0000 7f284d083640 1 -- 172.21.15.57:0/95097197 learned_addr learned my addr 172.21.15.57:0/95097197 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:28.067 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:27.314+0000 7f283ffff640 1 -- 172.21.15.57:0/95097197 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3881495607 0 0) 0x7f2848109c90 con 0x7f2848105860 2024-06-11T13:23:28.067 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:27.314+0000 7f283ffff640 1 -- 172.21.15.57:0/95097197 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2834003520 con 0x7f2848105860 2024-06-11T13:23:28.067 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:27.315+0000 7f283ffff640 1 -- 172.21.15.57:0/95097197 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3831590733 0 0) 0x7f2834003520 con 0x7f2848105860 2024-06-11T13:23:28.067 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:27.315+0000 7f283ffff640 1 -- 172.21.15.57:0/95097197 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f284810ae70 con 0x7f2848105860 2024-06-11T13:23:28.067 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:27.315+0000 7f283ffff640 1 -- 172.21.15.57:0/95097197 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f2844001580 con 0x7f2848105860 2024-06-11T13:23:28.067 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:27.315+0000 7f283ffff640 1 -- 172.21.15.57:0/95097197 <== mon.0 v1:172.21.15.57:6789/0 4 ==== config(25 keys) ==== 1037+0+0 (unknown 2947037021 0 0) 0x7f2844001d20 con 0x7f2848105860 2024-06-11T13:23:28.067 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:27.315+0000 7f283ffff640 1 -- 172.21.15.57:0/95097197 <== mon.0 v1:172.21.15.57:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f28440058d0 con 0x7f2848105860 2024-06-11T13:23:28.067 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:27.315+0000 7f284f2e7640 1 -- 172.21.15.57:0/95097197 >> v1:172.21.15.57:6789/0 conn(0x7f2848105860 legacy=0x7f2848105c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:28.067 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:27.315+0000 7f284f2e7640 1 -- 172.21.15.57:0/95097197 shutdown_connections 2024-06-11T13:23:28.067 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:27.315+0000 7f284f2e7640 1 -- 172.21.15.57:0/95097197 >> 172.21.15.57:0/95097197 conn(0x7f2848100e00 msgr2=0x7f2848103240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:28.068 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:27.315+0000 7f284f2e7640 1 -- 172.21.15.57:0/95097197 shutdown_connections 2024-06-11T13:23:28.068 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:27.316+0000 7f284f2e7640 1 -- 172.21.15.57:0/95097197 wait complete. 2024-06-11T13:23:28.068 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:27.316+0000 7f284f2e7640 1 Processor -- start 2024-06-11T13:23:28.068 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:27.316+0000 7f284f2e7640 1 -- start start 2024-06-11T13:23:28.068 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:27.316+0000 7f284f2e7640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f2848199b10 con 0x7f2848105860 2024-06-11T13:23:28.068 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:27.316+0000 7f284d083640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f2848105860 0x7f2848199400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:44202/0 (socket says 172.21.15.57:44202) 2024-06-11T13:23:28.068 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:27.316+0000 7f284d083640 1 -- 172.21.15.57:0/861147263 learned_addr learned my addr 172.21.15.57:0/861147263 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:28.068 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:27.317+0000 7f283e7fc640 1 -- 172.21.15.57:0/861147263 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2591694092 0 0) 0x7f2848199b10 con 0x7f2848105860 2024-06-11T13:23:28.068 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:27.317+0000 7f283e7fc640 1 -- 172.21.15.57:0/861147263 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2834005410 con 0x7f2848105860 2024-06-11T13:23:28.068 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:27.317+0000 7f283e7fc640 1 -- 172.21.15.57:0/861147263 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1624659825 0 0) 0x7f2834005410 con 0x7f2848105860 2024-06-11T13:23:28.069 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:27.317+0000 7f283e7fc640 1 -- 172.21.15.57:0/861147263 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2848199b10 con 0x7f2848105860 2024-06-11T13:23:28.069 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:27.317+0000 7f283e7fc640 1 -- 172.21.15.57:0/861147263 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f28440047e0 con 0x7f2848105860 2024-06-11T13:23:28.069 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:27.317+0000 7f283e7fc640 1 -- 172.21.15.57:0/861147263 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3098084840 0 0) 0x7f2848199b10 con 0x7f2848105860 2024-06-11T13:23:28.069 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:27.317+0000 7f283e7fc640 1 -- 172.21.15.57:0/861147263 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2848199ce0 con 0x7f2848105860 2024-06-11T13:23:28.069 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:27.317+0000 7f284f2e7640 1 -- 172.21.15.57:0/861147263 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f2848199ff0 con 0x7f2848105860 2024-06-11T13:23:28.069 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:27.317+0000 7f284f2e7640 1 -- 172.21.15.57:0/861147263 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f284819db80 con 0x7f2848105860 2024-06-11T13:23:28.069 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:27.317+0000 7f283e7fc640 1 -- 172.21.15.57:0/861147263 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1037+0+0 (unknown 2947037021 0 0) 0x7f2844001190 con 0x7f2848105860 2024-06-11T13:23:28.069 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:27.317+0000 7f283e7fc640 1 -- 172.21.15.57:0/861147263 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f2844005ed0 con 0x7f2848105860 2024-06-11T13:23:28.069 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:27.318+0000 7f283e7fc640 1 -- 172.21.15.57:0/861147263 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 8) ==== 46252+0+0 (unknown 1643827861 0 0) 0x7f2844011530 con 0x7f2848105860 2024-06-11T13:23:28.069 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:27.318+0000 7f284f2e7640 1 -- 172.21.15.57:0/861147263 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f282c0050b0 con 0x7f2848105860 2024-06-11T13:23:28.069 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:27.318+0000 7f283e7fc640 1 -- 172.21.15.57:0/861147263 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (unknown 4180383460 0 0) 0x7f28440477c0 con 0x7f2848105860 2024-06-11T13:23:28.069 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:27.323+0000 7f283e7fc640 1 -- 172.21.15.57:0/861147263 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f2844001cc0 con 0x7f2848105860 2024-06-11T13:23:28.069 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:27.511+0000 7f284f2e7640 1 -- 172.21.15.57:0/861147263 --> v1:172.21.15.57:6800/1064738157 -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f282c000bc0 con 0x7f283403a300 2024-06-11T13:23:28.069 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:27.512+0000 7f283e7fc640 1 -- 172.21.15.57:0/861147263 <== mgr.14118 v1:172.21.15.57:6800/1064738157 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (unknown 0 0 1568997241) 0x7f282c000bc0 con 0x7f283403a300 2024-06-11T13:23:28.070 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:27.514+0000 7f284f2e7640 1 -- 172.21.15.57:0/861147263 >> v1:172.21.15.57:6800/1064738157 conn(0x7f283403a300 legacy=0x7f283403c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:28.070 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:27.514+0000 7f284f2e7640 1 -- 172.21.15.57:0/861147263 >> v1:172.21.15.57:6789/0 conn(0x7f2848105860 legacy=0x7f2848199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:28.070 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:27.514+0000 7f284f2e7640 1 -- 172.21.15.57:0/861147263 shutdown_connections 2024-06-11T13:23:28.070 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:27.514+0000 7f284f2e7640 1 -- 172.21.15.57:0/861147263 >> 172.21.15.57:0/861147263 conn(0x7f2848100e00 msgr2=0x7f2848190190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:28.070 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:27.514+0000 7f284f2e7640 1 -- 172.21.15.57:0/861147263 shutdown_connections 2024-06-11T13:23:28.070 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:27.514+0000 7f284f2e7640 1 -- 172.21.15.57:0/861147263 wait complete. 2024-06-11T13:23:28.070 INFO:teuthology.orchestra.run.smithi057.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-06-11T13:23:28.070 INFO:teuthology.orchestra.run.smithi057.stdout:Adding key to root@localhost authorized_keys... 2024-06-11T13:23:28.070 INFO:teuthology.orchestra.run.smithi057.stdout:Adding host smithi057... 2024-06-11T13:23:28.780 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:28 smithi057 ceph-mon[29805]: from='client.14136 v1:172.21.15.57:0/861147263' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:23:29.711 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:29 smithi057 ceph-mon[29805]: from='client.14138 v1:172.21.15.57:0/1632490701' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi057", "addr": "172.21.15.57", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:23:30.623 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:30 smithi057 ceph-mon[29805]: Deploying cephadm binary to smithi057 2024-06-11T13:23:33.200 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout Added host 'smithi057' with addr '172.21.15.57' 2024-06-11T13:23:33.200 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:28.480+0000 7fcac3787640 1 Processor -- start 2024-06-11T13:23:33.201 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:28.481+0000 7fcac3787640 1 -- start start 2024-06-11T13:23:33.201 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:28.481+0000 7fcac3787640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fcabc10be50 con 0x7fcabc107a20 2024-06-11T13:23:33.201 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:28.481+0000 7fcac1523640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fcabc107a20 0x7fcabc107e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:44218/0 (socket says 172.21.15.57:44218) 2024-06-11T13:23:33.201 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:28.481+0000 7fcac1523640 1 -- 172.21.15.57:0/2231673588 learned_addr learned my addr 172.21.15.57:0/2231673588 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:33.201 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:28.481+0000 7fcab3fff640 1 -- 172.21.15.57:0/2231673588 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1341307838 0 0) 0x7fcabc10be50 con 0x7fcabc107a20 2024-06-11T13:23:33.201 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:28.481+0000 7fcab3fff640 1 -- 172.21.15.57:0/2231673588 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcaa8003520 con 0x7fcabc107a20 2024-06-11T13:23:33.201 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:28.482+0000 7fcab3fff640 1 -- 172.21.15.57:0/2231673588 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1690794215 0 0) 0x7fcaa8003520 con 0x7fcabc107a20 2024-06-11T13:23:33.201 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:28.482+0000 7fcab3fff640 1 -- 172.21.15.57:0/2231673588 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcabc10d030 con 0x7fcabc107a20 2024-06-11T13:23:33.201 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:28.482+0000 7fcab3fff640 1 -- 172.21.15.57:0/2231673588 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7fcab8001580 con 0x7fcabc107a20 2024-06-11T13:23:33.201 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:28.482+0000 7fcab3fff640 1 -- 172.21.15.57:0/2231673588 <== mon.0 v1:172.21.15.57:6789/0 4 ==== config(25 keys) ==== 1037+0+0 (unknown 2947037021 0 0) 0x7fcab8001d20 con 0x7fcabc107a20 2024-06-11T13:23:33.202 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:28.482+0000 7fcab3fff640 1 -- 172.21.15.57:0/2231673588 <== mon.0 v1:172.21.15.57:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7fcab80058d0 con 0x7fcabc107a20 2024-06-11T13:23:33.202 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:28.482+0000 7fcac3787640 1 -- 172.21.15.57:0/2231673588 >> v1:172.21.15.57:6789/0 conn(0x7fcabc107a20 legacy=0x7fcabc107e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:33.202 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:28.482+0000 7fcac3787640 1 -- 172.21.15.57:0/2231673588 shutdown_connections 2024-06-11T13:23:33.202 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:28.482+0000 7fcac3787640 1 -- 172.21.15.57:0/2231673588 >> 172.21.15.57:0/2231673588 conn(0x7fcabc07aea0 msgr2=0x7fcabc07b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:33.202 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:28.483+0000 7fcac3787640 1 -- 172.21.15.57:0/2231673588 shutdown_connections 2024-06-11T13:23:33.202 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:28.483+0000 7fcac3787640 1 -- 172.21.15.57:0/2231673588 wait complete. 2024-06-11T13:23:33.202 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:28.483+0000 7fcac3787640 1 Processor -- start 2024-06-11T13:23:33.202 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:28.483+0000 7fcac3787640 1 -- start start 2024-06-11T13:23:33.202 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:28.483+0000 7fcac3787640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fcabc19bd30 con 0x7fcabc107a20 2024-06-11T13:23:33.202 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:28.484+0000 7fcac1523640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fcabc107a20 0x7fcabc19b620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:44220/0 (socket says 172.21.15.57:44220) 2024-06-11T13:23:33.202 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:28.484+0000 7fcac1523640 1 -- 172.21.15.57:0/1632490701 learned_addr learned my addr 172.21.15.57:0/1632490701 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:33.202 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:28.484+0000 7fcab27fc640 1 -- 172.21.15.57:0/1632490701 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4208218543 0 0) 0x7fcabc19bd30 con 0x7fcabc107a20 2024-06-11T13:23:33.203 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:28.484+0000 7fcab27fc640 1 -- 172.21.15.57:0/1632490701 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcaa8005410 con 0x7fcabc107a20 2024-06-11T13:23:33.203 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:28.484+0000 7fcab27fc640 1 -- 172.21.15.57:0/1632490701 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 349658278 0 0) 0x7fcaa8005410 con 0x7fcabc107a20 2024-06-11T13:23:33.203 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:28.484+0000 7fcab27fc640 1 -- 172.21.15.57:0/1632490701 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fcabc19bd30 con 0x7fcabc107a20 2024-06-11T13:23:33.203 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:28.484+0000 7fcab27fc640 1 -- 172.21.15.57:0/1632490701 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7fcab80047e0 con 0x7fcabc107a20 2024-06-11T13:23:33.203 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:28.484+0000 7fcab27fc640 1 -- 172.21.15.57:0/1632490701 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 364655633 0 0) 0x7fcabc19bd30 con 0x7fcabc107a20 2024-06-11T13:23:33.203 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:28.484+0000 7fcab27fc640 1 -- 172.21.15.57:0/1632490701 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcabc19bf00 con 0x7fcabc107a20 2024-06-11T13:23:33.203 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:28.484+0000 7fcac3787640 1 -- 172.21.15.57:0/1632490701 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fcabc19c210 con 0x7fcabc107a20 2024-06-11T13:23:33.203 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:28.484+0000 7fcac3787640 1 -- 172.21.15.57:0/1632490701 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fcabc19fda0 con 0x7fcabc107a20 2024-06-11T13:23:33.204 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:28.484+0000 7fcab27fc640 1 -- 172.21.15.57:0/1632490701 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1037+0+0 (unknown 2947037021 0 0) 0x7fcab80049b0 con 0x7fcabc107a20 2024-06-11T13:23:33.204 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:28.484+0000 7fcab27fc640 1 -- 172.21.15.57:0/1632490701 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7fcab8006100 con 0x7fcabc107a20 2024-06-11T13:23:33.204 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:28.485+0000 7fcab27fc640 1 -- 172.21.15.57:0/1632490701 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 8) ==== 46252+0+0 (unknown 1643827861 0 0) 0x7fcab8011840 con 0x7fcabc107a20 2024-06-11T13:23:33.204 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:28.485+0000 7fcac3787640 1 -- 172.21.15.57:0/1632490701 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaa00050b0 con 0x7fcabc107a20 2024-06-11T13:23:33.204 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:28.486+0000 7fcab27fc640 1 -- 172.21.15.57:0/1632490701 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (unknown 4180383460 0 0) 0x7fcab8047b20 con 0x7fcabc107a20 2024-06-11T13:23:33.204 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:28.490+0000 7fcab27fc640 1 -- 172.21.15.57:0/1632490701 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7fcab8017480 con 0x7fcabc107a20 2024-06-11T13:23:33.204 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:28.680+0000 7fcac3787640 1 -- 172.21.15.57:0/1632490701 --> v1:172.21.15.57:6800/1064738157 -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi057", "addr": "172.21.15.57", "target": ["mon-mgr", ""]}) -- 0x7fcaa0000bc0 con 0x7fcaa803a300 2024-06-11T13:23:33.204 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:32.662+0000 7fcab27fc640 1 -- 172.21.15.57:0/1632490701 <== mgr.14118 v1:172.21.15.57:6800/1064738157 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (unknown 0 0 640604986) 0x7fcaa0000bc0 con 0x7fcaa803a300 2024-06-11T13:23:33.204 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:32.664+0000 7fcac3787640 1 -- 172.21.15.57:0/1632490701 >> v1:172.21.15.57:6800/1064738157 conn(0x7fcaa803a300 legacy=0x7fcaa803c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:33.204 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:32.664+0000 7fcac3787640 1 -- 172.21.15.57:0/1632490701 >> v1:172.21.15.57:6789/0 conn(0x7fcabc107a20 legacy=0x7fcabc19b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:33.204 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:32.665+0000 7fcac3787640 1 -- 172.21.15.57:0/1632490701 shutdown_connections 2024-06-11T13:23:33.204 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:32.665+0000 7fcac3787640 1 -- 172.21.15.57:0/1632490701 >> 172.21.15.57:0/1632490701 conn(0x7fcabc07aea0 msgr2=0x7fcabc106610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:33.205 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:32.665+0000 7fcac3787640 1 -- 172.21.15.57:0/1632490701 shutdown_connections 2024-06-11T13:23:33.205 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:32.665+0000 7fcac3787640 1 -- 172.21.15.57:0/1632490701 wait complete. 2024-06-11T13:23:33.205 INFO:teuthology.orchestra.run.smithi057.stdout:Deploying unmanaged mon service... 2024-06-11T13:23:33.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:33 smithi057 ceph-mon[29805]: from='mgr.14118 v1:172.21.15.57:0/2109654694' entity='mgr.y' 2024-06-11T13:23:33.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:33 smithi057 ceph-mon[29805]: Added host smithi057 2024-06-11T13:23:33.781 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:33 smithi057 ceph-mon[29805]: from='mgr.14118 v1:172.21.15.57:0/2109654694' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:23:34.568 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-06-11T13:23:34.568 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:33.645+0000 7f762ac6d640 1 Processor -- start 2024-06-11T13:23:34.568 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:33.645+0000 7f762ac6d640 1 -- start start 2024-06-11T13:23:34.568 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:33.645+0000 7f762ac6d640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f762407a330 con 0x7f762407b730 2024-06-11T13:23:34.568 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:33.645+0000 7f7628a09640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f762407b730 0x7f7624079b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:57200/0 (socket says 172.21.15.57:57200) 2024-06-11T13:23:34.569 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:33.645+0000 7f7628a09640 1 -- 172.21.15.57:0/1496148390 learned_addr learned my addr 172.21.15.57:0/1496148390 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:34.569 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:33.645+0000 7f76237fe640 1 -- 172.21.15.57:0/1496148390 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 525965956 0 0) 0x7f762407a330 con 0x7f762407b730 2024-06-11T13:23:34.569 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:33.645+0000 7f76237fe640 1 -- 172.21.15.57:0/1496148390 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7614003520 con 0x7f762407b730 2024-06-11T13:23:34.569 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:33.646+0000 7f76237fe640 1 -- 172.21.15.57:0/1496148390 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1119241702 0 0) 0x7f7614003520 con 0x7f762407b730 2024-06-11T13:23:34.569 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:33.646+0000 7f76237fe640 1 -- 172.21.15.57:0/1496148390 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f762407a500 con 0x7f762407b730 2024-06-11T13:23:34.569 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:33.646+0000 7f76237fe640 1 -- 172.21.15.57:0/1496148390 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f7618001580 con 0x7f762407b730 2024-06-11T13:23:34.569 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:33.646+0000 7f76237fe640 1 -- 172.21.15.57:0/1496148390 <== mon.0 v1:172.21.15.57:6789/0 4 ==== config(25 keys) ==== 1037+0+0 (unknown 2947037021 0 0) 0x7f7618001c50 con 0x7f762407b730 2024-06-11T13:23:34.569 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:33.646+0000 7f76237fe640 1 -- 172.21.15.57:0/1496148390 <== mon.0 v1:172.21.15.57:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f76180058d0 con 0x7f762407b730 2024-06-11T13:23:34.569 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:33.646+0000 7f762ac6d640 1 -- 172.21.15.57:0/1496148390 >> v1:172.21.15.57:6789/0 conn(0x7f762407b730 legacy=0x7f7624079b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:34.570 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:33.646+0000 7f762ac6d640 1 -- 172.21.15.57:0/1496148390 shutdown_connections 2024-06-11T13:23:34.570 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:33.646+0000 7f762ac6d640 1 -- 172.21.15.57:0/1496148390 >> 172.21.15.57:0/1496148390 conn(0x7f7624100e00 msgr2=0x7f7624103240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:34.570 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:33.646+0000 7f762ac6d640 1 -- 172.21.15.57:0/1496148390 shutdown_connections 2024-06-11T13:23:34.570 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:33.646+0000 7f762ac6d640 1 -- 172.21.15.57:0/1496148390 wait complete. 2024-06-11T13:23:34.570 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:33.647+0000 7f762ac6d640 1 Processor -- start 2024-06-11T13:23:34.570 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:33.647+0000 7f762ac6d640 1 -- start start 2024-06-11T13:23:34.570 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:33.647+0000 7f762ac6d640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f762419bd60 con 0x7f762407b730 2024-06-11T13:23:34.570 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:33.647+0000 7f7628a09640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f762407b730 0x7f762419b650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:57208/0 (socket says 172.21.15.57:57208) 2024-06-11T13:23:34.570 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:33.647+0000 7f7628a09640 1 -- 172.21.15.57:0/3268308856 learned_addr learned my addr 172.21.15.57:0/3268308856 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:34.570 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:33.647+0000 7f7621ffb640 1 -- 172.21.15.57:0/3268308856 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 995599296 0 0) 0x7f762419bd60 con 0x7f762407b730 2024-06-11T13:23:34.570 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:33.647+0000 7f7621ffb640 1 -- 172.21.15.57:0/3268308856 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7614005460 con 0x7f762407b730 2024-06-11T13:23:34.570 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:33.648+0000 7f7621ffb640 1 -- 172.21.15.57:0/3268308856 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3537737378 0 0) 0x7f7614005460 con 0x7f762407b730 2024-06-11T13:23:34.570 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:33.648+0000 7f7621ffb640 1 -- 172.21.15.57:0/3268308856 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f762419bd60 con 0x7f762407b730 2024-06-11T13:23:34.570 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:33.648+0000 7f7621ffb640 1 -- 172.21.15.57:0/3268308856 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f7618001080 con 0x7f762407b730 2024-06-11T13:23:34.571 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:33.648+0000 7f7621ffb640 1 -- 172.21.15.57:0/3268308856 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2017119356 0 0) 0x7f762419bd60 con 0x7f762407b730 2024-06-11T13:23:34.571 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:33.648+0000 7f7621ffb640 1 -- 172.21.15.57:0/3268308856 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f762419bf30 con 0x7f762407b730 2024-06-11T13:23:34.571 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:33.648+0000 7f762ac6d640 1 -- 172.21.15.57:0/3268308856 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f762419c240 con 0x7f762407b730 2024-06-11T13:23:34.571 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:33.648+0000 7f7621ffb640 1 -- 172.21.15.57:0/3268308856 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1037+0+0 (unknown 2947037021 0 0) 0x7f7618004940 con 0x7f762407b730 2024-06-11T13:23:34.571 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:33.648+0000 7f7621ffb640 1 -- 172.21.15.57:0/3268308856 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f76180061f0 con 0x7f762407b730 2024-06-11T13:23:34.571 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:33.648+0000 7f762ac6d640 1 -- 172.21.15.57:0/3268308856 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f762419fdd0 con 0x7f762407b730 2024-06-11T13:23:34.571 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:33.649+0000 7f7621ffb640 1 -- 172.21.15.57:0/3268308856 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 8) ==== 46252+0+0 (unknown 1643827861 0 0) 0x7f76180119a0 con 0x7f762407b730 2024-06-11T13:23:34.571 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:33.649+0000 7f7621ffb640 1 -- 172.21.15.57:0/3268308856 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (unknown 4180383460 0 0) 0x7f7618047860 con 0x7f762407b730 2024-06-11T13:23:34.571 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:33.649+0000 7f762ac6d640 1 -- 172.21.15.57:0/3268308856 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f762410cf00 con 0x7f762407b730 2024-06-11T13:23:34.571 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:33.654+0000 7f7621ffb640 1 -- 172.21.15.57:0/3268308856 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f7618017530 con 0x7f762407b730 2024-06-11T13:23:34.571 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:33.871+0000 7f762ac6d640 1 -- 172.21.15.57:0/3268308856 --> v1:172.21.15.57:6800/1064738157 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f762410ba90 con 0x7f761403a440 2024-06-11T13:23:34.571 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:33.877+0000 7f7621ffb640 1 -- 172.21.15.57:0/3268308856 <== mgr.14118 v1:172.21.15.57:6800/1064738157 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (unknown 0 0 3265049985) 0x7f762410ba90 con 0x7f761403a440 2024-06-11T13:23:34.571 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:33.879+0000 7f762ac6d640 1 -- 172.21.15.57:0/3268308856 >> v1:172.21.15.57:6800/1064738157 conn(0x7f761403a440 legacy=0x7f761403c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:34.571 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:33.879+0000 7f762ac6d640 1 -- 172.21.15.57:0/3268308856 >> v1:172.21.15.57:6789/0 conn(0x7f762407b730 legacy=0x7f762419b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:34.572 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:33.879+0000 7f762ac6d640 1 -- 172.21.15.57:0/3268308856 shutdown_connections 2024-06-11T13:23:34.572 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:33.879+0000 7f762ac6d640 1 -- 172.21.15.57:0/3268308856 >> 172.21.15.57:0/3268308856 conn(0x7f7624100e00 msgr2=0x7f7624109d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:34.572 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:33.880+0000 7f762ac6d640 1 -- 172.21.15.57:0/3268308856 shutdown_connections 2024-06-11T13:23:34.572 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:33.880+0000 7f762ac6d640 1 -- 172.21.15.57:0/3268308856 wait complete. 2024-06-11T13:23:34.572 INFO:teuthology.orchestra.run.smithi057.stdout:Deploying unmanaged mgr service... 2024-06-11T13:23:35.145 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:34 smithi057 ceph-mon[29805]: from='client.14140 v1:172.21.15.57:0/3268308856' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:23:35.145 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:34 smithi057 ceph-mon[29805]: Saving service mon spec with placement count:5 2024-06-11T13:23:35.145 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:34 smithi057 ceph-mon[29805]: from='mgr.14118 v1:172.21.15.57:0/2109654694' entity='mgr.y' 2024-06-11T13:23:35.836 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-06-11T13:23:35.836 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:34.974+0000 7f1fac490640 1 Processor -- start 2024-06-11T13:23:35.836 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:34.974+0000 7f1fac490640 1 -- start start 2024-06-11T13:23:35.836 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:34.974+0000 7f1fac490640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f1fa4109c90 con 0x7f1fa4105860 2024-06-11T13:23:35.836 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:34.974+0000 7f1faa22c640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f1fa4105860 0x7f1fa4105c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:57212/0 (socket says 172.21.15.57:57212) 2024-06-11T13:23:35.836 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:34.974+0000 7f1faa22c640 1 -- 172.21.15.57:0/3150039384 learned_addr learned my addr 172.21.15.57:0/3150039384 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:35.836 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:34.975+0000 7f1fa922a640 1 -- 172.21.15.57:0/3150039384 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3654694026 0 0) 0x7f1fa4109c90 con 0x7f1fa4105860 2024-06-11T13:23:35.837 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:34.975+0000 7f1fa922a640 1 -- 172.21.15.57:0/3150039384 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f90003520 con 0x7f1fa4105860 2024-06-11T13:23:35.837 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:34.975+0000 7f1fa922a640 1 -- 172.21.15.57:0/3150039384 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3887780245 0 0) 0x7f1f90003520 con 0x7f1fa4105860 2024-06-11T13:23:35.837 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:34.975+0000 7f1fa922a640 1 -- 172.21.15.57:0/3150039384 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa410ae70 con 0x7f1fa4105860 2024-06-11T13:23:35.837 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:34.975+0000 7f1fa922a640 1 -- 172.21.15.57:0/3150039384 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f1fa0001580 con 0x7f1fa4105860 2024-06-11T13:23:35.837 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:34.975+0000 7f1fa922a640 1 -- 172.21.15.57:0/3150039384 <== mon.0 v1:172.21.15.57:6789/0 4 ==== config(25 keys) ==== 1037+0+0 (unknown 2947037021 0 0) 0x7f1fa0001d20 con 0x7f1fa4105860 2024-06-11T13:23:35.837 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:34.975+0000 7f1fa922a640 1 -- 172.21.15.57:0/3150039384 <== mon.0 v1:172.21.15.57:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f1fa00058d0 con 0x7f1fa4105860 2024-06-11T13:23:35.837 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:34.975+0000 7f1fac490640 1 -- 172.21.15.57:0/3150039384 >> v1:172.21.15.57:6789/0 conn(0x7f1fa4105860 legacy=0x7f1fa4105c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:35.837 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:34.976+0000 7f1fac490640 1 -- 172.21.15.57:0/3150039384 shutdown_connections 2024-06-11T13:23:35.837 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:34.976+0000 7f1fac490640 1 -- 172.21.15.57:0/3150039384 >> 172.21.15.57:0/3150039384 conn(0x7f1fa4100e00 msgr2=0x7f1fa4103240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:35.837 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:34.976+0000 7f1fac490640 1 -- 172.21.15.57:0/3150039384 shutdown_connections 2024-06-11T13:23:35.837 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:34.976+0000 7f1fac490640 1 -- 172.21.15.57:0/3150039384 wait complete. 2024-06-11T13:23:35.837 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:34.976+0000 7f1fac490640 1 Processor -- start 2024-06-11T13:23:35.837 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:34.976+0000 7f1fac490640 1 -- start start 2024-06-11T13:23:35.838 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:34.976+0000 7f1fac490640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f1fa4199b10 con 0x7f1fa4105860 2024-06-11T13:23:35.838 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:34.976+0000 7f1faa22c640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f1fa4105860 0x7f1fa4199400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:57224/0 (socket says 172.21.15.57:57224) 2024-06-11T13:23:35.838 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:34.976+0000 7f1faa22c640 1 -- 172.21.15.57:0/3499370604 learned_addr learned my addr 172.21.15.57:0/3499370604 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:35.838 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:34.977+0000 7f1f9b7fe640 1 -- 172.21.15.57:0/3499370604 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2641916482 0 0) 0x7f1fa4199b10 con 0x7f1fa4105860 2024-06-11T13:23:35.838 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:34.977+0000 7f1f9b7fe640 1 -- 172.21.15.57:0/3499370604 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f90005410 con 0x7f1fa4105860 2024-06-11T13:23:35.838 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:34.977+0000 7f1f9b7fe640 1 -- 172.21.15.57:0/3499370604 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2208532641 0 0) 0x7f1f90005410 con 0x7f1fa4105860 2024-06-11T13:23:35.838 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:34.977+0000 7f1f9b7fe640 1 -- 172.21.15.57:0/3499370604 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1fa4199b10 con 0x7f1fa4105860 2024-06-11T13:23:35.838 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:34.977+0000 7f1f9b7fe640 1 -- 172.21.15.57:0/3499370604 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f1fa00047e0 con 0x7f1fa4105860 2024-06-11T13:23:35.838 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:34.977+0000 7f1f9b7fe640 1 -- 172.21.15.57:0/3499370604 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 430507984 0 0) 0x7f1fa4199b10 con 0x7f1fa4105860 2024-06-11T13:23:35.838 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:34.977+0000 7f1f9b7fe640 1 -- 172.21.15.57:0/3499370604 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa4199ce0 con 0x7f1fa4105860 2024-06-11T13:23:35.838 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:34.977+0000 7f1fac490640 1 -- 172.21.15.57:0/3499370604 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f1fa4199ff0 con 0x7f1fa4105860 2024-06-11T13:23:35.838 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:34.977+0000 7f1fac490640 1 -- 172.21.15.57:0/3499370604 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f1fa419db80 con 0x7f1fa4105860 2024-06-11T13:23:35.838 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:34.977+0000 7f1f9b7fe640 1 -- 172.21.15.57:0/3499370604 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1037+0+0 (unknown 2947037021 0 0) 0x7f1fa0001c00 con 0x7f1fa4105860 2024-06-11T13:23:35.839 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:34.977+0000 7f1f9b7fe640 1 -- 172.21.15.57:0/3499370604 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f1fa0005e80 con 0x7f1fa4105860 2024-06-11T13:23:35.839 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:34.978+0000 7f1f9b7fe640 1 -- 172.21.15.57:0/3499370604 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 8) ==== 46252+0+0 (unknown 1643827861 0 0) 0x7f1fa00114e0 con 0x7f1fa4105860 2024-06-11T13:23:35.839 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:34.978+0000 7f1fac490640 1 -- 172.21.15.57:0/3499370604 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f880050b0 con 0x7f1fa4105860 2024-06-11T13:23:35.839 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:34.978+0000 7f1f9b7fe640 1 -- 172.21.15.57:0/3499370604 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (unknown 4180383460 0 0) 0x7f1fa00477a0 con 0x7f1fa4105860 2024-06-11T13:23:35.839 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:34.983+0000 7f1f9b7fe640 1 -- 172.21.15.57:0/3499370604 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f1fa00173f0 con 0x7f1fa4105860 2024-06-11T13:23:35.839 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:35.181+0000 7f1fac490640 1 -- 172.21.15.57:0/3499370604 --> v1:172.21.15.57:6800/1064738157 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f1f88000bc0 con 0x7f1f9003a300 2024-06-11T13:23:35.839 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:35.186+0000 7f1f9b7fe640 1 -- 172.21.15.57:0/3499370604 <== mgr.14118 v1:172.21.15.57:6800/1064738157 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (unknown 0 0 325935098) 0x7f1f88000bc0 con 0x7f1f9003a300 2024-06-11T13:23:35.839 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:35.188+0000 7f1fac490640 1 -- 172.21.15.57:0/3499370604 >> v1:172.21.15.57:6800/1064738157 conn(0x7f1f9003a300 legacy=0x7f1f9003c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:35.839 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:35.188+0000 7f1fac490640 1 -- 172.21.15.57:0/3499370604 >> v1:172.21.15.57:6789/0 conn(0x7f1fa4105860 legacy=0x7f1fa4199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:35.839 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:35.188+0000 7f1fac490640 1 -- 172.21.15.57:0/3499370604 shutdown_connections 2024-06-11T13:23:35.839 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:35.188+0000 7f1fac490640 1 -- 172.21.15.57:0/3499370604 >> 172.21.15.57:0/3499370604 conn(0x7f1fa4100e00 msgr2=0x7f1fa4190190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:35.839 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:35.188+0000 7f1fac490640 1 -- 172.21.15.57:0/3499370604 shutdown_connections 2024-06-11T13:23:35.839 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:35.189+0000 7f1fac490640 1 -- 172.21.15.57:0/3499370604 wait complete. 2024-06-11T13:23:36.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:36 smithi057 ceph-mon[29805]: from='client.14142 v1:172.21.15.57:0/3499370604' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:23:36.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:36 smithi057 ceph-mon[29805]: Saving service mgr spec with placement count:2 2024-06-11T13:23:36.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:36 smithi057 ceph-mon[29805]: from='mgr.14118 v1:172.21.15.57:0/2109654694' entity='mgr.y' 2024-06-11T13:23:37.848 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:37 smithi057 ceph-mon[29805]: from='mgr.14118 v1:172.21.15.57:0/2109654694' entity='mgr.y' 2024-06-11T13:23:37.848 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:37 smithi057 ceph-mon[29805]: from='mgr.14118 v1:172.21.15.57:0/2109654694' entity='mgr.y' 2024-06-11T13:23:37.849 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:37 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3337602230' entity='client.admin' 2024-06-11T13:23:38.451 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:37.078+0000 7f9578cb0640 1 Processor -- start 2024-06-11T13:23:38.451 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:37.078+0000 7f9578cb0640 1 -- start start 2024-06-11T13:23:38.451 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:37.078+0000 7f9578cb0640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f9574108b80 con 0x7f9574104750 2024-06-11T13:23:38.451 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:37.078+0000 7f957259c640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f9574104750 0x7f9574104b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:57238/0 (socket says 172.21.15.57:57238) 2024-06-11T13:23:38.451 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:37.078+0000 7f957259c640 1 -- 172.21.15.57:0/3415478 learned_addr learned my addr 172.21.15.57:0/3415478 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:38.451 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:37.079+0000 7f957159a640 1 -- 172.21.15.57:0/3415478 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3297683883 0 0) 0x7f9574108b80 con 0x7f9574104750 2024-06-11T13:23:38.451 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:37.079+0000 7f957159a640 1 -- 172.21.15.57:0/3415478 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f955c003520 con 0x7f9574104750 2024-06-11T13:23:38.451 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:37.079+0000 7f957159a640 1 -- 172.21.15.57:0/3415478 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2975167537 0 0) 0x7f955c003520 con 0x7f9574104750 2024-06-11T13:23:38.451 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:37.079+0000 7f957159a640 1 -- 172.21.15.57:0/3415478 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9574109d60 con 0x7f9574104750 2024-06-11T13:23:38.452 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:37.079+0000 7f957159a640 1 -- 172.21.15.57:0/3415478 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f9564001580 con 0x7f9574104750 2024-06-11T13:23:38.452 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:37.079+0000 7f957159a640 1 -- 172.21.15.57:0/3415478 <== mon.0 v1:172.21.15.57:6789/0 4 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f95640048a0 con 0x7f9574104750 2024-06-11T13:23:38.452 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:37.079+0000 7f957159a640 1 -- 172.21.15.57:0/3415478 <== mon.0 v1:172.21.15.57:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f95640058d0 con 0x7f9574104750 2024-06-11T13:23:38.452 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:37.080+0000 7f9578cb0640 1 -- 172.21.15.57:0/3415478 >> v1:172.21.15.57:6789/0 conn(0x7f9574104750 legacy=0x7f9574104b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:38.452 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:37.080+0000 7f9578cb0640 1 -- 172.21.15.57:0/3415478 shutdown_connections 2024-06-11T13:23:38.452 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:37.080+0000 7f9578cb0640 1 -- 172.21.15.57:0/3415478 >> 172.21.15.57:0/3415478 conn(0x7f95740ffcb0 msgr2=0x7f9574102130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:38.452 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:37.080+0000 7f9578cb0640 1 -- 172.21.15.57:0/3415478 shutdown_connections 2024-06-11T13:23:38.452 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:37.080+0000 7f9578cb0640 1 -- 172.21.15.57:0/3415478 wait complete. 2024-06-11T13:23:38.452 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:37.081+0000 7f9578cb0640 1 Processor -- start 2024-06-11T13:23:38.452 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:37.081+0000 7f9578cb0640 1 -- start start 2024-06-11T13:23:38.452 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:37.081+0000 7f9578cb0640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f9574199b30 con 0x7f9574104750 2024-06-11T13:23:38.452 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:37.081+0000 7f957259c640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f9574104750 0x7f9574199420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:57250/0 (socket says 172.21.15.57:57250) 2024-06-11T13:23:38.452 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:37.081+0000 7f957259c640 1 -- 172.21.15.57:0/3337602230 learned_addr learned my addr 172.21.15.57:0/3337602230 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:38.453 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:37.081+0000 7f95637fe640 1 -- 172.21.15.57:0/3337602230 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2704776466 0 0) 0x7f9574199b30 con 0x7f9574104750 2024-06-11T13:23:38.453 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:37.081+0000 7f95637fe640 1 -- 172.21.15.57:0/3337602230 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f955c005410 con 0x7f9574104750 2024-06-11T13:23:38.453 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:37.082+0000 7f95637fe640 1 -- 172.21.15.57:0/3337602230 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3829906324 0 0) 0x7f955c005410 con 0x7f9574104750 2024-06-11T13:23:38.453 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:37.082+0000 7f95637fe640 1 -- 172.21.15.57:0/3337602230 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9574199b30 con 0x7f9574104750 2024-06-11T13:23:38.453 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:37.082+0000 7f95637fe640 1 -- 172.21.15.57:0/3337602230 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f9564001080 con 0x7f9574104750 2024-06-11T13:23:38.453 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:37.082+0000 7f95637fe640 1 -- 172.21.15.57:0/3337602230 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2822488857 0 0) 0x7f9574199b30 con 0x7f9574104750 2024-06-11T13:23:38.453 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:37.082+0000 7f95637fe640 1 -- 172.21.15.57:0/3337602230 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9574199d00 con 0x7f9574104750 2024-06-11T13:23:38.453 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:37.082+0000 7f9578cb0640 1 -- 172.21.15.57:0/3337602230 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f957419a010 con 0x7f9574104750 2024-06-11T13:23:38.453 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:37.082+0000 7f9578cb0640 1 -- 172.21.15.57:0/3337602230 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f957419dba0 con 0x7f9574104750 2024-06-11T13:23:38.454 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:37.082+0000 7f95637fe640 1 -- 172.21.15.57:0/3337602230 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f95640048a0 con 0x7f9574104750 2024-06-11T13:23:38.454 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:37.082+0000 7f95637fe640 1 -- 172.21.15.57:0/3337602230 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f9564005f30 con 0x7f9574104750 2024-06-11T13:23:38.454 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:37.083+0000 7f95637fe640 1 -- 172.21.15.57:0/3337602230 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 8) ==== 46252+0+0 (unknown 1643827861 0 0) 0x7f9564011650 con 0x7f9574104750 2024-06-11T13:23:38.454 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:37.083+0000 7f9578cb0640 1 -- 172.21.15.57:0/3337602230 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95540050b0 con 0x7f9574104750 2024-06-11T13:23:38.454 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:37.083+0000 7f95637fe640 1 -- 172.21.15.57:0/3337602230 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (unknown 4180383460 0 0) 0x7f9564047900 con 0x7f9574104750 2024-06-11T13:23:38.454 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:37.088+0000 7f95637fe640 1 -- 172.21.15.57:0/3337602230 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f9564017570 con 0x7f9574104750 2024-06-11T13:23:38.454 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:37.309+0000 7f9578cb0640 1 -- 172.21.15.57:0/3337602230 --> v1:172.21.15.57:6789/0 -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f95540053a0 con 0x7f9574104750 2024-06-11T13:23:38.454 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:37.311+0000 7f95637fe640 1 -- 172.21.15.57:0/3337602230 <== mon.0 v1:172.21.15.57:6789/0 10 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (unknown 2684808024 0 0) 0x7f9564016e70 con 0x7f9574104750 2024-06-11T13:23:38.454 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:37.314+0000 7f9578cb0640 1 -- 172.21.15.57:0/3337602230 >> v1:172.21.15.57:6800/1064738157 conn(0x7f955c03a450 legacy=0x7f955c03c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:38.454 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:37.314+0000 7f9578cb0640 1 -- 172.21.15.57:0/3337602230 >> v1:172.21.15.57:6789/0 conn(0x7f9574104750 legacy=0x7f9574199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:38.454 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:37.314+0000 7f9578cb0640 1 -- 172.21.15.57:0/3337602230 shutdown_connections 2024-06-11T13:23:38.454 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:37.314+0000 7f9578cb0640 1 -- 172.21.15.57:0/3337602230 >> 172.21.15.57:0/3337602230 conn(0x7f95740ffcb0 msgr2=0x7f9574107c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:38.455 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:37.314+0000 7f9578cb0640 1 -- 172.21.15.57:0/3337602230 shutdown_connections 2024-06-11T13:23:38.455 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:37.314+0000 7f9578cb0640 1 -- 172.21.15.57:0/3337602230 wait complete. 2024-06-11T13:23:39.090 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:39 smithi057 ceph-mon[29805]: from='mgr.14118 v1:172.21.15.57:0/2109654694' entity='mgr.y' 2024-06-11T13:23:39.090 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:39 smithi057 ceph-mon[29805]: from='mgr.14118 v1:172.21.15.57:0/2109654694' entity='mgr.y' 2024-06-11T13:23:39.666 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:38.889+0000 7fd4703f6640 1 Processor -- start 2024-06-11T13:23:39.666 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:38.889+0000 7fd4703f6640 1 -- start start 2024-06-11T13:23:39.666 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:38.889+0000 7fd4703f6640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fd468109c90 con 0x7fd468105860 2024-06-11T13:23:39.666 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:38.890+0000 7fd46e192640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fd468105860 0x7fd468105c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:41676/0 (socket says 172.21.15.57:41676) 2024-06-11T13:23:39.666 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:38.890+0000 7fd46e192640 1 -- 172.21.15.57:0/3371956934 learned_addr learned my addr 172.21.15.57:0/3371956934 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:39.666 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:38.890+0000 7fd46d190640 1 -- 172.21.15.57:0/3371956934 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 663657392 0 0) 0x7fd468109c90 con 0x7fd468105860 2024-06-11T13:23:39.667 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:38.890+0000 7fd46d190640 1 -- 172.21.15.57:0/3371956934 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd454003520 con 0x7fd468105860 2024-06-11T13:23:39.667 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:38.890+0000 7fd46d190640 1 -- 172.21.15.57:0/3371956934 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 270585245 0 0) 0x7fd454003520 con 0x7fd468105860 2024-06-11T13:23:39.667 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:38.890+0000 7fd46d190640 1 -- 172.21.15.57:0/3371956934 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd46810ae70 con 0x7fd468105860 2024-06-11T13:23:39.667 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:38.890+0000 7fd46d190640 1 -- 172.21.15.57:0/3371956934 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7fd464001580 con 0x7fd468105860 2024-06-11T13:23:39.667 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:38.890+0000 7fd46d190640 1 -- 172.21.15.57:0/3371956934 <== mon.0 v1:172.21.15.57:6789/0 4 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7fd4640048a0 con 0x7fd468105860 2024-06-11T13:23:39.667 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:38.890+0000 7fd46d190640 1 -- 172.21.15.57:0/3371956934 <== mon.0 v1:172.21.15.57:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7fd4640058d0 con 0x7fd468105860 2024-06-11T13:23:39.667 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:38.891+0000 7fd4703f6640 1 -- 172.21.15.57:0/3371956934 >> v1:172.21.15.57:6789/0 conn(0x7fd468105860 legacy=0x7fd468105c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:39.667 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:38.891+0000 7fd4703f6640 1 -- 172.21.15.57:0/3371956934 shutdown_connections 2024-06-11T13:23:39.667 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:38.891+0000 7fd4703f6640 1 -- 172.21.15.57:0/3371956934 >> 172.21.15.57:0/3371956934 conn(0x7fd468100e00 msgr2=0x7fd468103240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:39.667 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:38.891+0000 7fd4703f6640 1 -- 172.21.15.57:0/3371956934 shutdown_connections 2024-06-11T13:23:39.667 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:38.891+0000 7fd4703f6640 1 -- 172.21.15.57:0/3371956934 wait complete. 2024-06-11T13:23:39.667 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:38.892+0000 7fd4703f6640 1 Processor -- start 2024-06-11T13:23:39.667 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:38.892+0000 7fd4703f6640 1 -- start start 2024-06-11T13:23:39.667 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:38.892+0000 7fd4703f6640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fd46819f490 con 0x7fd468105860 2024-06-11T13:23:39.668 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:38.892+0000 7fd46e192640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fd468105860 0x7fd46807b420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:41690/0 (socket says 172.21.15.57:41690) 2024-06-11T13:23:39.668 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:38.892+0000 7fd46e192640 1 -- 172.21.15.57:0/436765534 learned_addr learned my addr 172.21.15.57:0/436765534 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:39.668 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:38.892+0000 7fd45f7fe640 1 -- 172.21.15.57:0/436765534 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4096874547 0 0) 0x7fd46819f490 con 0x7fd468105860 2024-06-11T13:23:39.668 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:38.892+0000 7fd45f7fe640 1 -- 172.21.15.57:0/436765534 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd454005410 con 0x7fd468105860 2024-06-11T13:23:39.668 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:38.892+0000 7fd45f7fe640 1 -- 172.21.15.57:0/436765534 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 88354625 0 0) 0x7fd454005410 con 0x7fd468105860 2024-06-11T13:23:39.668 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:38.892+0000 7fd45f7fe640 1 -- 172.21.15.57:0/436765534 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd46819f490 con 0x7fd468105860 2024-06-11T13:23:39.668 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:38.892+0000 7fd45f7fe640 1 -- 172.21.15.57:0/436765534 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7fd464001080 con 0x7fd468105860 2024-06-11T13:23:39.668 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:38.893+0000 7fd45f7fe640 1 -- 172.21.15.57:0/436765534 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3625064800 0 0) 0x7fd46819f490 con 0x7fd468105860 2024-06-11T13:23:39.668 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:38.893+0000 7fd45f7fe640 1 -- 172.21.15.57:0/436765534 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4681a0670 con 0x7fd468105860 2024-06-11T13:23:39.668 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:38.893+0000 7fd45f7fe640 1 -- 172.21.15.57:0/436765534 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7fd464001540 con 0x7fd468105860 2024-06-11T13:23:39.668 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:38.893+0000 7fd4703f6640 1 -- 172.21.15.57:0/436765534 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fd46819f660 con 0x7fd468105860 2024-06-11T13:23:39.668 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:38.893+0000 7fd45f7fe640 1 -- 172.21.15.57:0/436765534 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7fd464006240 con 0x7fd468105860 2024-06-11T13:23:39.668 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:38.893+0000 7fd4703f6640 1 -- 172.21.15.57:0/436765534 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fd46819fba0 con 0x7fd468105860 2024-06-11T13:23:39.668 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:38.894+0000 7fd45f7fe640 1 -- 172.21.15.57:0/436765534 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 8) ==== 46252+0+0 (unknown 1643827861 0 0) 0x7fd4640075b0 con 0x7fd468105860 2024-06-11T13:23:39.669 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:38.894+0000 7fd45f7fe640 1 -- 172.21.15.57:0/436765534 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (unknown 4180383460 0 0) 0x7fd4640479c0 con 0x7fd468105860 2024-06-11T13:23:39.669 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:38.894+0000 7fd4703f6640 1 -- 172.21.15.57:0/436765534 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd468063000 con 0x7fd468105860 2024-06-11T13:23:39.669 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:38.899+0000 7fd45f7fe640 1 -- 172.21.15.57:0/436765534 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7fd464017840 con 0x7fd468105860 2024-06-11T13:23:39.669 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:39.095+0000 7fd4703f6640 1 -- 172.21.15.57:0/436765534 --> v1:172.21.15.57:6789/0 -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fd46802d2e0 con 0x7fd468105860 2024-06-11T13:23:39.669 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:39.097+0000 7fd45f7fe640 1 -- 172.21.15.57:0/436765534 <== mon.0 v1:172.21.15.57:6789/0 10 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (unknown 1496729260 0 0) 0x7fd464004870 con 0x7fd468105860 2024-06-11T13:23:39.669 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:39.100+0000 7fd4703f6640 1 -- 172.21.15.57:0/436765534 >> v1:172.21.15.57:6800/1064738157 conn(0x7fd45403a4a0 legacy=0x7fd45403c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:39.669 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:39.100+0000 7fd4703f6640 1 -- 172.21.15.57:0/436765534 >> v1:172.21.15.57:6789/0 conn(0x7fd468105860 legacy=0x7fd46807b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:39.669 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:39.100+0000 7fd4703f6640 1 -- 172.21.15.57:0/436765534 shutdown_connections 2024-06-11T13:23:39.669 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:39.100+0000 7fd4703f6640 1 -- 172.21.15.57:0/436765534 >> 172.21.15.57:0/436765534 conn(0x7fd468100e00 msgr2=0x7fd468103240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:39.669 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:39.100+0000 7fd4703f6640 1 -- 172.21.15.57:0/436765534 shutdown_connections 2024-06-11T13:23:39.669 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:39.100+0000 7fd4703f6640 1 -- 172.21.15.57:0/436765534 wait complete. 2024-06-11T13:23:39.669 INFO:teuthology.orchestra.run.smithi057.stdout:Enabling the dashboard module... 2024-06-11T13:23:40.255 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:40 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/436765534' entity='client.admin' 2024-06-11T13:23:41.433 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:41 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/993403540' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-06-11T13:23:41.434 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:41 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ignoring --setuser ceph since I am not root 2024-06-11T13:23:41.434 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:41 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ignoring --setgroup ceph since I am not root 2024-06-11T13:23:41.434 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:41 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:41.280+0000 7f342c3c7040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-11T13:23:41.434 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:41 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:41.439+0000 7f342c3c7040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-11T13:23:41.772 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:41 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:41.515+0000 7f342c3c7040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-11T13:23:41.772 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:41 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:41.778+0000 7f342c3c7040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-11T13:23:41.914 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:40.079+0000 7f9da04cc640 1 Processor -- start 2024-06-11T13:23:41.914 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:40.079+0000 7f9da04cc640 1 -- start start 2024-06-11T13:23:41.914 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:40.079+0000 7f9da04cc640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f9d98109c90 con 0x7f9d98105860 2024-06-11T13:23:41.914 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:40.080+0000 7f9d9e268640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f9d98105860 0x7f9d98105c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:41694/0 (socket says 172.21.15.57:41694) 2024-06-11T13:23:41.914 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:40.080+0000 7f9d9e268640 1 -- 172.21.15.57:0/2189295164 learned_addr learned my addr 172.21.15.57:0/2189295164 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:41.914 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:40.080+0000 7f9d9d266640 1 -- 172.21.15.57:0/2189295164 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3043607534 0 0) 0x7f9d98109c90 con 0x7f9d98105860 2024-06-11T13:23:41.914 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:40.080+0000 7f9d9d266640 1 -- 172.21.15.57:0/2189295164 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d80003520 con 0x7f9d98105860 2024-06-11T13:23:41.914 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:40.080+0000 7f9d9d266640 1 -- 172.21.15.57:0/2189295164 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 189008547 0 0) 0x7f9d80003520 con 0x7f9d98105860 2024-06-11T13:23:41.914 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:40.080+0000 7f9d9d266640 1 -- 172.21.15.57:0/2189295164 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d9810ae70 con 0x7f9d98105860 2024-06-11T13:23:41.915 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:40.080+0000 7f9d9d266640 1 -- 172.21.15.57:0/2189295164 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f9d88001580 con 0x7f9d98105860 2024-06-11T13:23:41.915 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:40.080+0000 7f9d9d266640 1 -- 172.21.15.57:0/2189295164 <== mon.0 v1:172.21.15.57:6789/0 4 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f9d880048a0 con 0x7f9d98105860 2024-06-11T13:23:41.915 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:40.080+0000 7f9d9d266640 1 -- 172.21.15.57:0/2189295164 <== mon.0 v1:172.21.15.57:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f9d880058d0 con 0x7f9d98105860 2024-06-11T13:23:41.915 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:40.081+0000 7f9da04cc640 1 -- 172.21.15.57:0/2189295164 >> v1:172.21.15.57:6789/0 conn(0x7f9d98105860 legacy=0x7f9d98105c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:41.915 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:40.081+0000 7f9da04cc640 1 -- 172.21.15.57:0/2189295164 shutdown_connections 2024-06-11T13:23:41.915 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:40.081+0000 7f9da04cc640 1 -- 172.21.15.57:0/2189295164 >> 172.21.15.57:0/2189295164 conn(0x7f9d98100e00 msgr2=0x7f9d98103240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:41.915 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:40.081+0000 7f9da04cc640 1 -- 172.21.15.57:0/2189295164 shutdown_connections 2024-06-11T13:23:41.915 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:40.081+0000 7f9da04cc640 1 -- 172.21.15.57:0/2189295164 wait complete. 2024-06-11T13:23:41.915 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:40.082+0000 7f9da04cc640 1 Processor -- start 2024-06-11T13:23:41.915 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:40.082+0000 7f9da04cc640 1 -- start start 2024-06-11T13:23:41.915 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:40.082+0000 7f9da04cc640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f9d98199b30 con 0x7f9d98105860 2024-06-11T13:23:41.915 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:40.082+0000 7f9d9e268640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f9d98105860 0x7f9d98199420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:41706/0 (socket says 172.21.15.57:41706) 2024-06-11T13:23:41.916 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:40.082+0000 7f9d9e268640 1 -- 172.21.15.57:0/993403540 learned_addr learned my addr 172.21.15.57:0/993403540 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:41.916 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:40.082+0000 7f9d977fe640 1 -- 172.21.15.57:0/993403540 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2533192555 0 0) 0x7f9d98199b30 con 0x7f9d98105860 2024-06-11T13:23:41.916 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:40.082+0000 7f9d977fe640 1 -- 172.21.15.57:0/993403540 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d80005410 con 0x7f9d98105860 2024-06-11T13:23:41.916 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:40.082+0000 7f9d977fe640 1 -- 172.21.15.57:0/993403540 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 976585821 0 0) 0x7f9d80005410 con 0x7f9d98105860 2024-06-11T13:23:41.916 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:40.082+0000 7f9d977fe640 1 -- 172.21.15.57:0/993403540 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9d98199b30 con 0x7f9d98105860 2024-06-11T13:23:41.916 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:40.082+0000 7f9d977fe640 1 -- 172.21.15.57:0/993403540 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f9d88001080 con 0x7f9d98105860 2024-06-11T13:23:41.916 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:40.083+0000 7f9d977fe640 1 -- 172.21.15.57:0/993403540 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 71753940 0 0) 0x7f9d98199b30 con 0x7f9d98105860 2024-06-11T13:23:41.916 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:40.083+0000 7f9d977fe640 1 -- 172.21.15.57:0/993403540 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d98199d00 con 0x7f9d98105860 2024-06-11T13:23:41.916 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:40.083+0000 7f9da04cc640 1 -- 172.21.15.57:0/993403540 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f9d9819a010 con 0x7f9d98105860 2024-06-11T13:23:41.916 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:40.083+0000 7f9da04cc640 1 -- 172.21.15.57:0/993403540 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f9d9819dba0 con 0x7f9d98105860 2024-06-11T13:23:41.916 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:40.083+0000 7f9d977fe640 1 -- 172.21.15.57:0/993403540 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f9d880048a0 con 0x7f9d98105860 2024-06-11T13:23:41.916 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:40.083+0000 7f9d977fe640 1 -- 172.21.15.57:0/993403540 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f9d88005f00 con 0x7f9d98105860 2024-06-11T13:23:41.916 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:40.084+0000 7f9d977fe640 1 -- 172.21.15.57:0/993403540 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 8) ==== 46252+0+0 (unknown 1643827861 0 0) 0x7f9d88011560 con 0x7f9d98105860 2024-06-11T13:23:41.916 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:40.084+0000 7f9da04cc640 1 -- 172.21.15.57:0/993403540 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d840050b0 con 0x7f9d98105860 2024-06-11T13:23:41.917 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:40.084+0000 7f9d977fe640 1 -- 172.21.15.57:0/993403540 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (unknown 4180383460 0 0) 0x7f9d88047860 con 0x7f9d98105860 2024-06-11T13:23:41.917 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:40.088+0000 7f9d977fe640 1 -- 172.21.15.57:0/993403540 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f9d88017470 con 0x7f9d98105860 2024-06-11T13:23:41.917 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:40.335+0000 7f9da04cc640 1 -- 172.21.15.57:0/993403540 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f9d840053a0 con 0x7f9d98105860 2024-06-11T13:23:41.917 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:41.118+0000 7f9d977fe640 1 -- 172.21.15.57:0/993403540 <== mon.0 v1:172.21.15.57:6789/0 10 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (unknown 1498667528 0 0) 0x7f9d88016d70 con 0x7f9d98105860 2024-06-11T13:23:41.917 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:41.120+0000 7f9d977fe640 1 -- 172.21.15.57:0/993403540 <== mon.0 v1:172.21.15.57:6789/0 11 ==== mgrmap(e 9) ==== 46265+0+0 (unknown 2946861924 0 0) 0x7f9d88004b80 con 0x7f9d98105860 2024-06-11T13:23:41.917 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:41.121+0000 7f9da04cc640 1 -- 172.21.15.57:0/993403540 >> v1:172.21.15.57:6800/1064738157 conn(0x7f9d8003a4f0 legacy=0x7f9d8003c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:41.917 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:41.121+0000 7f9da04cc640 1 -- 172.21.15.57:0/993403540 >> v1:172.21.15.57:6789/0 conn(0x7f9d98105860 legacy=0x7f9d98199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:41.917 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:41.121+0000 7f9da04cc640 1 -- 172.21.15.57:0/993403540 shutdown_connections 2024-06-11T13:23:41.917 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:41.121+0000 7f9da04cc640 1 -- 172.21.15.57:0/993403540 >> 172.21.15.57:0/993403540 conn(0x7f9d98100e00 msgr2=0x7f9d981901b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:41.917 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:41.121+0000 7f9da04cc640 1 -- 172.21.15.57:0/993403540 shutdown_connections 2024-06-11T13:23:41.917 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:41.122+0000 7f9da04cc640 1 -- 172.21.15.57:0/993403540 wait complete. 2024-06-11T13:23:42.031 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:41 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:41.844+0000 7f342c3c7040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-11T13:23:42.300 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:42 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/993403540' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-06-11T13:23:42.300 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:42 smithi057 ceph-mon[29805]: mgrmap e9: y(active, since 20s) 2024-06-11T13:23:43.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:43 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2966728616' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-06-11T13:23:43.173 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:42 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:42.879+0000 7f342c3c7040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-11T13:23:43.173 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:43 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:43.005+0000 7f342c3c7040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-11T13:23:43.173 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:43 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:43.067+0000 7f342c3c7040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-11T13:23:43.181 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout { 2024-06-11T13:23:43.182 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-06-11T13:23:43.182 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-11T13:23:43.182 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-06-11T13:23:43.182 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-06-11T13:23:43.182 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout } 2024-06-11T13:23:43.182 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:42.305+0000 7f93af3f0640 1 Processor -- start 2024-06-11T13:23:43.182 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:42.306+0000 7f93af3f0640 1 -- start start 2024-06-11T13:23:43.182 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:42.306+0000 7f93af3f0640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f93a807a350 con 0x7f93a807b750 2024-06-11T13:23:43.182 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:42.306+0000 7f93ad18c640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f93a807b750 0x7f93a8079bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:41728/0 (socket says 172.21.15.57:41728) 2024-06-11T13:23:43.183 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:42.306+0000 7f93ad18c640 1 -- 172.21.15.57:0/4142366937 learned_addr learned my addr 172.21.15.57:0/4142366937 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:43.183 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:42.307+0000 7f939ffff640 1 -- 172.21.15.57:0/4142366937 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2817806398 0 0) 0x7f93a807a350 con 0x7f93a807b750 2024-06-11T13:23:43.183 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:42.307+0000 7f939ffff640 1 -- 172.21.15.57:0/4142366937 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9394003520 con 0x7f93a807b750 2024-06-11T13:23:43.183 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:42.307+0000 7f939ffff640 1 -- 172.21.15.57:0/4142366937 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2113063095 0 0) 0x7f9394003520 con 0x7f93a807b750 2024-06-11T13:23:43.183 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:42.307+0000 7f939ffff640 1 -- 172.21.15.57:0/4142366937 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93a807a520 con 0x7f93a807b750 2024-06-11T13:23:43.183 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:42.307+0000 7f939ffff640 1 -- 172.21.15.57:0/4142366937 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f93a4001580 con 0x7f93a807b750 2024-06-11T13:23:43.183 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:42.307+0000 7f939ffff640 1 -- 172.21.15.57:0/4142366937 <== mon.0 v1:172.21.15.57:6789/0 4 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f93a4001c70 con 0x7f93a807b750 2024-06-11T13:23:43.183 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:42.307+0000 7f939ffff640 1 -- 172.21.15.57:0/4142366937 <== mon.0 v1:172.21.15.57:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f93a40058d0 con 0x7f93a807b750 2024-06-11T13:23:43.183 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:42.308+0000 7f93af3f0640 1 -- 172.21.15.57:0/4142366937 >> v1:172.21.15.57:6789/0 conn(0x7f93a807b750 legacy=0x7f93a8079bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:43.183 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:42.308+0000 7f93af3f0640 1 -- 172.21.15.57:0/4142366937 shutdown_connections 2024-06-11T13:23:43.183 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:42.308+0000 7f93af3f0640 1 -- 172.21.15.57:0/4142366937 >> 172.21.15.57:0/4142366937 conn(0x7f93a8100dc0 msgr2=0x7f93a8103240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:43.183 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:42.308+0000 7f93af3f0640 1 -- 172.21.15.57:0/4142366937 shutdown_connections 2024-06-11T13:23:43.184 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:42.308+0000 7f93af3f0640 1 -- 172.21.15.57:0/4142366937 wait complete. 2024-06-11T13:23:43.184 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:42.309+0000 7f93af3f0640 1 Processor -- start 2024-06-11T13:23:43.184 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:42.309+0000 7f93af3f0640 1 -- start start 2024-06-11T13:23:43.184 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:42.309+0000 7f93af3f0640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f93a819be00 con 0x7f93a807b750 2024-06-11T13:23:43.184 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:42.309+0000 7f93ad18c640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f93a807b750 0x7f93a819b6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:41742/0 (socket says 172.21.15.57:41742) 2024-06-11T13:23:43.184 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:42.309+0000 7f93ad18c640 1 -- 172.21.15.57:0/2966728616 learned_addr learned my addr 172.21.15.57:0/2966728616 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:43.184 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:42.309+0000 7f939e7fc640 1 -- 172.21.15.57:0/2966728616 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3993439183 0 0) 0x7f93a819be00 con 0x7f93a807b750 2024-06-11T13:23:43.184 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:42.309+0000 7f939e7fc640 1 -- 172.21.15.57:0/2966728616 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9394005460 con 0x7f93a807b750 2024-06-11T13:23:43.184 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:42.310+0000 7f939e7fc640 1 -- 172.21.15.57:0/2966728616 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1256031498 0 0) 0x7f9394005460 con 0x7f93a807b750 2024-06-11T13:23:43.184 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:42.310+0000 7f939e7fc640 1 -- 172.21.15.57:0/2966728616 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f93a819be00 con 0x7f93a807b750 2024-06-11T13:23:43.184 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:42.310+0000 7f939e7fc640 1 -- 172.21.15.57:0/2966728616 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f93a4001080 con 0x7f93a807b750 2024-06-11T13:23:43.184 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:42.310+0000 7f939e7fc640 1 -- 172.21.15.57:0/2966728616 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 20847822 0 0) 0x7f93a819be00 con 0x7f93a807b750 2024-06-11T13:23:43.184 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:42.310+0000 7f939e7fc640 1 -- 172.21.15.57:0/2966728616 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93a819bfd0 con 0x7f93a807b750 2024-06-11T13:23:43.184 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:42.310+0000 7f93af3f0640 1 -- 172.21.15.57:0/2966728616 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f93a819c2e0 con 0x7f93a807b750 2024-06-11T13:23:43.185 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:42.310+0000 7f93af3f0640 1 -- 172.21.15.57:0/2966728616 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f93a819fe70 con 0x7f93a807b750 2024-06-11T13:23:43.185 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:42.310+0000 7f939e7fc640 1 -- 172.21.15.57:0/2966728616 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f93a40049d0 con 0x7f93a807b750 2024-06-11T13:23:43.185 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:42.310+0000 7f939e7fc640 1 -- 172.21.15.57:0/2966728616 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f93a40061e0 con 0x7f93a807b750 2024-06-11T13:23:43.185 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:42.311+0000 7f939e7fc640 1 -- 172.21.15.57:0/2966728616 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 9) ==== 46265+0+0 (unknown 2946861924 0 0) 0x7f93a4011870 con 0x7f93a807b750 2024-06-11T13:23:43.185 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:42.311+0000 7f93af3f0640 1 -- 172.21.15.57:0/2966728616 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f938c0050b0 con 0x7f93a807b750 2024-06-11T13:23:43.185 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:42.312+0000 7f939e7fc640 1 -- 172.21.15.57:0/2966728616 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (unknown 4180383460 0 0) 0x7f93a40478a0 con 0x7f93a807b750 2024-06-11T13:23:43.185 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:42.312+0000 7f93ac98b640 1 -- 172.21.15.57:0/2966728616 >> v1:172.21.15.57:6800/1064738157 conn(0x7f939403a530 legacy=0x7f939403c9f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.57:6800/1064738157 2024-06-11T13:23:43.185 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:42.321+0000 7f939e7fc640 1 -- 172.21.15.57:0/2966728616 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f93a4017520 con 0x7f93a807b750 2024-06-11T13:23:43.185 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:42.513+0000 7f93ac98b640 1 -- 172.21.15.57:0/2966728616 >> v1:172.21.15.57:6800/1064738157 conn(0x7f939403a530 legacy=0x7f939403c9f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.57:6800/1064738157 2024-06-11T13:23:43.185 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:42.558+0000 7f93af3f0640 1 -- 172.21.15.57:0/2966728616 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f938c005bb0 con 0x7f93a807b750 2024-06-11T13:23:43.185 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:42.558+0000 7f939e7fc640 1 -- 172.21.15.57:0/2966728616 <== mon.0 v1:172.21.15.57:6789/0 10 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (unknown 2005831594 0 1748205903) 0x7f93a4016e20 con 0x7f93a807b750 2024-06-11T13:23:43.185 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:42.560+0000 7f93af3f0640 1 -- 172.21.15.57:0/2966728616 >> v1:172.21.15.57:6800/1064738157 conn(0x7f939403a530 legacy=0x7f939403c9f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-11T13:23:43.186 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:42.560+0000 7f93af3f0640 1 -- 172.21.15.57:0/2966728616 >> v1:172.21.15.57:6789/0 conn(0x7f93a807b750 legacy=0x7f93a819b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:43.186 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:42.560+0000 7f93af3f0640 1 -- 172.21.15.57:0/2966728616 shutdown_connections 2024-06-11T13:23:43.186 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:42.560+0000 7f93af3f0640 1 -- 172.21.15.57:0/2966728616 >> 172.21.15.57:0/2966728616 conn(0x7f93a8100dc0 msgr2=0x7f93a8109db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:43.186 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:42.560+0000 7f93af3f0640 1 -- 172.21.15.57:0/2966728616 shutdown_connections 2024-06-11T13:23:43.186 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:42.560+0000 7f93af3f0640 1 -- 172.21.15.57:0/2966728616 wait complete. 2024-06-11T13:23:43.186 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for the mgr to restart... 2024-06-11T13:23:43.186 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for mgr epoch 9... 2024-06-11T13:23:43.441 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:43 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:43.303+0000 7f342c3c7040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-11T13:23:43.442 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:43 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:43.370+0000 7f342c3c7040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-11T13:23:43.442 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:43 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:43.446+0000 7f342c3c7040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-11T13:23:43.781 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:43 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:43.612+0000 7f342c3c7040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-11T13:23:44.425 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:44 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:44.081+0000 7f342c3c7040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-11T13:23:44.425 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:44 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:44.429+0000 7f342c3c7040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-11T13:23:44.686 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:44 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:44.502+0000 7f342c3c7040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-11T13:23:44.686 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:44 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:44.666+0000 7f342c3c7040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-11T13:23:45.271 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:45 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:45.007+0000 7f342c3c7040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-11T13:23:45.271 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:45 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:45.074+0000 7f342c3c7040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-11T13:23:45.531 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:45 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:45.276+0000 7f342c3c7040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-11T13:23:45.531 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:45 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:45.436+0000 7f342c3c7040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-11T13:23:45.531 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:45 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:45.506+0000 7f342c3c7040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-11T13:23:46.031 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:45 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:45.597+0000 7f342c3c7040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-11T13:23:46.031 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:45 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:45.661+0000 7f342c3c7040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-11T13:23:46.531 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:46.247+0000 7f342c3c7040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-11T13:23:46.531 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:46.420+0000 7f342c3c7040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-11T13:23:46.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:46 smithi057 ceph-mon[29805]: Active manager daemon y restarted 2024-06-11T13:23:46.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:46 smithi057 ceph-mon[29805]: Activating manager daemon y 2024-06-11T13:23:46.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:46 smithi057 ceph-mon[29805]: osdmap e3: 0 total, 0 up, 0 in 2024-06-11T13:23:46.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:46 smithi057 ceph-mon[29805]: mgrmap e10: y(active, starting, since 0.00841777s) 2024-06-11T13:23:46.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:46 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-11T13:23:46.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:46 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-11T13:23:46.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:46 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-11T13:23:46.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:46 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-11T13:23:46.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:46 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-11T13:23:46.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:46 smithi057 ceph-mon[29805]: Manager daemon y is now available 2024-06-11T13:23:46.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:46 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:23:46.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:46 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-11T13:23:46.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:46 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-11T13:23:48.309 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout { 2024-06-11T13:23:48.309 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-06-11T13:23:48.309 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "initialized": true 2024-06-11T13:23:48.309 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout } 2024-06-11T13:23:48.309 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:43.596+0000 7fc2b61bb640 1 Processor -- start 2024-06-11T13:23:48.309 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:43.596+0000 7fc2b61bb640 1 -- start start 2024-06-11T13:23:48.310 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:43.596+0000 7fc2b61bb640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fc2b0078e90 con 0x7fc2b0106310 2024-06-11T13:23:48.310 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:43.596+0000 7fc2af7fe640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fc2b0106310 0x7fc2b0108700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:41746/0 (socket says 172.21.15.57:41746) 2024-06-11T13:23:48.310 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:43.596+0000 7fc2af7fe640 1 -- 172.21.15.57:0/3699668848 learned_addr learned my addr 172.21.15.57:0/3699668848 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:48.310 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:43.597+0000 7fc2ae7fc640 1 -- 172.21.15.57:0/3699668848 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4177095840 0 0) 0x7fc2b0078e90 con 0x7fc2b0106310 2024-06-11T13:23:48.310 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:43.597+0000 7fc2ae7fc640 1 -- 172.21.15.57:0/3699668848 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc298003520 con 0x7fc2b0106310 2024-06-11T13:23:48.310 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:43.597+0000 7fc2ae7fc640 1 -- 172.21.15.57:0/3699668848 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1196871283 0 0) 0x7fc298003520 con 0x7fc2b0106310 2024-06-11T13:23:48.310 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:43.597+0000 7fc2ae7fc640 1 -- 172.21.15.57:0/3699668848 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2b010ae30 con 0x7fc2b0106310 2024-06-11T13:23:48.310 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:43.597+0000 7fc2ae7fc640 1 -- 172.21.15.57:0/3699668848 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7fc2a0001580 con 0x7fc2b0106310 2024-06-11T13:23:48.310 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:43.597+0000 7fc2ae7fc640 1 -- 172.21.15.57:0/3699668848 <== mon.0 v1:172.21.15.57:6789/0 4 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7fc2a0001c70 con 0x7fc2b0106310 2024-06-11T13:23:48.310 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:43.597+0000 7fc2ae7fc640 1 -- 172.21.15.57:0/3699668848 <== mon.0 v1:172.21.15.57:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7fc2a00058d0 con 0x7fc2b0106310 2024-06-11T13:23:48.310 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:43.598+0000 7fc2b61bb640 1 -- 172.21.15.57:0/3699668848 >> v1:172.21.15.57:6789/0 conn(0x7fc2b0106310 legacy=0x7fc2b0108700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:48.310 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:43.598+0000 7fc2b61bb640 1 -- 172.21.15.57:0/3699668848 shutdown_connections 2024-06-11T13:23:48.310 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:43.598+0000 7fc2b61bb640 1 -- 172.21.15.57:0/3699668848 >> 172.21.15.57:0/3699668848 conn(0x7fc2b00ffcc0 msgr2=0x7fc2b0102140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:48.311 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:43.598+0000 7fc2b61bb640 1 -- 172.21.15.57:0/3699668848 shutdown_connections 2024-06-11T13:23:48.311 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:43.598+0000 7fc2b61bb640 1 -- 172.21.15.57:0/3699668848 wait complete. 2024-06-11T13:23:48.311 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:43.599+0000 7fc2b61bb640 1 Processor -- start 2024-06-11T13:23:48.311 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:43.599+0000 7fc2b61bb640 1 -- start start 2024-06-11T13:23:48.311 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:43.599+0000 7fc2b61bb640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fc2b019f4f0 con 0x7fc2b0106310 2024-06-11T13:23:48.311 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:43.600+0000 7fc2af7fe640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fc2b0106310 0x7fc2b0105b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:41754/0 (socket says 172.21.15.57:41754) 2024-06-11T13:23:48.311 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:43.600+0000 7fc2af7fe640 1 -- 172.21.15.57:0/4253651379 learned_addr learned my addr 172.21.15.57:0/4253651379 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:48.311 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:43.600+0000 7fc2acff9640 1 -- 172.21.15.57:0/4253651379 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2636589196 0 0) 0x7fc2b019f4f0 con 0x7fc2b0106310 2024-06-11T13:23:48.311 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:43.600+0000 7fc2acff9640 1 -- 172.21.15.57:0/4253651379 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc298005460 con 0x7fc2b0106310 2024-06-11T13:23:48.311 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:43.600+0000 7fc2acff9640 1 -- 172.21.15.57:0/4253651379 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4218269443 0 0) 0x7fc298005460 con 0x7fc2b0106310 2024-06-11T13:23:48.311 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:43.600+0000 7fc2acff9640 1 -- 172.21.15.57:0/4253651379 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc2b019f4f0 con 0x7fc2b0106310 2024-06-11T13:23:48.311 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:43.600+0000 7fc2acff9640 1 -- 172.21.15.57:0/4253651379 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7fc2a0001080 con 0x7fc2b0106310 2024-06-11T13:23:48.311 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:43.600+0000 7fc2acff9640 1 -- 172.21.15.57:0/4253651379 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1625024051 0 0) 0x7fc2b019f4f0 con 0x7fc2b0106310 2024-06-11T13:23:48.312 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:43.600+0000 7fc2acff9640 1 -- 172.21.15.57:0/4253651379 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2b01a06d0 con 0x7fc2b0106310 2024-06-11T13:23:48.312 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:43.600+0000 7fc2b61bb640 1 -- 172.21.15.57:0/4253651379 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fc2b019f6c0 con 0x7fc2b0106310 2024-06-11T13:23:48.312 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:43.600+0000 7fc2b61bb640 1 -- 172.21.15.57:0/4253651379 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fc2b019fc00 con 0x7fc2b0106310 2024-06-11T13:23:48.312 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:43.601+0000 7fc2acff9640 1 -- 172.21.15.57:0/4253651379 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7fc2a0001c30 con 0x7fc2b0106310 2024-06-11T13:23:48.312 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:43.601+0000 7fc2acff9640 1 -- 172.21.15.57:0/4253651379 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7fc2a0005f70 con 0x7fc2b0106310 2024-06-11T13:23:48.312 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:43.602+0000 7fc2acff9640 1 -- 172.21.15.57:0/4253651379 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 9) ==== 46265+0+0 (unknown 2946861924 0 0) 0x7fc2a00115e0 con 0x7fc2b0106310 2024-06-11T13:23:48.312 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:43.602+0000 7fc2acff9640 1 -- 172.21.15.57:0/4253651379 --> v1:172.21.15.57:6800/1064738157 -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc2a0001080 con 0x7fc29803a530 2024-06-11T13:23:48.312 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:43.602+0000 7fc2aeffd640 1 -- 172.21.15.57:0/4253651379 >> v1:172.21.15.57:6800/1064738157 conn(0x7fc29803a530 legacy=0x7fc29803c9f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.57:6800/1064738157 2024-06-11T13:23:48.312 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:43.602+0000 7fc2acff9640 1 -- 172.21.15.57:0/4253651379 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (unknown 4180383460 0 0) 0x7fc2a00478a0 con 0x7fc2b0106310 2024-06-11T13:23:48.312 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:43.803+0000 7fc2aeffd640 1 -- 172.21.15.57:0/4253651379 >> v1:172.21.15.57:6800/1064738157 conn(0x7fc29803a530 legacy=0x7fc29803c9f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.57:6800/1064738157 2024-06-11T13:23:48.312 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:44.204+0000 7fc2aeffd640 1 -- 172.21.15.57:0/4253651379 >> v1:172.21.15.57:6800/1064738157 conn(0x7fc29803a530 legacy=0x7fc29803c9f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.57:6800/1064738157 2024-06-11T13:23:48.312 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:45.005+0000 7fc2aeffd640 1 -- 172.21.15.57:0/4253651379 >> v1:172.21.15.57:6800/1064738157 conn(0x7fc29803a530 legacy=0x7fc29803c9f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.57:6800/1064738157 2024-06-11T13:23:48.312 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:46.429+0000 7fc2acff9640 1 -- 172.21.15.57:0/4253651379 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mgrmap(e 10) ==== 46067+0+0 (unknown 3982314839 0 0) 0x7fc2a00461e0 con 0x7fc2b0106310 2024-06-11T13:23:48.313 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:46.429+0000 7fc2acff9640 1 -- 172.21.15.57:0/4253651379 >> v1:172.21.15.57:6800/1064738157 conn(0x7fc29803a530 legacy=0x7fc29803c9f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-11T13:23:48.313 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:47.425+0000 7fc2acff9640 1 -- 172.21.15.57:0/4253651379 <== mon.0 v1:172.21.15.57:6789/0 10 ==== mgrmap(e 11) ==== 46159+0+0 (unknown 2527421345 0 0) 0x7fc2a0013430 con 0x7fc2b0106310 2024-06-11T13:23:48.313 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:47.426+0000 7fc2acff9640 1 -- 172.21.15.57:0/4253651379 --> v1:172.21.15.57:6800/2989665238 -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc2a0001080 con 0x7fc29803ef00 2024-06-11T13:23:48.313 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:47.427+0000 7fc2acff9640 1 -- 172.21.15.57:0/4253651379 <== mgr.14150 v1:172.21.15.57:6800/2989665238 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (unknown 0 0 4109177037) 0x7fc2a0001080 con 0x7fc29803ef00 2024-06-11T13:23:48.313 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:47.433+0000 7fc2b61bb640 1 -- 172.21.15.57:0/4253651379 --> v1:172.21.15.57:6800/2989665238 -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fc29c0025a0 con 0x7fc29803ef00 2024-06-11T13:23:48.313 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:47.433+0000 7fc2acff9640 1 -- 172.21.15.57:0/4253651379 <== mgr.14150 v1:172.21.15.57:6800/2989665238 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (unknown 0 0 3086460295) 0x7fc29c0025a0 con 0x7fc29803ef00 2024-06-11T13:23:48.313 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:47.433+0000 7fc2b61bb640 1 -- 172.21.15.57:0/4253651379 >> v1:172.21.15.57:6800/2989665238 conn(0x7fc29803ef00 legacy=0x7fc2980412f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:48.313 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:47.433+0000 7fc2b61bb640 1 -- 172.21.15.57:0/4253651379 >> v1:172.21.15.57:6789/0 conn(0x7fc2b0106310 legacy=0x7fc2b0105b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:48.313 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:47.433+0000 7fc2b61bb640 1 -- 172.21.15.57:0/4253651379 shutdown_connections 2024-06-11T13:23:48.313 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:47.433+0000 7fc2b61bb640 1 -- 172.21.15.57:0/4253651379 >> 172.21.15.57:0/4253651379 conn(0x7fc2b00ffcc0 msgr2=0x7fc2b0190170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:48.313 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:47.433+0000 7fc2b61bb640 1 -- 172.21.15.57:0/4253651379 shutdown_connections 2024-06-11T13:23:48.313 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:47.433+0000 7fc2b61bb640 1 -- 172.21.15.57:0/4253651379 wait complete. 2024-06-11T13:23:48.313 INFO:teuthology.orchestra.run.smithi057.stdout:mgr epoch 9 is available 2024-06-11T13:23:48.313 INFO:teuthology.orchestra.run.smithi057.stdout:Generating a dashboard self-signed certificate... 2024-06-11T13:23:48.695 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:48 smithi057 ceph-mon[29805]: from='client.14154 v1:172.21.15.57:0/4253651379' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-06-11T13:23:48.696 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:48 smithi057 ceph-mon[29805]: mgrmap e11: y(active, since 1.00616s) 2024-06-11T13:23:48.696 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:48 smithi057 ceph-mon[29805]: from='client.14154 v1:172.21.15.57:0/4253651379' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-06-11T13:23:48.696 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:48 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:23:49.281 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:48 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:48.968+0000 7f3392990640 -1 log_channel(cephadm) log [ERR] : [11/Jun/2024:13:23:48] ENGINE Error in HTTPServer.serve 2024-06-11T13:23:49.282 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:48 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: Traceback (most recent call last): 2024-06-11T13:23:49.282 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:48 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-06-11T13:23:49.282 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:48 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: self._connections.run(self.expiration_interval) 2024-06-11T13:23:49.282 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:48 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-06-11T13:23:49.282 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:48 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: self._run(expiration_interval) 2024-06-11T13:23:49.282 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:48 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-06-11T13:23:49.282 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:48 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: new_conn = self._from_server_socket(self.server.socket) 2024-06-11T13:23:49.282 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:48 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-06-11T13:23:49.282 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:48 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-06-11T13:23:49.282 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:48 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-06-11T13:23:49.282 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:48 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: s = self.context.wrap_socket( 2024-06-11T13:23:49.282 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:48 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-06-11T13:23:49.282 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:48 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: return self.sslsocket_class._create( 2024-06-11T13:23:49.283 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:48 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-06-11T13:23:49.283 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:48 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: self.do_handshake() 2024-06-11T13:23:49.283 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:48 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-06-11T13:23:49.283 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:48 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: self._sslobj.do_handshake() 2024-06-11T13:23:49.283 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:48 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1129) 2024-06-11T13:23:49.283 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:23:48 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:23:49.650 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-06-11T13:23:49.650 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:48.782+0000 7f2d2f392640 1 Processor -- start 2024-06-11T13:23:49.650 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:48.783+0000 7f2d2f392640 1 -- start start 2024-06-11T13:23:49.650 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:48.783+0000 7f2d2f392640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f2d28108b80 con 0x7f2d28104750 2024-06-11T13:23:49.650 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:48.783+0000 7f2d2d12e640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f2d28104750 0x7f2d28104b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:41824/0 (socket says 172.21.15.57:41824) 2024-06-11T13:23:49.650 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:48.783+0000 7f2d2d12e640 1 -- 172.21.15.57:0/3627549607 learned_addr learned my addr 172.21.15.57:0/3627549607 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:49.650 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:48.784+0000 7f2d1bfff640 1 -- 172.21.15.57:0/3627549607 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1832779235 0 0) 0x7f2d28108b80 con 0x7f2d28104750 2024-06-11T13:23:49.650 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:48.784+0000 7f2d1bfff640 1 -- 172.21.15.57:0/3627549607 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d14005730 con 0x7f2d28104750 2024-06-11T13:23:49.650 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:48.784+0000 7f2d1bfff640 1 -- 172.21.15.57:0/3627549607 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1202114591 0 0) 0x7f2d14005730 con 0x7f2d28104750 2024-06-11T13:23:49.650 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:48.784+0000 7f2d1bfff640 1 -- 172.21.15.57:0/3627549607 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d28109d60 con 0x7f2d28104750 2024-06-11T13:23:49.651 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:48.784+0000 7f2d1bfff640 1 -- 172.21.15.57:0/3627549607 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f2d1c001580 con 0x7f2d28104750 2024-06-11T13:23:49.651 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:48.784+0000 7f2d1bfff640 1 -- 172.21.15.57:0/3627549607 <== mon.0 v1:172.21.15.57:6789/0 4 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f2d1c0048a0 con 0x7f2d28104750 2024-06-11T13:23:49.651 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:48.784+0000 7f2d1bfff640 1 -- 172.21.15.57:0/3627549607 <== mon.0 v1:172.21.15.57:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f2d1c0058d0 con 0x7f2d28104750 2024-06-11T13:23:49.651 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:48.784+0000 7f2d2f392640 1 -- 172.21.15.57:0/3627549607 >> v1:172.21.15.57:6789/0 conn(0x7f2d28104750 legacy=0x7f2d28104b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:49.651 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:48.784+0000 7f2d2f392640 1 -- 172.21.15.57:0/3627549607 shutdown_connections 2024-06-11T13:23:49.651 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:48.784+0000 7f2d2f392640 1 -- 172.21.15.57:0/3627549607 >> 172.21.15.57:0/3627549607 conn(0x7f2d280ffcb0 msgr2=0x7f2d28102130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:49.651 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:48.785+0000 7f2d2f392640 1 -- 172.21.15.57:0/3627549607 shutdown_connections 2024-06-11T13:23:49.651 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:48.785+0000 7f2d2f392640 1 -- 172.21.15.57:0/3627549607 wait complete. 2024-06-11T13:23:49.651 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:48.785+0000 7f2d2f392640 1 Processor -- start 2024-06-11T13:23:49.651 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:48.785+0000 7f2d2f392640 1 -- start start 2024-06-11T13:23:49.651 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:48.785+0000 7f2d2f392640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f2d28197920 con 0x7f2d28104750 2024-06-11T13:23:49.651 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:48.785+0000 7f2d2d12e640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f2d28104750 0x7f2d28197210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:41830/0 (socket says 172.21.15.57:41830) 2024-06-11T13:23:49.651 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:48.785+0000 7f2d2d12e640 1 -- 172.21.15.57:0/3287493789 learned_addr learned my addr 172.21.15.57:0/3287493789 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:49.651 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:48.786+0000 7f2d1a7fc640 1 -- 172.21.15.57:0/3287493789 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2845362319 0 0) 0x7f2d28197920 con 0x7f2d28104750 2024-06-11T13:23:49.652 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:48.786+0000 7f2d1a7fc640 1 -- 172.21.15.57:0/3287493789 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d14007620 con 0x7f2d28104750 2024-06-11T13:23:49.652 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:48.786+0000 7f2d1a7fc640 1 -- 172.21.15.57:0/3287493789 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3451039187 0 0) 0x7f2d14007620 con 0x7f2d28104750 2024-06-11T13:23:49.652 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:48.786+0000 7f2d1a7fc640 1 -- 172.21.15.57:0/3287493789 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2d28197920 con 0x7f2d28104750 2024-06-11T13:23:49.652 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:48.786+0000 7f2d1a7fc640 1 -- 172.21.15.57:0/3287493789 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f2d1c001080 con 0x7f2d28104750 2024-06-11T13:23:49.652 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:48.786+0000 7f2d1a7fc640 1 -- 172.21.15.57:0/3287493789 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1533320351 0 0) 0x7f2d28197920 con 0x7f2d28104750 2024-06-11T13:23:49.652 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:48.786+0000 7f2d1a7fc640 1 -- 172.21.15.57:0/3287493789 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d28197af0 con 0x7f2d28104750 2024-06-11T13:23:49.652 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:48.786+0000 7f2d2f392640 1 -- 172.21.15.57:0/3287493789 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f2d28197e00 con 0x7f2d28104750 2024-06-11T13:23:49.652 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:48.786+0000 7f2d2f392640 1 -- 172.21.15.57:0/3287493789 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f2d2819b990 con 0x7f2d28104750 2024-06-11T13:23:49.652 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:48.786+0000 7f2d1a7fc640 1 -- 172.21.15.57:0/3287493789 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f2d1c005880 con 0x7f2d28104750 2024-06-11T13:23:49.652 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:48.786+0000 7f2d1a7fc640 1 -- 172.21.15.57:0/3287493789 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f2d1c006210 con 0x7f2d28104750 2024-06-11T13:23:49.653 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:48.787+0000 7f2d1a7fc640 1 -- 172.21.15.57:0/3287493789 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 12) ==== 46265+0+0 (unknown 1871827014 0 0) 0x7f2d1c011980 con 0x7f2d28104750 2024-06-11T13:23:49.653 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:48.787+0000 7f2d2f392640 1 -- 172.21.15.57:0/3287493789 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d0c0050b0 con 0x7f2d28104750 2024-06-11T13:23:49.653 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:48.787+0000 7f2d1a7fc640 1 -- 172.21.15.57:0/3287493789 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(3..3 src has 1..3) ==== 1069+0+0 (unknown 1210373348 0 0) 0x7f2d1c047a50 con 0x7f2d28104750 2024-06-11T13:23:49.653 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:48.792+0000 7f2d1a7fc640 1 -- 172.21.15.57:0/3287493789 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f2d1c017620 con 0x7f2d28104750 2024-06-11T13:23:49.653 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:49.001+0000 7f2d2f392640 1 -- 172.21.15.57:0/3287493789 --> v1:172.21.15.57:6800/2989665238 -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f2d0c000bc0 con 0x7f2d1403c6b0 2024-06-11T13:23:49.653 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:49.090+0000 7f2d1a7fc640 1 -- 172.21.15.57:0/3287493789 <== mgr.14150 v1:172.21.15.57:6800/2989665238 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (unknown 0 0 3317752739) 0x7f2d0c000bc0 con 0x7f2d1403c6b0 2024-06-11T13:23:49.653 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:49.093+0000 7f2d2f392640 1 -- 172.21.15.57:0/3287493789 >> v1:172.21.15.57:6800/2989665238 conn(0x7f2d1403c6b0 legacy=0x7f2d1403eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:49.653 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:49.093+0000 7f2d2f392640 1 -- 172.21.15.57:0/3287493789 >> v1:172.21.15.57:6789/0 conn(0x7f2d28104750 legacy=0x7f2d28197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:49.653 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:49.093+0000 7f2d2f392640 1 -- 172.21.15.57:0/3287493789 shutdown_connections 2024-06-11T13:23:49.653 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:49.093+0000 7f2d2f392640 1 -- 172.21.15.57:0/3287493789 >> 172.21.15.57:0/3287493789 conn(0x7f2d280ffcb0 msgr2=0x7f2d28102130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:49.653 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:49.093+0000 7f2d2f392640 1 -- 172.21.15.57:0/3287493789 shutdown_connections 2024-06-11T13:23:49.653 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:49.093+0000 7f2d2f392640 1 -- 172.21.15.57:0/3287493789 wait complete. 2024-06-11T13:23:49.653 INFO:teuthology.orchestra.run.smithi057.stdout:Creating initial admin user... 2024-06-11T13:23:49.753 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:49 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:23:49.753 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:49 smithi057 ceph-mon[29805]: [11/Jun/2024:13:23:48] ENGINE Bus STARTING 2024-06-11T13:23:49.753 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:49 smithi057 ceph-mon[29805]: mgrmap e12: y(active, since 2s) 2024-06-11T13:23:49.753 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:49 smithi057 ceph-mon[29805]: [11/Jun/2024:13:23:48] ENGINE Serving on http://172.21.15.57:8765 2024-06-11T13:23:49.753 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:49 smithi057 ceph-mon[29805]: [11/Jun/2024:13:23:48] ENGINE Serving on https://172.21.15.57:7150 2024-06-11T13:23:49.753 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:49 smithi057 ceph-mon[29805]: [11/Jun/2024:13:23:48] ENGINE Bus STARTED 2024-06-11T13:23:49.753 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:49 smithi057 ceph-mon[29805]: [11/Jun/2024:13:23:48] ENGINE Error in HTTPServer.serve 2024-06-11T13:23:49.753 INFO:journalctl@ceph.mon.a.smithi057.stdout: Traceback (most recent call last): 2024-06-11T13:23:49.753 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-06-11T13:23:49.753 INFO:journalctl@ceph.mon.a.smithi057.stdout: self._connections.run(self.expiration_interval) 2024-06-11T13:23:49.753 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-06-11T13:23:49.754 INFO:journalctl@ceph.mon.a.smithi057.stdout: self._run(expiration_interval) 2024-06-11T13:23:49.754 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-06-11T13:23:49.754 INFO:journalctl@ceph.mon.a.smithi057.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-06-11T13:23:49.754 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-06-11T13:23:49.754 INFO:journalctl@ceph.mon.a.smithi057.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-06-11T13:23:49.754 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-06-11T13:23:49.754 INFO:journalctl@ceph.mon.a.smithi057.stdout: s = self.context.wrap_socket( 2024-06-11T13:23:49.754 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-06-11T13:23:49.754 INFO:journalctl@ceph.mon.a.smithi057.stdout: return self.sslsocket_class._create( 2024-06-11T13:23:49.754 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-06-11T13:23:49.754 INFO:journalctl@ceph.mon.a.smithi057.stdout: self.do_handshake() 2024-06-11T13:23:49.754 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-06-11T13:23:49.754 INFO:journalctl@ceph.mon.a.smithi057.stdout: self._sslobj.do_handshake() 2024-06-11T13:23:49.754 INFO:journalctl@ceph.mon.a.smithi057.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1129) 2024-06-11T13:23:49.755 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:49 smithi057 ceph-mon[29805]: from='client.14162 v1:172.21.15.57:0/3287493789' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:23:49.755 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:49 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:23:49.755 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:49 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:23:51.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:51 smithi057 ceph-mon[29805]: from='client.14164 v1:172.21.15.57:0/3808190233' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:23:52.054 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$5OVo1.pWPNk0jxbu6vK8D.xYyzx/cELPQVgnYnsGmYGXT7Z0CiLhe", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1718112231, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-06-11T13:23:52.054 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:50.959+0000 7f32edab5640 1 Processor -- start 2024-06-11T13:23:52.054 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:50.960+0000 7f32edab5640 1 -- start start 2024-06-11T13:23:52.054 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:50.960+0000 7f32edab5640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f32e8108b80 con 0x7f32e8104750 2024-06-11T13:23:52.054 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:50.960+0000 7f32e77fe640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f32e8104750 0x7f32e8104b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:55000/0 (socket says 172.21.15.57:55000) 2024-06-11T13:23:52.055 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:50.960+0000 7f32e77fe640 1 -- 172.21.15.57:0/1750962700 learned_addr learned my addr 172.21.15.57:0/1750962700 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:52.055 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:50.960+0000 7f32e67fc640 1 -- 172.21.15.57:0/1750962700 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2233734792 0 0) 0x7f32e8108b80 con 0x7f32e8104750 2024-06-11T13:23:52.055 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:50.960+0000 7f32e67fc640 1 -- 172.21.15.57:0/1750962700 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32d0003520 con 0x7f32e8104750 2024-06-11T13:23:52.055 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:50.960+0000 7f32e67fc640 1 -- 172.21.15.57:0/1750962700 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2368287666 0 0) 0x7f32d0003520 con 0x7f32e8104750 2024-06-11T13:23:52.055 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:50.961+0000 7f32e67fc640 1 -- 172.21.15.57:0/1750962700 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e8109d60 con 0x7f32e8104750 2024-06-11T13:23:52.055 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:50.961+0000 7f32e67fc640 1 -- 172.21.15.57:0/1750962700 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f32d8001580 con 0x7f32e8104750 2024-06-11T13:23:52.055 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:50.961+0000 7f32e67fc640 1 -- 172.21.15.57:0/1750962700 <== mon.0 v1:172.21.15.57:6789/0 4 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f32d80048a0 con 0x7f32e8104750 2024-06-11T13:23:52.055 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:50.961+0000 7f32e67fc640 1 -- 172.21.15.57:0/1750962700 <== mon.0 v1:172.21.15.57:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f32d80058d0 con 0x7f32e8104750 2024-06-11T13:23:52.055 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:50.961+0000 7f32edab5640 1 -- 172.21.15.57:0/1750962700 >> v1:172.21.15.57:6789/0 conn(0x7f32e8104750 legacy=0x7f32e8104b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:52.055 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:50.961+0000 7f32edab5640 1 -- 172.21.15.57:0/1750962700 shutdown_connections 2024-06-11T13:23:52.055 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:50.961+0000 7f32edab5640 1 -- 172.21.15.57:0/1750962700 >> 172.21.15.57:0/1750962700 conn(0x7f32e80ffcb0 msgr2=0x7f32e8102130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:52.055 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:50.961+0000 7f32edab5640 1 -- 172.21.15.57:0/1750962700 shutdown_connections 2024-06-11T13:23:52.057 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:50.961+0000 7f32edab5640 1 -- 172.21.15.57:0/1750962700 wait complete. 2024-06-11T13:23:52.057 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:50.962+0000 7f32edab5640 1 Processor -- start 2024-06-11T13:23:52.057 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:50.962+0000 7f32edab5640 1 -- start start 2024-06-11T13:23:52.057 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:50.962+0000 7f32edab5640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f32e8199b20 con 0x7f32e8104750 2024-06-11T13:23:52.057 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:50.963+0000 7f32e77fe640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f32e8104750 0x7f32e8199410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:55010/0 (socket says 172.21.15.57:55010) 2024-06-11T13:23:52.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:50.963+0000 7f32e77fe640 1 -- 172.21.15.57:0/3808190233 learned_addr learned my addr 172.21.15.57:0/3808190233 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:52.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:50.963+0000 7f32e4ff9640 1 -- 172.21.15.57:0/3808190233 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1223301179 0 0) 0x7f32e8199b20 con 0x7f32e8104750 2024-06-11T13:23:52.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:50.963+0000 7f32e4ff9640 1 -- 172.21.15.57:0/3808190233 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32d0005410 con 0x7f32e8104750 2024-06-11T13:23:52.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:50.963+0000 7f32e4ff9640 1 -- 172.21.15.57:0/3808190233 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1609448685 0 0) 0x7f32d0005410 con 0x7f32e8104750 2024-06-11T13:23:52.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:50.963+0000 7f32e4ff9640 1 -- 172.21.15.57:0/3808190233 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f32e8199b20 con 0x7f32e8104750 2024-06-11T13:23:52.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:50.963+0000 7f32e4ff9640 1 -- 172.21.15.57:0/3808190233 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f32d8001080 con 0x7f32e8104750 2024-06-11T13:23:52.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:50.963+0000 7f32e4ff9640 1 -- 172.21.15.57:0/3808190233 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2066796371 0 0) 0x7f32e8199b20 con 0x7f32e8104750 2024-06-11T13:23:52.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:50.963+0000 7f32e4ff9640 1 -- 172.21.15.57:0/3808190233 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e8199cf0 con 0x7f32e8104750 2024-06-11T13:23:52.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:50.963+0000 7f32edab5640 1 -- 172.21.15.57:0/3808190233 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f32e819a000 con 0x7f32e8104750 2024-06-11T13:23:52.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:50.963+0000 7f32edab5640 1 -- 172.21.15.57:0/3808190233 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f32e819db90 con 0x7f32e8104750 2024-06-11T13:23:52.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:50.963+0000 7f32e4ff9640 1 -- 172.21.15.57:0/3808190233 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f32d80048a0 con 0x7f32e8104750 2024-06-11T13:23:52.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:50.963+0000 7f32e4ff9640 1 -- 172.21.15.57:0/3808190233 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f32d8005f30 con 0x7f32e8104750 2024-06-11T13:23:52.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:50.964+0000 7f32e4ff9640 1 -- 172.21.15.57:0/3808190233 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 12) ==== 46265+0+0 (unknown 1871827014 0 0) 0x7f32d80115a0 con 0x7f32e8104750 2024-06-11T13:23:52.058 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:50.964+0000 7f32edab5640 1 -- 172.21.15.57:0/3808190233 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32d40050b0 con 0x7f32e8104750 2024-06-11T13:23:52.059 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:50.965+0000 7f32e4ff9640 1 -- 172.21.15.57:0/3808190233 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(3..3 src has 1..3) ==== 1069+0+0 (unknown 1210373348 0 0) 0x7f32d8047880 con 0x7f32e8104750 2024-06-11T13:23:52.059 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:50.969+0000 7f32e4ff9640 1 -- 172.21.15.57:0/3808190233 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f32d8017500 con 0x7f32e8104750 2024-06-11T13:23:52.059 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:51.170+0000 7f32edab5640 1 -- 172.21.15.57:0/3808190233 --> v1:172.21.15.57:6800/2989665238 -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f32d4002420 con 0x7f32d003a4f0 2024-06-11T13:23:52.059 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:51.419+0000 7f32e4ff9640 1 -- 172.21.15.57:0/3808190233 <== mgr.14150 v1:172.21.15.57:6800/2989665238 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (unknown 0 0 337219911) 0x7f32d4002420 con 0x7f32d003a4f0 2024-06-11T13:23:52.059 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:51.422+0000 7f32edab5640 1 -- 172.21.15.57:0/3808190233 >> v1:172.21.15.57:6800/2989665238 conn(0x7f32d003a4f0 legacy=0x7f32d003c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:52.059 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:51.422+0000 7f32edab5640 1 -- 172.21.15.57:0/3808190233 >> v1:172.21.15.57:6789/0 conn(0x7f32e8104750 legacy=0x7f32e8199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:52.059 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:51.422+0000 7f32edab5640 1 -- 172.21.15.57:0/3808190233 shutdown_connections 2024-06-11T13:23:52.059 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:51.422+0000 7f32edab5640 1 -- 172.21.15.57:0/3808190233 >> 172.21.15.57:0/3808190233 conn(0x7f32e80ffcb0 msgr2=0x7f32e8108580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:52.059 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:51.422+0000 7f32edab5640 1 -- 172.21.15.57:0/3808190233 shutdown_connections 2024-06-11T13:23:52.059 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:51.422+0000 7f32edab5640 1 -- 172.21.15.57:0/3808190233 wait complete. 2024-06-11T13:23:52.059 INFO:teuthology.orchestra.run.smithi057.stdout:Fetching dashboard port number... 2024-06-11T13:23:52.658 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:52 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:23:53.237 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 8443 2024-06-11T13:23:53.237 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:52.469+0000 7f2f73c66640 1 Processor -- start 2024-06-11T13:23:53.237 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:52.470+0000 7f2f73c66640 1 -- start start 2024-06-11T13:23:53.237 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:52.470+0000 7f2f73c66640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f2f6c109c90 con 0x7f2f6c105860 2024-06-11T13:23:53.237 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:52.470+0000 7f2f71a02640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f2f6c105860 0x7f2f6c105c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:55022/0 (socket says 172.21.15.57:55022) 2024-06-11T13:23:53.237 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:52.470+0000 7f2f71a02640 1 -- 172.21.15.57:0/3875814965 learned_addr learned my addr 172.21.15.57:0/3875814965 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:53.237 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:52.471+0000 7f2f70a00640 1 -- 172.21.15.57:0/3875814965 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 207675274 0 0) 0x7f2f6c109c90 con 0x7f2f6c105860 2024-06-11T13:23:53.237 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:52.471+0000 7f2f70a00640 1 -- 172.21.15.57:0/3875814965 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f58003520 con 0x7f2f6c105860 2024-06-11T13:23:53.237 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:52.471+0000 7f2f70a00640 1 -- 172.21.15.57:0/3875814965 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2422937791 0 0) 0x7f2f58003520 con 0x7f2f6c105860 2024-06-11T13:23:53.237 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:52.471+0000 7f2f70a00640 1 -- 172.21.15.57:0/3875814965 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f6c10ae70 con 0x7f2f6c105860 2024-06-11T13:23:53.237 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:52.471+0000 7f2f70a00640 1 -- 172.21.15.57:0/3875814965 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f2f68001580 con 0x7f2f6c105860 2024-06-11T13:23:53.238 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:52.471+0000 7f2f70a00640 1 -- 172.21.15.57:0/3875814965 <== mon.0 v1:172.21.15.57:6789/0 4 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f2f680048a0 con 0x7f2f6c105860 2024-06-11T13:23:53.238 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:52.471+0000 7f2f70a00640 1 -- 172.21.15.57:0/3875814965 <== mon.0 v1:172.21.15.57:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f2f680058d0 con 0x7f2f6c105860 2024-06-11T13:23:53.238 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:52.472+0000 7f2f73c66640 1 -- 172.21.15.57:0/3875814965 >> v1:172.21.15.57:6789/0 conn(0x7f2f6c105860 legacy=0x7f2f6c105c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:53.238 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:52.472+0000 7f2f73c66640 1 -- 172.21.15.57:0/3875814965 shutdown_connections 2024-06-11T13:23:53.238 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:52.472+0000 7f2f73c66640 1 -- 172.21.15.57:0/3875814965 >> 172.21.15.57:0/3875814965 conn(0x7f2f6c100e00 msgr2=0x7f2f6c103240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:53.238 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:52.472+0000 7f2f73c66640 1 -- 172.21.15.57:0/3875814965 shutdown_connections 2024-06-11T13:23:53.238 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:52.472+0000 7f2f73c66640 1 -- 172.21.15.57:0/3875814965 wait complete. 2024-06-11T13:23:53.238 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:52.473+0000 7f2f73c66640 1 Processor -- start 2024-06-11T13:23:53.238 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:52.473+0000 7f2f73c66640 1 -- start start 2024-06-11T13:23:53.238 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:52.473+0000 7f2f73c66640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f2f6c199b30 con 0x7f2f6c105860 2024-06-11T13:23:53.238 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:52.473+0000 7f2f71a02640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f2f6c105860 0x7f2f6c199420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:55028/0 (socket says 172.21.15.57:55028) 2024-06-11T13:23:53.238 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:52.473+0000 7f2f71a02640 1 -- 172.21.15.57:0/2642664368 learned_addr learned my addr 172.21.15.57:0/2642664368 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:53.238 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:52.473+0000 7f2f62ffd640 1 -- 172.21.15.57:0/2642664368 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1883616175 0 0) 0x7f2f6c199b30 con 0x7f2f6c105860 2024-06-11T13:23:53.239 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:52.473+0000 7f2f62ffd640 1 -- 172.21.15.57:0/2642664368 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f58005410 con 0x7f2f6c105860 2024-06-11T13:23:53.239 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:52.473+0000 7f2f62ffd640 1 -- 172.21.15.57:0/2642664368 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1033299456 0 0) 0x7f2f58005410 con 0x7f2f6c105860 2024-06-11T13:23:53.239 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:52.473+0000 7f2f62ffd640 1 -- 172.21.15.57:0/2642664368 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2f6c199b30 con 0x7f2f6c105860 2024-06-11T13:23:53.239 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:52.473+0000 7f2f62ffd640 1 -- 172.21.15.57:0/2642664368 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f2f68001080 con 0x7f2f6c105860 2024-06-11T13:23:53.239 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:52.474+0000 7f2f62ffd640 1 -- 172.21.15.57:0/2642664368 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1349471758 0 0) 0x7f2f6c199b30 con 0x7f2f6c105860 2024-06-11T13:23:53.239 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:52.474+0000 7f2f62ffd640 1 -- 172.21.15.57:0/2642664368 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f6c199d00 con 0x7f2f6c105860 2024-06-11T13:23:53.239 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:52.474+0000 7f2f73c66640 1 -- 172.21.15.57:0/2642664368 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f2f6c19a010 con 0x7f2f6c105860 2024-06-11T13:23:53.239 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:52.474+0000 7f2f73c66640 1 -- 172.21.15.57:0/2642664368 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f2f6c19dba0 con 0x7f2f6c105860 2024-06-11T13:23:53.239 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:52.474+0000 7f2f62ffd640 1 -- 172.21.15.57:0/2642664368 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f2f680048a0 con 0x7f2f6c105860 2024-06-11T13:23:53.239 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:52.474+0000 7f2f62ffd640 1 -- 172.21.15.57:0/2642664368 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f2f68005f70 con 0x7f2f6c105860 2024-06-11T13:23:53.239 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:52.475+0000 7f2f62ffd640 1 -- 172.21.15.57:0/2642664368 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 12) ==== 46265+0+0 (unknown 1871827014 0 0) 0x7f2f68011600 con 0x7f2f6c105860 2024-06-11T13:23:53.240 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:52.475+0000 7f2f73c66640 1 -- 172.21.15.57:0/2642664368 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f500050b0 con 0x7f2f6c105860 2024-06-11T13:23:53.240 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:52.475+0000 7f2f62ffd640 1 -- 172.21.15.57:0/2642664368 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(3..3 src has 1..3) ==== 1069+0+0 (unknown 1210373348 0 0) 0x7f2f680479e0 con 0x7f2f6c105860 2024-06-11T13:23:53.240 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:52.480+0000 7f2f62ffd640 1 -- 172.21.15.57:0/2642664368 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f2f68049e20 con 0x7f2f6c105860 2024-06-11T13:23:53.240 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:52.664+0000 7f2f73c66640 1 -- 172.21.15.57:0/2642664368 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f2f500053a0 con 0x7f2f6c105860 2024-06-11T13:23:53.240 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:52.664+0000 7f2f62ffd640 1 -- 172.21.15.57:0/2642664368 <== mon.0 v1:172.21.15.57:6789/0 10 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (unknown 3713421687 0 83753974) 0x7f2f68017580 con 0x7f2f6c105860 2024-06-11T13:23:53.240 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:52.666+0000 7f2f73c66640 1 -- 172.21.15.57:0/2642664368 >> v1:172.21.15.57:6800/2989665238 conn(0x7f2f5803a540 legacy=0x7f2f5803ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:53.240 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:52.666+0000 7f2f73c66640 1 -- 172.21.15.57:0/2642664368 >> v1:172.21.15.57:6789/0 conn(0x7f2f6c105860 legacy=0x7f2f6c199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:53.240 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:52.666+0000 7f2f73c66640 1 -- 172.21.15.57:0/2642664368 shutdown_connections 2024-06-11T13:23:53.240 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:52.666+0000 7f2f73c66640 1 -- 172.21.15.57:0/2642664368 >> 172.21.15.57:0/2642664368 conn(0x7f2f6c100e00 msgr2=0x7f2f6c1901b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:53.240 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:52.667+0000 7f2f73c66640 1 -- 172.21.15.57:0/2642664368 shutdown_connections 2024-06-11T13:23:53.240 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:52.667+0000 7f2f73c66640 1 -- 172.21.15.57:0/2642664368 wait complete. 2024-06-11T13:23:53.247 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-11T13:23:53.247 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout disabled 2024-06-11T13:23:53.256 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-11T13:23:53.256 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout inactive 2024-06-11T13:23:53.256 INFO:teuthology.orchestra.run.smithi057.stdout:firewalld.service is not enabled 2024-06-11T13:23:53.257 INFO:teuthology.orchestra.run.smithi057.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-06-11T13:23:53.259 INFO:teuthology.orchestra.run.smithi057.stdout:Ceph Dashboard is now available at: 2024-06-11T13:23:53.259 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:23:53.259 INFO:teuthology.orchestra.run.smithi057.stdout: URL: https://smithi057.front.sepia.ceph.com:8443/ 2024-06-11T13:23:53.259 INFO:teuthology.orchestra.run.smithi057.stdout: User: admin 2024-06-11T13:23:53.259 INFO:teuthology.orchestra.run.smithi057.stdout: Password: rz62dsavaf 2024-06-11T13:23:53.259 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:23:53.260 INFO:teuthology.orchestra.run.smithi057.stdout:Saving cluster configuration to /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/config directory 2024-06-11T13:23:53.260 INFO:teuthology.orchestra.run.smithi057.stdout:Enabling autotune for osd_memory_target 2024-06-11T13:23:53.484 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:53 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2642664368' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-06-11T13:23:54.519 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:53.690+0000 7f6f097af640 1 Processor -- start 2024-06-11T13:23:54.519 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:53.690+0000 7f6f097af640 1 -- start start 2024-06-11T13:23:54.519 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:53.690+0000 7f6f097af640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f6f0410beb0 con 0x7f6f04107a80 2024-06-11T13:23:54.519 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:53.691+0000 7f6f02ffd640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f6f04107a80 0x7f6f04107ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:55038/0 (socket says 172.21.15.57:55038) 2024-06-11T13:23:54.519 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:53.691+0000 7f6f02ffd640 1 -- 172.21.15.57:0/571305580 learned_addr learned my addr 172.21.15.57:0/571305580 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:54.519 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:53.691+0000 7f6f027fc640 1 -- 172.21.15.57:0/571305580 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 564019947 0 0) 0x7f6f0410beb0 con 0x7f6f04107a80 2024-06-11T13:23:54.519 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:53.691+0000 7f6f027fc640 1 -- 172.21.15.57:0/571305580 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ef0003520 con 0x7f6f04107a80 2024-06-11T13:23:54.519 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:53.691+0000 7f6f027fc640 1 -- 172.21.15.57:0/571305580 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 747477627 0 0) 0x7f6ef0003520 con 0x7f6f04107a80 2024-06-11T13:23:54.519 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:53.691+0000 7f6f027fc640 1 -- 172.21.15.57:0/571305580 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f0410d090 con 0x7f6f04107a80 2024-06-11T13:23:54.520 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:53.691+0000 7f6f027fc640 1 -- 172.21.15.57:0/571305580 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f6ef4001580 con 0x7f6f04107a80 2024-06-11T13:23:54.520 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:53.691+0000 7f6f027fc640 1 -- 172.21.15.57:0/571305580 <== mon.0 v1:172.21.15.57:6789/0 4 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f6ef40048a0 con 0x7f6f04107a80 2024-06-11T13:23:54.520 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:53.691+0000 7f6f027fc640 1 -- 172.21.15.57:0/571305580 <== mon.0 v1:172.21.15.57:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f6ef40058d0 con 0x7f6f04107a80 2024-06-11T13:23:54.520 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:53.692+0000 7f6f097af640 1 -- 172.21.15.57:0/571305580 >> v1:172.21.15.57:6789/0 conn(0x7f6f04107a80 legacy=0x7f6f04107ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:54.520 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:53.692+0000 7f6f097af640 1 -- 172.21.15.57:0/571305580 shutdown_connections 2024-06-11T13:23:54.520 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:53.692+0000 7f6f097af640 1 -- 172.21.15.57:0/571305580 >> 172.21.15.57:0/571305580 conn(0x7f6f0407af20 msgr2=0x7f6f0407b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:54.520 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:53.692+0000 7f6f097af640 1 -- 172.21.15.57:0/571305580 shutdown_connections 2024-06-11T13:23:54.520 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:53.692+0000 7f6f097af640 1 -- 172.21.15.57:0/571305580 wait complete. 2024-06-11T13:23:54.520 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:53.692+0000 7f6f097af640 1 Processor -- start 2024-06-11T13:23:54.520 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:53.693+0000 7f6f097af640 1 -- start start 2024-06-11T13:23:54.520 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:53.693+0000 7f6f097af640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f6f0419bd30 con 0x7f6f04107a80 2024-06-11T13:23:54.520 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:53.693+0000 7f6f02ffd640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f6f04107a80 0x7f6f0419b620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:55040/0 (socket says 172.21.15.57:55040) 2024-06-11T13:23:54.520 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:53.693+0000 7f6f02ffd640 1 -- 172.21.15.57:0/578200998 learned_addr learned my addr 172.21.15.57:0/578200998 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:54.520 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:53.693+0000 7f6f00ff9640 1 -- 172.21.15.57:0/578200998 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2243829717 0 0) 0x7f6f0419bd30 con 0x7f6f04107a80 2024-06-11T13:23:54.520 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:53.693+0000 7f6f00ff9640 1 -- 172.21.15.57:0/578200998 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ef0005410 con 0x7f6f04107a80 2024-06-11T13:23:54.521 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:53.693+0000 7f6f00ff9640 1 -- 172.21.15.57:0/578200998 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2697065917 0 0) 0x7f6ef0005410 con 0x7f6f04107a80 2024-06-11T13:23:54.521 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:53.693+0000 7f6f00ff9640 1 -- 172.21.15.57:0/578200998 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6f0419bd30 con 0x7f6f04107a80 2024-06-11T13:23:54.521 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:53.693+0000 7f6f00ff9640 1 -- 172.21.15.57:0/578200998 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f6ef4001080 con 0x7f6f04107a80 2024-06-11T13:23:54.521 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:53.694+0000 7f6f00ff9640 1 -- 172.21.15.57:0/578200998 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4018755658 0 0) 0x7f6f0419bd30 con 0x7f6f04107a80 2024-06-11T13:23:54.521 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:53.694+0000 7f6f00ff9640 1 -- 172.21.15.57:0/578200998 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f0419bf00 con 0x7f6f04107a80 2024-06-11T13:23:54.521 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:53.694+0000 7f6f097af640 1 -- 172.21.15.57:0/578200998 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f6f0419c210 con 0x7f6f04107a80 2024-06-11T13:23:54.521 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:53.694+0000 7f6f097af640 1 -- 172.21.15.57:0/578200998 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f6f0419fda0 con 0x7f6f04107a80 2024-06-11T13:23:54.521 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:53.694+0000 7f6f00ff9640 1 -- 172.21.15.57:0/578200998 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f6ef40048a0 con 0x7f6f04107a80 2024-06-11T13:23:54.521 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:53.694+0000 7f6f00ff9640 1 -- 172.21.15.57:0/578200998 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f6ef4005f30 con 0x7f6f04107a80 2024-06-11T13:23:54.521 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:53.695+0000 7f6f00ff9640 1 -- 172.21.15.57:0/578200998 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 13) ==== 46308+0+0 (unknown 2805797583 0 0) 0x7f6ef40115c0 con 0x7f6f04107a80 2024-06-11T13:23:54.521 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:53.695+0000 7f6f097af640 1 -- 172.21.15.57:0/578200998 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ee40050b0 con 0x7f6f04107a80 2024-06-11T13:23:54.521 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:53.695+0000 7f6f00ff9640 1 -- 172.21.15.57:0/578200998 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(3..3 src has 1..3) ==== 1069+0+0 (unknown 1210373348 0 0) 0x7f6ef40479d0 con 0x7f6f04107a80 2024-06-11T13:23:54.521 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:53.699+0000 7f6f00ff9640 1 -- 172.21.15.57:0/578200998 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f6ef40175d0 con 0x7f6f04107a80 2024-06-11T13:23:54.522 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:53.887+0000 7f6f097af640 1 -- 172.21.15.57:0/578200998 --> v1:172.21.15.57:6789/0 -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f6ee40053a0 con 0x7f6f04107a80 2024-06-11T13:23:54.522 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:53.887+0000 7f6f00ff9640 1 -- 172.21.15.57:0/578200998 <== mon.0 v1:172.21.15.57:6789/0 10 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (unknown 689008156 0 0) 0x7f6ef4016ed0 con 0x7f6f04107a80 2024-06-11T13:23:54.522 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:53.890+0000 7f6f097af640 1 -- 172.21.15.57:0/578200998 >> v1:172.21.15.57:6800/2989665238 conn(0x7f6ef003a590 legacy=0x7f6ef003ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:54.522 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:53.890+0000 7f6f097af640 1 -- 172.21.15.57:0/578200998 >> v1:172.21.15.57:6789/0 conn(0x7f6f04107a80 legacy=0x7f6f0419b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:54.522 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:53.890+0000 7f6f097af640 1 -- 172.21.15.57:0/578200998 shutdown_connections 2024-06-11T13:23:54.522 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:53.890+0000 7f6f097af640 1 -- 172.21.15.57:0/578200998 >> 172.21.15.57:0/578200998 conn(0x7f6f0407af20 msgr2=0x7f6f04104250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:54.522 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:53.890+0000 7f6f097af640 1 -- 172.21.15.57:0/578200998 shutdown_connections 2024-06-11T13:23:54.522 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:53.890+0000 7f6f097af640 1 -- 172.21.15.57:0/578200998 wait complete. 2024-06-11T13:23:54.740 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:54 smithi057 ceph-mon[29805]: mgrmap e13: y(active, since 7s) 2024-06-11T13:23:55.777 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:54.928+0000 7f2da8dcf640 1 Processor -- start 2024-06-11T13:23:55.777 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:54.929+0000 7f2da8dcf640 1 -- start start 2024-06-11T13:23:55.777 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:54.929+0000 7f2da8dcf640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f2da4108b70 con 0x7f2da4104740 2024-06-11T13:23:55.777 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:54.929+0000 7f2da259c640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f2da4104740 0x7f2da4104b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:55052/0 (socket says 172.21.15.57:55052) 2024-06-11T13:23:55.777 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:54.929+0000 7f2da259c640 1 -- 172.21.15.57:0/520735539 learned_addr learned my addr 172.21.15.57:0/520735539 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:55.777 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:54.930+0000 7f2da159a640 1 -- 172.21.15.57:0/520735539 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 33623452 0 0) 0x7f2da4108b70 con 0x7f2da4104740 2024-06-11T13:23:55.777 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:54.930+0000 7f2da159a640 1 -- 172.21.15.57:0/520735539 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d8c003520 con 0x7f2da4104740 2024-06-11T13:23:55.778 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:54.930+0000 7f2da159a640 1 -- 172.21.15.57:0/520735539 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 632961552 0 0) 0x7f2d8c003520 con 0x7f2da4104740 2024-06-11T13:23:55.778 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:54.930+0000 7f2da159a640 1 -- 172.21.15.57:0/520735539 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2da4109d50 con 0x7f2da4104740 2024-06-11T13:23:55.778 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:54.930+0000 7f2da159a640 1 -- 172.21.15.57:0/520735539 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f2d94001580 con 0x7f2da4104740 2024-06-11T13:23:55.778 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:54.930+0000 7f2da159a640 1 -- 172.21.15.57:0/520735539 <== mon.0 v1:172.21.15.57:6789/0 4 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f2d940048a0 con 0x7f2da4104740 2024-06-11T13:23:55.778 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:54.930+0000 7f2da159a640 1 -- 172.21.15.57:0/520735539 <== mon.0 v1:172.21.15.57:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f2d940058d0 con 0x7f2da4104740 2024-06-11T13:23:55.778 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:54.931+0000 7f2da8dcf640 1 -- 172.21.15.57:0/520735539 >> v1:172.21.15.57:6789/0 conn(0x7f2da4104740 legacy=0x7f2da4104b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:55.778 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:54.931+0000 7f2da8dcf640 1 -- 172.21.15.57:0/520735539 shutdown_connections 2024-06-11T13:23:55.778 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:54.931+0000 7f2da8dcf640 1 -- 172.21.15.57:0/520735539 >> 172.21.15.57:0/520735539 conn(0x7f2da40ffca0 msgr2=0x7f2da4102120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:55.778 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:54.931+0000 7f2da8dcf640 1 -- 172.21.15.57:0/520735539 shutdown_connections 2024-06-11T13:23:55.778 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:54.931+0000 7f2da8dcf640 1 -- 172.21.15.57:0/520735539 wait complete. 2024-06-11T13:23:55.779 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:54.932+0000 7f2da8dcf640 1 Processor -- start 2024-06-11T13:23:55.779 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:54.932+0000 7f2da8dcf640 1 -- start start 2024-06-11T13:23:55.779 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:54.932+0000 7f2da8dcf640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f2da4199bc0 con 0x7f2da4104740 2024-06-11T13:23:55.779 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:54.932+0000 7f2da259c640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f2da4104740 0x7f2da41994b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:55066/0 (socket says 172.21.15.57:55066) 2024-06-11T13:23:55.779 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:54.932+0000 7f2da259c640 1 -- 172.21.15.57:0/1590129540 learned_addr learned my addr 172.21.15.57:0/1590129540 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:55.779 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:54.932+0000 7f2d937fe640 1 -- 172.21.15.57:0/1590129540 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 734030558 0 0) 0x7f2da4199bc0 con 0x7f2da4104740 2024-06-11T13:23:55.779 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:54.932+0000 7f2d937fe640 1 -- 172.21.15.57:0/1590129540 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d8c005410 con 0x7f2da4104740 2024-06-11T13:23:55.779 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:54.932+0000 7f2d937fe640 1 -- 172.21.15.57:0/1590129540 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2248658816 0 0) 0x7f2d8c005410 con 0x7f2da4104740 2024-06-11T13:23:55.779 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:54.932+0000 7f2d937fe640 1 -- 172.21.15.57:0/1590129540 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2da4199bc0 con 0x7f2da4104740 2024-06-11T13:23:55.779 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:54.932+0000 7f2d937fe640 1 -- 172.21.15.57:0/1590129540 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f2d94001080 con 0x7f2da4104740 2024-06-11T13:23:55.779 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:54.933+0000 7f2d937fe640 1 -- 172.21.15.57:0/1590129540 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2981995464 0 0) 0x7f2da4199bc0 con 0x7f2da4104740 2024-06-11T13:23:55.780 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:54.933+0000 7f2d937fe640 1 -- 172.21.15.57:0/1590129540 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2da4199d90 con 0x7f2da4104740 2024-06-11T13:23:55.780 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:54.933+0000 7f2da8dcf640 1 -- 172.21.15.57:0/1590129540 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f2da419a0a0 con 0x7f2da4104740 2024-06-11T13:23:55.780 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:54.933+0000 7f2da8dcf640 1 -- 172.21.15.57:0/1590129540 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f2da419dc30 con 0x7f2da4104740 2024-06-11T13:23:55.780 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:54.933+0000 7f2d937fe640 1 -- 172.21.15.57:0/1590129540 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f2d940048a0 con 0x7f2da4104740 2024-06-11T13:23:55.780 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:54.933+0000 7f2d937fe640 1 -- 172.21.15.57:0/1590129540 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f2d94005f30 con 0x7f2da4104740 2024-06-11T13:23:55.780 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:54.933+0000 7f2d937fe640 1 -- 172.21.15.57:0/1590129540 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 13) ==== 46308+0+0 (unknown 2805797583 0 0) 0x7f2d940115c0 con 0x7f2da4104740 2024-06-11T13:23:55.780 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:54.934+0000 7f2da8dcf640 1 -- 172.21.15.57:0/1590129540 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d84005080 con 0x7f2da4104740 2024-06-11T13:23:55.780 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:54.934+0000 7f2d937fe640 1 -- 172.21.15.57:0/1590129540 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(3..3 src has 1..3) ==== 1069+0+0 (unknown 1210373348 0 0) 0x7f2d94047cb0 con 0x7f2da4104740 2024-06-11T13:23:55.780 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:54.938+0000 7f2d937fe640 1 -- 172.21.15.57:0/1590129540 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f2d940175d0 con 0x7f2da4104740 2024-06-11T13:23:55.780 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:55.219+0000 7f2da8dcf640 1 -- 172.21.15.57:0/1590129540 --> v1:172.21.15.57:6789/0 -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f2d84005370 con 0x7f2da4104740 2024-06-11T13:23:55.780 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:55.222+0000 7f2d937fe640 1 -- 172.21.15.57:0/1590129540 <== mon.0 v1:172.21.15.57:6789/0 10 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (unknown 1169358022 0 0) 0x7f2d94016ed0 con 0x7f2da4104740 2024-06-11T13:23:55.780 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-06-11T13:23:55.780 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:55.224+0000 7f2da8dcf640 1 -- 172.21.15.57:0/1590129540 >> v1:172.21.15.57:6800/2989665238 conn(0x7f2d8c03a590 legacy=0x7f2d8c03ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:55.781 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:55.224+0000 7f2da8dcf640 1 -- 172.21.15.57:0/1590129540 >> v1:172.21.15.57:6789/0 conn(0x7f2da4104740 legacy=0x7f2da41994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:55.781 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:55.224+0000 7f2da8dcf640 1 -- 172.21.15.57:0/1590129540 shutdown_connections 2024-06-11T13:23:55.781 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:55.224+0000 7f2da8dcf640 1 -- 172.21.15.57:0/1590129540 >> 172.21.15.57:0/1590129540 conn(0x7f2da40ffca0 msgr2=0x7f2da40788f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:55.781 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:55.224+0000 7f2da8dcf640 1 -- 172.21.15.57:0/1590129540 shutdown_connections 2024-06-11T13:23:55.781 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-06-11T13:23:55.224+0000 7f2da8dcf640 1 -- 172.21.15.57:0/1590129540 wait complete. 2024-06-11T13:23:55.781 INFO:teuthology.orchestra.run.smithi057.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-06-11T13:23:55.781 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:23:55.781 INFO:teuthology.orchestra.run.smithi057.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-06-11T13:23:55.781 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:23:55.781 INFO:teuthology.orchestra.run.smithi057.stdout:Or, if you are only running a single cluster on this host: 2024-06-11T13:23:55.781 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:23:55.781 INFO:teuthology.orchestra.run.smithi057.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-06-11T13:23:55.782 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:23:55.782 INFO:teuthology.orchestra.run.smithi057.stdout:Please consider enabling telemetry to help improve Ceph: 2024-06-11T13:23:55.782 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:23:55.782 INFO:teuthology.orchestra.run.smithi057.stdout: ceph telemetry on 2024-06-11T13:23:55.782 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:23:55.782 INFO:teuthology.orchestra.run.smithi057.stdout:For more information see: 2024-06-11T13:23:55.782 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:23:55.782 INFO:teuthology.orchestra.run.smithi057.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-06-11T13:23:55.782 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:23:55.782 INFO:teuthology.orchestra.run.smithi057.stdout:Bootstrap complete. 2024-06-11T13:23:55.823 INFO:tasks.cephadm:Fetching config... 2024-06-11T13:23:55.823 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-06-11T13:23:55.823 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-06-11T13:23:55.841 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-06-11T13:23:55.841 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-06-11T13:23:55.841 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-06-11T13:23:55.898 INFO:tasks.cephadm:Fetching mon keyring... 2024-06-11T13:23:55.898 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-06-11T13:23:55.899 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.a/keyring of=/dev/stdout 2024-06-11T13:23:55.965 INFO:tasks.cephadm:Fetching pub ssh key... 2024-06-11T13:23:55.965 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-06-11T13:23:55.965 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-06-11T13:23:56.022 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-06-11T13:23:56.022 DEBUG:teuthology.orchestra.run.smithi057:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCfXBP/feJoo2nBQu0Lrvey6bhe7zfLyuifJivQLPpxIfQW8iKSdDsj7T13YZCD1Lh8rX4ya0gRvvibdkjj3mAKd13VN4VIgEZmLZEhFjosfLSmKjviz8tbBMJr6QtxrYJOycrkSrdTiToLBi53HrrYumt0BkXg/Pr4brn+cyIKr3k3lRkj8S5602Yrh7WN3lThdcpZk0OUJF+VMWvMJeTUPypvkiaTtHV4+AQ7TVEtJpyAiZC3ofIU+rkmhZWkk+NE+oDoroZgPq9VDusu7UmEIylpbCE0NATJP8dgqLs+At5eFFfTwoRuTx3JYPTYtVLBGHpgyiifNKkbSccFHzr15OUQX4cbpsZ+yYbV3ZR5tkbKhHcSusGiU1bkKTRRZUteu8gk+elCeHfFY9sTDGTuBKCGhdHBn+dJLY4AW6uUB8HZTJVSktmuc/raFh0jVgcpxbPbpWTtlwNpIPMp19qJ9XNqJl0sESkKGZeejbGK/OZxyA6HHDV9iLiShLYu728= ceph-98a6635e-27f5-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-11T13:23:56.098 INFO:teuthology.orchestra.run.smithi057.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCfXBP/feJoo2nBQu0Lrvey6bhe7zfLyuifJivQLPpxIfQW8iKSdDsj7T13YZCD1Lh8rX4ya0gRvvibdkjj3mAKd13VN4VIgEZmLZEhFjosfLSmKjviz8tbBMJr6QtxrYJOycrkSrdTiToLBi53HrrYumt0BkXg/Pr4brn+cyIKr3k3lRkj8S5602Yrh7WN3lThdcpZk0OUJF+VMWvMJeTUPypvkiaTtHV4+AQ7TVEtJpyAiZC3ofIU+rkmhZWkk+NE+oDoroZgPq9VDusu7UmEIylpbCE0NATJP8dgqLs+At5eFFfTwoRuTx3JYPTYtVLBGHpgyiifNKkbSccFHzr15OUQX4cbpsZ+yYbV3ZR5tkbKhHcSusGiU1bkKTRRZUteu8gk+elCeHfFY9sTDGTuBKCGhdHBn+dJLY4AW6uUB8HZTJVSktmuc/raFh0jVgcpxbPbpWTtlwNpIPMp19qJ9XNqJl0sESkKGZeejbGK/OZxyA6HHDV9iLiShLYu728= ceph-98a6635e-27f5-11ef-bc9b-c7b262605968 2024-06-11T13:23:56.111 DEBUG:teuthology.orchestra.run.smithi107:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCfXBP/feJoo2nBQu0Lrvey6bhe7zfLyuifJivQLPpxIfQW8iKSdDsj7T13YZCD1Lh8rX4ya0gRvvibdkjj3mAKd13VN4VIgEZmLZEhFjosfLSmKjviz8tbBMJr6QtxrYJOycrkSrdTiToLBi53HrrYumt0BkXg/Pr4brn+cyIKr3k3lRkj8S5602Yrh7WN3lThdcpZk0OUJF+VMWvMJeTUPypvkiaTtHV4+AQ7TVEtJpyAiZC3ofIU+rkmhZWkk+NE+oDoroZgPq9VDusu7UmEIylpbCE0NATJP8dgqLs+At5eFFfTwoRuTx3JYPTYtVLBGHpgyiifNKkbSccFHzr15OUQX4cbpsZ+yYbV3ZR5tkbKhHcSusGiU1bkKTRRZUteu8gk+elCeHfFY9sTDGTuBKCGhdHBn+dJLY4AW6uUB8HZTJVSktmuc/raFh0jVgcpxbPbpWTtlwNpIPMp19qJ9XNqJl0sESkKGZeejbGK/OZxyA6HHDV9iLiShLYu728= ceph-98a6635e-27f5-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-11T13:23:56.152 INFO:teuthology.orchestra.run.smithi107.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCfXBP/feJoo2nBQu0Lrvey6bhe7zfLyuifJivQLPpxIfQW8iKSdDsj7T13YZCD1Lh8rX4ya0gRvvibdkjj3mAKd13VN4VIgEZmLZEhFjosfLSmKjviz8tbBMJr6QtxrYJOycrkSrdTiToLBi53HrrYumt0BkXg/Pr4brn+cyIKr3k3lRkj8S5602Yrh7WN3lThdcpZk0OUJF+VMWvMJeTUPypvkiaTtHV4+AQ7TVEtJpyAiZC3ofIU+rkmhZWkk+NE+oDoroZgPq9VDusu7UmEIylpbCE0NATJP8dgqLs+At5eFFfTwoRuTx3JYPTYtVLBGHpgyiifNKkbSccFHzr15OUQX4cbpsZ+yYbV3ZR5tkbKhHcSusGiU1bkKTRRZUteu8gk+elCeHfFY9sTDGTuBKCGhdHBn+dJLY4AW6uUB8HZTJVSktmuc/raFh0jVgcpxbPbpWTtlwNpIPMp19qJ9XNqJl0sESkKGZeejbGK/OZxyA6HHDV9iLiShLYu728= ceph-98a6635e-27f5-11ef-bc9b-c7b262605968 2024-06-11T13:23:56.165 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-06-11T13:23:56.347 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:56 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1590129540' entity='client.admin' 2024-06-11T13:23:56.412 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.a/config 2024-06-11T13:23:58.933 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:58 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:23:58.933 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:58 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:23:58.933 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:58 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi057", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:23:58.933 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:58 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:23:58.933 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:58 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:23:58.933 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:23:58 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:23:59.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:23:59.027+0000 7f86dce3b640 1 -- 172.21.15.57:0/345400283 >> v1:172.21.15.57:6789/0 conn(0x7f86d81002c0 legacy=0x7f86d81006e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:59.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:23:59.027+0000 7f86dce3b640 1 -- 172.21.15.57:0/345400283 shutdown_connections 2024-06-11T13:23:59.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:23:59.027+0000 7f86dce3b640 1 -- 172.21.15.57:0/345400283 >> 172.21.15.57:0/345400283 conn(0x7f86d80fb880 msgr2=0x7f86d80fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:59.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:23:59.027+0000 7f86dce3b640 1 -- 172.21.15.57:0/345400283 shutdown_connections 2024-06-11T13:23:59.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:23:59.027+0000 7f86dce3b640 1 -- 172.21.15.57:0/345400283 wait complete. 2024-06-11T13:23:59.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:23:59.028+0000 7f86dce3b640 1 Processor -- start 2024-06-11T13:23:59.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:23:59.028+0000 7f86dce3b640 1 -- start start 2024-06-11T13:23:59.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:23:59.028+0000 7f86dce3b640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f86d819be50 con 0x7f86d81002c0 2024-06-11T13:23:59.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:23:59.028+0000 7f86d659c640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f86d81002c0 0x7f86d80fd6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:51200/0 (socket says 172.21.15.57:51200) 2024-06-11T13:23:59.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:23:59.028+0000 7f86d659c640 1 -- 172.21.15.57:0/1417383585 learned_addr learned my addr 172.21.15.57:0/1417383585 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:23:59.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:23:59.028+0000 7f86c77fe640 1 -- 172.21.15.57:0/1417383585 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1937020014 0 0) 0x7f86d819be50 con 0x7f86d81002c0 2024-06-11T13:23:59.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:23:59.028+0000 7f86c77fe640 1 -- 172.21.15.57:0/1417383585 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f86c0005410 con 0x7f86d81002c0 2024-06-11T13:23:59.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:23:59.029+0000 7f86c77fe640 1 -- 172.21.15.57:0/1417383585 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3616668085 0 0) 0x7f86c0005410 con 0x7f86d81002c0 2024-06-11T13:23:59.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:23:59.029+0000 7f86c77fe640 1 -- 172.21.15.57:0/1417383585 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f86d819be50 con 0x7f86d81002c0 2024-06-11T13:23:59.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:23:59.029+0000 7f86c77fe640 1 -- 172.21.15.57:0/1417383585 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f86c8001480 con 0x7f86d81002c0 2024-06-11T13:23:59.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:23:59.029+0000 7f86c77fe640 1 -- 172.21.15.57:0/1417383585 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2995895139 0 0) 0x7f86d819be50 con 0x7f86d81002c0 2024-06-11T13:23:59.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:23:59.029+0000 7f86c77fe640 1 -- 172.21.15.57:0/1417383585 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86d819d030 con 0x7f86d81002c0 2024-06-11T13:23:59.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:23:59.029+0000 7f86dce3b640 1 -- 172.21.15.57:0/1417383585 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f86d819c020 con 0x7f86d81002c0 2024-06-11T13:23:59.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:23:59.029+0000 7f86dce3b640 1 -- 172.21.15.57:0/1417383585 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f86d819c560 con 0x7f86d81002c0 2024-06-11T13:23:59.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:23:59.029+0000 7f86c77fe640 1 -- 172.21.15.57:0/1417383585 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f86c8003650 con 0x7f86d81002c0 2024-06-11T13:23:59.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:23:59.029+0000 7f86c77fe640 1 -- 172.21.15.57:0/1417383585 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f86c8004bd0 con 0x7f86d81002c0 2024-06-11T13:23:59.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:23:59.030+0000 7f86c77fe640 1 -- 172.21.15.57:0/1417383585 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 13) ==== 46308+0+0 (unknown 2805797583 0 0) 0x7f86c8004d90 con 0x7f86d81002c0 2024-06-11T13:23:59.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:23:59.030+0000 7f86dce3b640 1 -- 172.21.15.57:0/1417383585 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86b80050b0 con 0x7f86d81002c0 2024-06-11T13:23:59.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:23:59.030+0000 7f86c77fe640 1 -- 172.21.15.57:0/1417383585 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(3..3 src has 1..3) ==== 1069+0+0 (unknown 1210373348 0 0) 0x7f86c8047930 con 0x7f86d81002c0 2024-06-11T13:23:59.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:23:59.037+0000 7f86c77fe640 1 -- 172.21.15.57:0/1417383585 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f86c80173d0 con 0x7f86d81002c0 2024-06-11T13:23:59.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:23:59.278+0000 7f86dce3b640 1 -- 172.21.15.57:0/1417383585 --> v1:172.21.15.57:6789/0 -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f86b80053a0 con 0x7f86d81002c0 2024-06-11T13:23:59.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:23:59.286+0000 7f86c77fe640 1 -- 172.21.15.57:0/1417383585 <== mon.0 v1:172.21.15.57:6789/0 10 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (unknown 3028693289 0 0) 0x7f86c8016cd0 con 0x7f86d81002c0 2024-06-11T13:23:59.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:23:59.288+0000 7f86dce3b640 1 -- 172.21.15.57:0/1417383585 >> v1:172.21.15.57:6800/2989665238 conn(0x7f86c003a460 legacy=0x7f86c003c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:59.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:23:59.288+0000 7f86dce3b640 1 -- 172.21.15.57:0/1417383585 >> v1:172.21.15.57:6789/0 conn(0x7f86d81002c0 legacy=0x7f86d80fd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:23:59.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:23:59.288+0000 7f86dce3b640 1 -- 172.21.15.57:0/1417383585 shutdown_connections 2024-06-11T13:23:59.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:23:59.288+0000 7f86dce3b640 1 -- 172.21.15.57:0/1417383585 >> 172.21.15.57:0/1417383585 conn(0x7f86d80fb880 msgr2=0x7f86d8102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:23:59.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:23:59.288+0000 7f86dce3b640 1 -- 172.21.15.57:0/1417383585 shutdown_connections 2024-06-11T13:23:59.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:23:59.288+0000 7f86dce3b640 1 -- 172.21.15.57:0/1417383585 wait complete. 2024-06-11T13:23:59.975 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-06-11T13:23:59.975 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-06-11T13:24:00.455 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.a/config 2024-06-11T13:24:00.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:00 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1417383585' entity='client.admin' 2024-06-11T13:24:00.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:00 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:24:00.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:00 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:01.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:01.923+0000 7fe01a20d640 1 -- 172.21.15.57:0/2061785527 >> v1:172.21.15.57:6789/0 conn(0x7fe014101e80 legacy=0x7fe014104270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:24:01.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:01.923+0000 7fe01a20d640 1 -- 172.21.15.57:0/2061785527 shutdown_connections 2024-06-11T13:24:01.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:01.923+0000 7fe01a20d640 1 -- 172.21.15.57:0/2061785527 >> 172.21.15.57:0/2061785527 conn(0x7fe0140fb8b0 msgr2=0x7fe0140fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:24:01.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:01.923+0000 7fe01a20d640 1 -- 172.21.15.57:0/2061785527 shutdown_connections 2024-06-11T13:24:01.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:01.923+0000 7fe01a20d640 1 -- 172.21.15.57:0/2061785527 wait complete. 2024-06-11T13:24:01.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:01.923+0000 7fe01a20d640 1 Processor -- start 2024-06-11T13:24:01.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:01.924+0000 7fe01a20d640 1 -- start start 2024-06-11T13:24:01.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:01.924+0000 7fe01a20d640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fe01419bea0 con 0x7fe014101e80 2024-06-11T13:24:01.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:01.924+0000 7fe0137fe640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fe014101e80 0x7fe014101700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:51210/0 (socket says 172.21.15.57:51210) 2024-06-11T13:24:01.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:01.924+0000 7fe0137fe640 1 -- 172.21.15.57:0/1867963228 learned_addr learned my addr 172.21.15.57:0/1867963228 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:24:01.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:01.924+0000 7fe010ff9640 1 -- 172.21.15.57:0/1867963228 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3127651482 0 0) 0x7fe01419bea0 con 0x7fe014101e80 2024-06-11T13:24:01.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:01.924+0000 7fe010ff9640 1 -- 172.21.15.57:0/1867963228 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe000005410 con 0x7fe014101e80 2024-06-11T13:24:01.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:01.924+0000 7fe010ff9640 1 -- 172.21.15.57:0/1867963228 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3787959827 0 0) 0x7fe000005410 con 0x7fe014101e80 2024-06-11T13:24:01.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:01.924+0000 7fe010ff9640 1 -- 172.21.15.57:0/1867963228 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe01419bea0 con 0x7fe014101e80 2024-06-11T13:24:01.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:01.924+0000 7fe010ff9640 1 -- 172.21.15.57:0/1867963228 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7fe0040046c0 con 0x7fe014101e80 2024-06-11T13:24:01.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:01.925+0000 7fe010ff9640 1 -- 172.21.15.57:0/1867963228 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2341742766 0 0) 0x7fe01419bea0 con 0x7fe014101e80 2024-06-11T13:24:01.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:01.925+0000 7fe010ff9640 1 -- 172.21.15.57:0/1867963228 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe01419d080 con 0x7fe014101e80 2024-06-11T13:24:01.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:01.925+0000 7fe01a20d640 1 -- 172.21.15.57:0/1867963228 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fe01419c070 con 0x7fe014101e80 2024-06-11T13:24:01.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:01.925+0000 7fe01a20d640 1 -- 172.21.15.57:0/1867963228 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fe01419c5b0 con 0x7fe014101e80 2024-06-11T13:24:01.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:01.925+0000 7fe010ff9640 1 -- 172.21.15.57:0/1867963228 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7fe004001190 con 0x7fe014101e80 2024-06-11T13:24:01.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:01.925+0000 7fe010ff9640 1 -- 172.21.15.57:0/1867963228 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7fe004004d40 con 0x7fe014101e80 2024-06-11T13:24:01.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:01.926+0000 7fe010ff9640 1 -- 172.21.15.57:0/1867963228 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 13) ==== 46308+0+0 (unknown 2805797583 0 0) 0x7fe004004f00 con 0x7fe014101e80 2024-06-11T13:24:01.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:01.926+0000 7fe01a20d640 1 -- 172.21.15.57:0/1867963228 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdff40050b0 con 0x7fe014101e80 2024-06-11T13:24:01.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:01.926+0000 7fe010ff9640 1 -- 172.21.15.57:0/1867963228 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(3..3 src has 1..3) ==== 1069+0+0 (unknown 1210373348 0 0) 0x7fe0040478b0 con 0x7fe014101e80 2024-06-11T13:24:01.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:01.931+0000 7fe010ff9640 1 -- 172.21.15.57:0/1867963228 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7fe0040174f0 con 0x7fe014101e80 2024-06-11T13:24:02.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:02.121+0000 7fe01a20d640 1 -- 172.21.15.57:0/1867963228 --> v1:172.21.15.57:6800/2989665238 -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fdff4000bc0 con 0x7fe00003a3e0 2024-06-11T13:24:02.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:02.125+0000 7fe010ff9640 1 -- 172.21.15.57:0/1867963228 <== mgr.14150 v1:172.21.15.57:6800/2989665238 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (unknown 0 0 0) 0x7fdff4000bc0 con 0x7fe00003a3e0 2024-06-11T13:24:02.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:02.127+0000 7fe01a20d640 1 -- 172.21.15.57:0/1867963228 >> v1:172.21.15.57:6800/2989665238 conn(0x7fe00003a3e0 legacy=0x7fe00003c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:24:02.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:02.127+0000 7fe01a20d640 1 -- 172.21.15.57:0/1867963228 >> v1:172.21.15.57:6789/0 conn(0x7fe014101e80 legacy=0x7fe014101700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:24:02.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:02.127+0000 7fe01a20d640 1 -- 172.21.15.57:0/1867963228 shutdown_connections 2024-06-11T13:24:02.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:02.127+0000 7fe01a20d640 1 -- 172.21.15.57:0/1867963228 >> 172.21.15.57:0/1867963228 conn(0x7fe0140fb8b0 msgr2=0x7fe014104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:24:02.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:02.127+0000 7fe01a20d640 1 -- 172.21.15.57:0/1867963228 shutdown_connections 2024-06-11T13:24:02.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:02.127+0000 7fe01a20d640 1 -- 172.21.15.57:0/1867963228 wait complete. 2024-06-11T13:24:02.847 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi107 2024-06-11T13:24:02.847 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-06-11T13:24:02.847 DEBUG:teuthology.orchestra.run.smithi107:> dd of=/etc/ceph/ceph.conf 2024-06-11T13:24:02.866 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-06-11T13:24:02.866 DEBUG:teuthology.orchestra.run.smithi107:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-11T13:24:02.924 INFO:tasks.cephadm:Adding host smithi107 to orchestrator... 2024-06-11T13:24:02.924 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph orch host add smithi107 2024-06-11T13:24:03.185 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.a/config 2024-06-11T13:24:03.393 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:03 smithi057 ceph-mon[29805]: from='client.14174 v1:172.21.15.57:0/1867963228' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:24:03.394 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:03 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:03.394 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:03 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:24:03.394 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:03 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:24:03.394 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:03 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:24:03.394 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:03 smithi057 ceph-mon[29805]: Updating smithi057:/etc/ceph/ceph.conf 2024-06-11T13:24:03.394 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:03 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:03.394 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:03 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:03.394 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:03 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:04.444 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:04 smithi057 ceph-mon[29805]: Updating smithi057:/var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-11T13:24:04.444 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:04 smithi057 ceph-mon[29805]: Updating smithi057:/etc/ceph/ceph.client.admin.keyring 2024-06-11T13:24:04.444 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:04 smithi057 ceph-mon[29805]: Updating smithi057:/var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-11T13:24:04.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:04.533+0000 7fc61238a640 1 -- 172.21.15.57:0/1109038988 >> v1:172.21.15.57:6789/0 conn(0x7fc60c102560 legacy=0x7fc60c102940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:24:04.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:04.533+0000 7fc61238a640 1 -- 172.21.15.57:0/1109038988 shutdown_connections 2024-06-11T13:24:04.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:04.533+0000 7fc61238a640 1 -- 172.21.15.57:0/1109038988 >> 172.21.15.57:0/1109038988 conn(0x7fc60c0fbdf0 msgr2=0x7fc60c0fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:24:04.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:04.534+0000 7fc61238a640 1 -- 172.21.15.57:0/1109038988 shutdown_connections 2024-06-11T13:24:04.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:04.534+0000 7fc61238a640 1 -- 172.21.15.57:0/1109038988 wait complete. 2024-06-11T13:24:04.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:04.534+0000 7fc61238a640 1 Processor -- start 2024-06-11T13:24:04.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:04.534+0000 7fc61238a640 1 -- start start 2024-06-11T13:24:04.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:04.534+0000 7fc61238a640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fc60c1968c0 con 0x7fc60c102560 2024-06-11T13:24:04.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:04.535+0000 7fc60bfff640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fc60c102560 0x7fc60c1961b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:51230/0 (socket says 172.21.15.57:51230) 2024-06-11T13:24:04.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:04.535+0000 7fc60bfff640 1 -- 172.21.15.57:0/1262242838 learned_addr learned my addr 172.21.15.57:0/1262242838 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:24:04.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:04.535+0000 7fc6097fa640 1 -- 172.21.15.57:0/1262242838 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3014813895 0 0) 0x7fc60c1968c0 con 0x7fc60c102560 2024-06-11T13:24:04.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:04.535+0000 7fc6097fa640 1 -- 172.21.15.57:0/1262242838 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc5f4005410 con 0x7fc60c102560 2024-06-11T13:24:04.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:04.535+0000 7fc6097fa640 1 -- 172.21.15.57:0/1262242838 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4175375456 0 0) 0x7fc5f4005410 con 0x7fc60c102560 2024-06-11T13:24:04.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:04.535+0000 7fc6097fa640 1 -- 172.21.15.57:0/1262242838 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc60c1968c0 con 0x7fc60c102560 2024-06-11T13:24:04.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:04.535+0000 7fc6097fa640 1 -- 172.21.15.57:0/1262242838 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7fc5fc0045d0 con 0x7fc60c102560 2024-06-11T13:24:04.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:04.535+0000 7fc6097fa640 1 -- 172.21.15.57:0/1262242838 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1868469495 0 0) 0x7fc60c1968c0 con 0x7fc60c102560 2024-06-11T13:24:04.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:04.535+0000 7fc6097fa640 1 -- 172.21.15.57:0/1262242838 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc60c196a90 con 0x7fc60c102560 2024-06-11T13:24:04.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:04.535+0000 7fc61238a640 1 -- 172.21.15.57:0/1262242838 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fc60c196d80 con 0x7fc60c102560 2024-06-11T13:24:04.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:04.535+0000 7fc61238a640 1 -- 172.21.15.57:0/1262242838 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fc60c19eee0 con 0x7fc60c102560 2024-06-11T13:24:04.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:04.535+0000 7fc6097fa640 1 -- 172.21.15.57:0/1262242838 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7fc5fc001c00 con 0x7fc60c102560 2024-06-11T13:24:04.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:04.535+0000 7fc6097fa640 1 -- 172.21.15.57:0/1262242838 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7fc5fc005040 con 0x7fc60c102560 2024-06-11T13:24:04.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:04.536+0000 7fc6097fa640 1 -- 172.21.15.57:0/1262242838 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 13) ==== 46308+0+0 (unknown 2805797583 0 0) 0x7fc5fc005200 con 0x7fc60c102560 2024-06-11T13:24:04.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:04.536+0000 7fc61238a640 1 -- 172.21.15.57:0/1262242838 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc60c063000 con 0x7fc60c102560 2024-06-11T13:24:04.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:04.536+0000 7fc6097fa640 1 -- 172.21.15.57:0/1262242838 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(3..3 src has 1..3) ==== 1069+0+0 (unknown 1210373348 0 0) 0x7fc5fc047940 con 0x7fc60c102560 2024-06-11T13:24:04.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:04.541+0000 7fc6097fa640 1 -- 172.21.15.57:0/1262242838 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7fc5fc017450 con 0x7fc60c102560 2024-06-11T13:24:04.724 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:04.730+0000 7fc61238a640 1 -- 172.21.15.57:0/1262242838 --> v1:172.21.15.57:6800/2989665238 -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi107", "target": ["mon-mgr", ""]}) -- 0x7fc60c10c190 con 0x7fc5f403a480 2024-06-11T13:24:05.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:05 smithi057 ceph-mon[29805]: from='client.14176 v1:172.21.15.57:0/1262242838' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi107", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:24:06.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:06 smithi057 ceph-mon[29805]: Deploying cephadm binary to smithi107 2024-06-11T13:24:07.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:07 smithi057 ceph-mon[29805]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:08.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:08.622+0000 7fc6097fa640 1 -- 172.21.15.57:0/1262242838 <== mgr.14150 v1:172.21.15.57:6800/2989665238 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (unknown 0 0 1030021163) 0x7fc60c10c190 con 0x7fc5f403a480 2024-06-11T13:24:08.617 INFO:teuthology.orchestra.run.smithi057.stdout:Added host 'smithi107' with addr '172.21.15.107' 2024-06-11T13:24:08.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:08.625+0000 7fc61238a640 1 -- 172.21.15.57:0/1262242838 >> v1:172.21.15.57:6800/2989665238 conn(0x7fc5f403a480 legacy=0x7fc5f403c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:24:08.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:08.625+0000 7fc61238a640 1 -- 172.21.15.57:0/1262242838 >> v1:172.21.15.57:6789/0 conn(0x7fc60c102560 legacy=0x7fc60c1961b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:24:08.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:08.625+0000 7fc61238a640 1 -- 172.21.15.57:0/1262242838 shutdown_connections 2024-06-11T13:24:08.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:08.625+0000 7fc61238a640 1 -- 172.21.15.57:0/1262242838 >> 172.21.15.57:0/1262242838 conn(0x7fc60c0fbdf0 msgr2=0x7fc60c107a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:24:08.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:08.625+0000 7fc61238a640 1 -- 172.21.15.57:0/1262242838 shutdown_connections 2024-06-11T13:24:08.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:08.625+0000 7fc61238a640 1 -- 172.21.15.57:0/1262242838 wait complete. 2024-06-11T13:24:09.210 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph orch host ls --format=json 2024-06-11T13:24:09.453 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.a/config 2024-06-11T13:24:09.842 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:09 smithi057 ceph-mon[29805]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:09.842 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:09 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:09.842 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:09 smithi057 ceph-mon[29805]: Added host smithi107 2024-06-11T13:24:09.842 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:09 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:24:09.842 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:09 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:09.842 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:09 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:10.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:10.924+0000 7fe105143640 1 -- 172.21.15.57:0/2974151979 >> v1:172.21.15.57:6789/0 conn(0x7fe100100500 legacy=0x7fe1001008e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:24:10.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:10.924+0000 7fe105143640 1 -- 172.21.15.57:0/2974151979 shutdown_connections 2024-06-11T13:24:10.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:10.924+0000 7fe105143640 1 -- 172.21.15.57:0/2974151979 >> 172.21.15.57:0/2974151979 conn(0x7fe1000fbdf0 msgr2=0x7fe1000fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:24:10.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:10.924+0000 7fe105143640 1 -- 172.21.15.57:0/2974151979 shutdown_connections 2024-06-11T13:24:10.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:10.924+0000 7fe105143640 1 -- 172.21.15.57:0/2974151979 wait complete. 2024-06-11T13:24:10.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:10.925+0000 7fe105143640 1 Processor -- start 2024-06-11T13:24:10.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:10.925+0000 7fe105143640 1 -- start start 2024-06-11T13:24:10.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:10.925+0000 7fe105143640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fe100196870 con 0x7fe100100500 2024-06-11T13:24:10.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:10.925+0000 7fe0fed9d640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fe100100500 0x7fe100196160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:55536/0 (socket says 172.21.15.57:55536) 2024-06-11T13:24:10.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:10.925+0000 7fe0fed9d640 1 -- 172.21.15.57:0/2849430365 learned_addr learned my addr 172.21.15.57:0/2849430365 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:24:10.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:10.926+0000 7fe0e7fff640 1 -- 172.21.15.57:0/2849430365 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3768803332 0 0) 0x7fe100196870 con 0x7fe100100500 2024-06-11T13:24:10.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:10.926+0000 7fe0e7fff640 1 -- 172.21.15.57:0/2849430365 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe0e8005410 con 0x7fe100100500 2024-06-11T13:24:10.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:10.926+0000 7fe0e7fff640 1 -- 172.21.15.57:0/2849430365 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2383571138 0 0) 0x7fe0e8005410 con 0x7fe100100500 2024-06-11T13:24:10.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:10.926+0000 7fe0e7fff640 1 -- 172.21.15.57:0/2849430365 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe100196870 con 0x7fe100100500 2024-06-11T13:24:10.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:10.926+0000 7fe0e7fff640 1 -- 172.21.15.57:0/2849430365 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7fe0f0001480 con 0x7fe100100500 2024-06-11T13:24:10.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:10.926+0000 7fe0e7fff640 1 -- 172.21.15.57:0/2849430365 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3820634078 0 0) 0x7fe100196870 con 0x7fe100100500 2024-06-11T13:24:10.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:10.926+0000 7fe0e7fff640 1 -- 172.21.15.57:0/2849430365 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe100196a40 con 0x7fe100100500 2024-06-11T13:24:10.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:10.926+0000 7fe105143640 1 -- 172.21.15.57:0/2849430365 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fe100196d30 con 0x7fe100100500 2024-06-11T13:24:10.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:10.926+0000 7fe105143640 1 -- 172.21.15.57:0/2849430365 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fe10019ee90 con 0x7fe100100500 2024-06-11T13:24:10.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:10.926+0000 7fe0e7fff640 1 -- 172.21.15.57:0/2849430365 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7fe0f00036b0 con 0x7fe100100500 2024-06-11T13:24:10.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:10.926+0000 7fe0e7fff640 1 -- 172.21.15.57:0/2849430365 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7fe0f0004cf0 con 0x7fe100100500 2024-06-11T13:24:10.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:10.927+0000 7fe0e7fff640 1 -- 172.21.15.57:0/2849430365 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 13) ==== 46308+0+0 (unknown 2805797583 0 0) 0x7fe0f0004eb0 con 0x7fe100100500 2024-06-11T13:24:10.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:10.927+0000 7fe105143640 1 -- 172.21.15.57:0/2849430365 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0ec005080 con 0x7fe100100500 2024-06-11T13:24:10.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:10.927+0000 7fe0e7fff640 1 -- 172.21.15.57:0/2849430365 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(3..3 src has 1..3) ==== 1069+0+0 (unknown 1210373348 0 0) 0x7fe0f0047770 con 0x7fe100100500 2024-06-11T13:24:10.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:10.932+0000 7fe0e7fff640 1 -- 172.21.15.57:0/2849430365 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7fe0f0017450 con 0x7fe100100500 2024-06-11T13:24:11.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:10 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:11.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:11.124+0000 7fe105143640 1 -- 172.21.15.57:0/2849430365 --> v1:172.21.15.57:6800/2989665238 -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe0ec000b90 con 0x7fe0e803a430 2024-06-11T13:24:11.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:11.125+0000 7fe0e7fff640 1 -- 172.21.15.57:0/2849430365 <== mgr.14150 v1:172.21.15.57:6800/2989665238 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (unknown 0 0 2106243602) 0x7fe0ec000b90 con 0x7fe0e803a430 2024-06-11T13:24:11.120 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:24:11.120 INFO:teuthology.orchestra.run.smithi057.stdout:[{"addr": "172.21.15.57", "hostname": "smithi057", "labels": [], "status": ""}, {"addr": "172.21.15.107", "hostname": "smithi107", "labels": [], "status": ""}] 2024-06-11T13:24:11.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:11.128+0000 7fe105143640 1 -- 172.21.15.57:0/2849430365 >> v1:172.21.15.57:6800/2989665238 conn(0x7fe0e803a430 legacy=0x7fe0e803c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:24:11.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:11.128+0000 7fe105143640 1 -- 172.21.15.57:0/2849430365 >> v1:172.21.15.57:6789/0 conn(0x7fe100100500 legacy=0x7fe100196160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:24:11.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:11.128+0000 7fe105143640 1 -- 172.21.15.57:0/2849430365 shutdown_connections 2024-06-11T13:24:11.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:11.128+0000 7fe105143640 1 -- 172.21.15.57:0/2849430365 >> 172.21.15.57:0/2849430365 conn(0x7fe1000fbdf0 msgr2=0x7fe100069860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:24:11.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:11.128+0000 7fe105143640 1 -- 172.21.15.57:0/2849430365 shutdown_connections 2024-06-11T13:24:11.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:11.128+0000 7fe105143640 1 -- 172.21.15.57:0/2849430365 wait complete. 2024-06-11T13:24:11.699 INFO:tasks.cephadm:Setting crush tunables to default 2024-06-11T13:24:11.699 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph osd crush tunables default 2024-06-11T13:24:11.944 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.a/config 2024-06-11T13:24:12.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:11 smithi057 ceph-mon[29805]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:12.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:11 smithi057 ceph-mon[29805]: from='client.14178 v1:172.21.15.57:0/2849430365' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-11T13:24:13.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:13.489+0000 7f773cf3b640 1 -- 172.21.15.57:0/983371538 >> v1:172.21.15.57:6789/0 conn(0x7f7738069090 legacy=0x7f7738069470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:24:13.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:13.489+0000 7f773cf3b640 1 -- 172.21.15.57:0/983371538 shutdown_connections 2024-06-11T13:24:13.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:13.489+0000 7f773cf3b640 1 -- 172.21.15.57:0/983371538 >> 172.21.15.57:0/983371538 conn(0x7f7738075dc0 msgr2=0x7f77380761d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:24:13.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:13.489+0000 7f773cf3b640 1 -- 172.21.15.57:0/983371538 shutdown_connections 2024-06-11T13:24:13.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:13.489+0000 7f773cf3b640 1 -- 172.21.15.57:0/983371538 wait complete. 2024-06-11T13:24:13.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:13.490+0000 7f773cf3b640 1 Processor -- start 2024-06-11T13:24:13.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:13.490+0000 7f773cf3b640 1 -- start start 2024-06-11T13:24:13.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:13.490+0000 7f773cf3b640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f7738102220 con 0x7f7738069090 2024-06-11T13:24:13.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:13.491+0000 7f773659c640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f7738069090 0x7f7738100330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:55562/0 (socket says 172.21.15.57:55562) 2024-06-11T13:24:13.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:13.491+0000 7f773659c640 1 -- 172.21.15.57:0/16036252 learned_addr learned my addr 172.21.15.57:0/16036252 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:24:13.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:13.491+0000 7f772f7fe640 1 -- 172.21.15.57:0/16036252 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 167361745 0 0) 0x7f7738102220 con 0x7f7738069090 2024-06-11T13:24:13.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:13.491+0000 7f772f7fe640 1 -- 172.21.15.57:0/16036252 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7724005460 con 0x7f7738069090 2024-06-11T13:24:13.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:13.491+0000 7f772f7fe640 1 -- 172.21.15.57:0/16036252 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 46077046 0 0) 0x7f7724005460 con 0x7f7738069090 2024-06-11T13:24:13.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:13.491+0000 7f772f7fe640 1 -- 172.21.15.57:0/16036252 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7738102220 con 0x7f7738069090 2024-06-11T13:24:13.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:13.491+0000 7f772f7fe640 1 -- 172.21.15.57:0/16036252 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f77200018e0 con 0x7f7738069090 2024-06-11T13:24:13.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:13.491+0000 7f772f7fe640 1 -- 172.21.15.57:0/16036252 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2898522157 0 0) 0x7f7738102220 con 0x7f7738069090 2024-06-11T13:24:13.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:13.491+0000 7f772f7fe640 1 -- 172.21.15.57:0/16036252 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7738100a40 con 0x7f7738069090 2024-06-11T13:24:13.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:13.491+0000 7f773cf3b640 1 -- 172.21.15.57:0/16036252 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f7738100d30 con 0x7f7738069090 2024-06-11T13:24:13.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:13.491+0000 7f773cf3b640 1 -- 172.21.15.57:0/16036252 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f77381a1370 con 0x7f7738069090 2024-06-11T13:24:13.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:13.491+0000 7f772f7fe640 1 -- 172.21.15.57:0/16036252 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f7720001c10 con 0x7f7738069090 2024-06-11T13:24:13.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:13.491+0000 7f772f7fe640 1 -- 172.21.15.57:0/16036252 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f7720004c90 con 0x7f7738069090 2024-06-11T13:24:13.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:13.492+0000 7f772f7fe640 1 -- 172.21.15.57:0/16036252 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 13) ==== 46308+0+0 (unknown 2805797583 0 0) 0x7f7720004e30 con 0x7f7738069090 2024-06-11T13:24:13.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:13.492+0000 7f773cf3b640 1 -- 172.21.15.57:0/16036252 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7718005080 con 0x7f7738069090 2024-06-11T13:24:13.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:13.493+0000 7f772f7fe640 1 -- 172.21.15.57:0/16036252 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(3..3 src has 1..3) ==== 1069+0+0 (unknown 1210373348 0 0) 0x7f7720047790 con 0x7f7738069090 2024-06-11T13:24:13.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:13.497+0000 7f772f7fe640 1 -- 172.21.15.57:0/16036252 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f77200173c0 con 0x7f7738069090 2024-06-11T13:24:13.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:13.692+0000 7f773cf3b640 1 -- 172.21.15.57:0/16036252 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f7718005370 con 0x7f7738069090 2024-06-11T13:24:14.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:13 smithi057 ceph-mon[29805]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:14.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:13 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:14.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:13 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:14.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:13 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:14.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:13 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:14.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:13 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi107", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:24:14.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:13 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:24:14.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:13 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:24:14.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:13 smithi057 ceph-mon[29805]: Updating smithi107:/etc/ceph/ceph.conf 2024-06-11T13:24:14.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:13 smithi057 ceph-mon[29805]: Updating smithi107:/var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-11T13:24:14.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:13 smithi057 ceph-mon[29805]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-06-11T13:24:14.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:13 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:14.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:13 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:14.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:13 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:14.615 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:14.619+0000 7f772f7fe640 1 -- 172.21.15.57:0/16036252 <== mon.0 v1:172.21.15.57:6789/0 10 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (unknown 3126668360 0 0) 0x7f7720016cc0 con 0x7f7738069090 2024-06-11T13:24:14.615 INFO:teuthology.orchestra.run.smithi057.stderr:adjusted tunables profile to default 2024-06-11T13:24:14.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:14.621+0000 7f773cf3b640 1 -- 172.21.15.57:0/16036252 >> v1:172.21.15.57:6800/2989665238 conn(0x7f772403a3b0 legacy=0x7f772403c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:24:14.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:14.621+0000 7f773cf3b640 1 -- 172.21.15.57:0/16036252 >> v1:172.21.15.57:6789/0 conn(0x7f7738069090 legacy=0x7f7738100330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:24:14.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:14.621+0000 7f773cf3b640 1 -- 172.21.15.57:0/16036252 shutdown_connections 2024-06-11T13:24:14.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:14.621+0000 7f773cf3b640 1 -- 172.21.15.57:0/16036252 >> 172.21.15.57:0/16036252 conn(0x7f7738075dc0 msgr2=0x7f7738108ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:24:14.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:14.621+0000 7f773cf3b640 1 -- 172.21.15.57:0/16036252 shutdown_connections 2024-06-11T13:24:14.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:14.621+0000 7f773cf3b640 1 -- 172.21.15.57:0/16036252 wait complete. 2024-06-11T13:24:14.935 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:14 smithi057 ceph-mon[29805]: Updating smithi107:/var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-11T13:24:14.935 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:14 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/16036252' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-06-11T13:24:15.252 INFO:tasks.cephadm:Adding mon.a on smithi057 2024-06-11T13:24:15.252 INFO:tasks.cephadm:Adding mon.c on smithi057 2024-06-11T13:24:15.252 INFO:tasks.cephadm:Adding mon.b on smithi107 2024-06-11T13:24:15.252 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph orch apply mon '3;smithi057:[v1:172.21.15.57:6789]=a;smithi057:[v1:172.21.15.57:6790]=c;smithi107:[v1:172.21.15.107:6789]=b' 2024-06-11T13:24:15.489 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-11T13:24:16.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:15 smithi057 ceph-mon[29805]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:16.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:15 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/16036252' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-06-11T13:24:16.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:15 smithi057 ceph-mon[29805]: osdmap e4: 0 total, 0 up, 0 in 2024-06-11T13:24:16.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:16.954+0000 7f08899cf640 1 -- 172.21.15.107:0/3118839664 >> v1:172.21.15.57:6789/0 conn(0x7f08841025e0 legacy=0x7f08841029c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:24:16.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:16.954+0000 7f08899cf640 1 -- 172.21.15.107:0/3118839664 shutdown_connections 2024-06-11T13:24:16.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:16.954+0000 7f08899cf640 1 -- 172.21.15.107:0/3118839664 >> 172.21.15.107:0/3118839664 conn(0x7f08840fbdd0 msgr2=0x7f08840fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:24:16.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:16.954+0000 7f08899cf640 1 -- 172.21.15.107:0/3118839664 shutdown_connections 2024-06-11T13:24:16.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:16.954+0000 7f08899cf640 1 -- 172.21.15.107:0/3118839664 wait complete. 2024-06-11T13:24:16.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:16.955+0000 7f08899cf640 1 Processor -- start 2024-06-11T13:24:16.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:16.955+0000 7f08899cf640 1 -- start start 2024-06-11T13:24:16.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:16.955+0000 7f08899cf640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f0884196940 con 0x7f08841025e0 2024-06-11T13:24:16.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:16.956+0000 7f0882ffd640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f08841025e0 0x7f0884196230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.107:49696/0 (socket says 172.21.15.107:49696) 2024-06-11T13:24:16.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:16.956+0000 7f0882ffd640 1 -- 172.21.15.107:0/2413997773 learned_addr learned my addr 172.21.15.107:0/2413997773 (peer_addr_for_me v1:172.21.15.107:0/0) 2024-06-11T13:24:16.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:16.956+0000 7f08889cd640 1 -- 172.21.15.107:0/2413997773 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 561174059 0 0) 0x7f0884196940 con 0x7f08841025e0 2024-06-11T13:24:16.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:16.956+0000 7f08889cd640 1 -- 172.21.15.107:0/2413997773 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0870005410 con 0x7f08841025e0 2024-06-11T13:24:16.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:16.956+0000 7f08889cd640 1 -- 172.21.15.107:0/2413997773 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1273232639 0 0) 0x7f0870005410 con 0x7f08841025e0 2024-06-11T13:24:16.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:16.956+0000 7f08889cd640 1 -- 172.21.15.107:0/2413997773 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0884196940 con 0x7f08841025e0 2024-06-11T13:24:16.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:16.956+0000 7f08889cd640 1 -- 172.21.15.107:0/2413997773 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f08740045d0 con 0x7f08841025e0 2024-06-11T13:24:16.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:16.956+0000 7f08889cd640 1 -- 172.21.15.107:0/2413997773 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3396385706 0 0) 0x7f0884196940 con 0x7f08841025e0 2024-06-11T13:24:16.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:16.956+0000 7f08889cd640 1 -- 172.21.15.107:0/2413997773 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0884196b10 con 0x7f08841025e0 2024-06-11T13:24:16.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:16.956+0000 7f08899cf640 1 -- 172.21.15.107:0/2413997773 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f0884196e00 con 0x7f08841025e0 2024-06-11T13:24:16.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:16.956+0000 7f08899cf640 1 -- 172.21.15.107:0/2413997773 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f088419ef60 con 0x7f08841025e0 2024-06-11T13:24:16.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:16.956+0000 7f08889cd640 1 -- 172.21.15.107:0/2413997773 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f08740036b0 con 0x7f08841025e0 2024-06-11T13:24:16.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:16.956+0000 7f08889cd640 1 -- 172.21.15.107:0/2413997773 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f0874004d70 con 0x7f08841025e0 2024-06-11T13:24:16.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:16.957+0000 7f08899cf640 1 -- 172.21.15.107:0/2413997773 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0864005080 con 0x7f08841025e0 2024-06-11T13:24:16.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:16.958+0000 7f08889cd640 1 -- 172.21.15.107:0/2413997773 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 13) ==== 46308+0+0 (unknown 2805797583 0 0) 0x7f0874004fb0 con 0x7f08841025e0 2024-06-11T13:24:16.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:16.958+0000 7f08889cd640 1 -- 172.21.15.107:0/2413997773 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(4..4 src has 1..4) ==== 1069+0+0 (unknown 1503136288 0 0) 0x7f0874047830 con 0x7f08841025e0 2024-06-11T13:24:16.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:16.962+0000 7f08889cd640 1 -- 172.21.15.107:0/2413997773 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f0874014bb0 con 0x7f08841025e0 2024-06-11T13:24:17.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:17.156+0000 7f08899cf640 1 -- 172.21.15.107:0/2413997773 --> v1:172.21.15.57:6800/2989665238 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi057:[v1:172.21.15.57:6789]=a;smithi057:[v1:172.21.15.57:6790]=c;smithi107:[v1:172.21.15.107:6789]=b", "target": ["mon-mgr", ""]}) -- 0x7f0864000c60 con 0x7f087003a3e0 2024-06-11T13:24:17.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:17.168+0000 7f08889cd640 1 -- 172.21.15.107:0/2413997773 <== mgr.14150 v1:172.21.15.57:6800/2989665238 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (unknown 0 0 3265049985) 0x7f0864000c60 con 0x7f087003a3e0 2024-06-11T13:24:17.162 INFO:teuthology.orchestra.run.smithi107.stdout:Scheduled mon update... 2024-06-11T13:24:17.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:17.170+0000 7f08899cf640 1 -- 172.21.15.107:0/2413997773 >> v1:172.21.15.57:6800/2989665238 conn(0x7f087003a3e0 legacy=0x7f087003c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:24:17.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:17.170+0000 7f08899cf640 1 -- 172.21.15.107:0/2413997773 >> v1:172.21.15.57:6789/0 conn(0x7f08841025e0 legacy=0x7f0884196230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:24:17.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:17.170+0000 7f08899cf640 1 -- 172.21.15.107:0/2413997773 shutdown_connections 2024-06-11T13:24:17.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:17.170+0000 7f08899cf640 1 -- 172.21.15.107:0/2413997773 >> 172.21.15.107:0/2413997773 conn(0x7f08840fbdd0 msgr2=0x7f08841079a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:24:17.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:17.170+0000 7f08899cf640 1 -- 172.21.15.107:0/2413997773 shutdown_connections 2024-06-11T13:24:17.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:17.170+0000 7f08899cf640 1 -- 172.21.15.107:0/2413997773 wait complete. 2024-06-11T13:24:17.733 DEBUG:teuthology.orchestra.run.smithi057:mon.c> sudo journalctl -f -n 0 -u ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@mon.c.service 2024-06-11T13:24:17.736 DEBUG:teuthology.orchestra.run.smithi107:mon.b> sudo journalctl -f -n 0 -u ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@mon.b.service 2024-06-11T13:24:17.738 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-11T13:24:17.738 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-11T13:24:18.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:18 smithi057 ceph-mon[29805]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:18.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:18 smithi057 ceph-mon[29805]: from='client.14182 v1:172.21.15.107:0/2413997773' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi057:[v1:172.21.15.57:6789]=a;smithi057:[v1:172.21.15.57:6790]=c;smithi107:[v1:172.21.15.107:6789]=b", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:24:18.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:18 smithi057 ceph-mon[29805]: Saving service mon spec with placement smithi057:[v1:172.21.15.57:6789]=a;smithi057:[v1:172.21.15.57:6790]=c;smithi107:[v1:172.21.15.107:6789]=b;count:3 2024-06-11T13:24:18.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:18 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:18.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:18 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:24:18.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:18 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:24:18.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:18 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:24:18.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:18 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:18.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:18 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-11T13:24:18.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:18 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:24:19.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:19 smithi057 ceph-mon[29805]: Deploying daemon mon.b on smithi107 2024-06-11T13:24:19.838 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.b/config 2024-06-11T13:24:20.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:20 smithi057 ceph-mon[29805]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:21.042 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:20 smithi107 systemd[1]: Starting Ceph mon.b for 98a6635e-27f5-11ef-bc9b-c7b262605968... 2024-06-11T13:24:21.531 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:21 smithi057 ceph-mon[29805]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:21.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:21.565+0000 7f79c0364640 1 -- 172.21.15.107:0/3052532450 >> v1:172.21.15.57:6789/0 conn(0x7f79b8100600 legacy=0x7f79b81009e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:24:21.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:21.565+0000 7f79c0364640 1 -- 172.21.15.107:0/3052532450 shutdown_connections 2024-06-11T13:24:21.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:21.565+0000 7f79c0364640 1 -- 172.21.15.107:0/3052532450 >> 172.21.15.107:0/3052532450 conn(0x7f79b80fbe90 msgr2=0x7f79b80fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:24:21.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:21.565+0000 7f79c0364640 1 -- 172.21.15.107:0/3052532450 shutdown_connections 2024-06-11T13:24:21.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:21.565+0000 7f79c0364640 1 -- 172.21.15.107:0/3052532450 wait complete. 2024-06-11T13:24:21.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:21.566+0000 7f79c0364640 1 Processor -- start 2024-06-11T13:24:21.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:21.566+0000 7f79c0364640 1 -- start start 2024-06-11T13:24:21.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:21.566+0000 7f79c0364640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f79b8196990 con 0x7f79b8100600 2024-06-11T13:24:21.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:21.566+0000 7f79be100640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f79b8100600 0x7f79b8196280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.107:33080/0 (socket says 172.21.15.107:33080) 2024-06-11T13:24:21.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:21.566+0000 7f79be100640 1 -- 172.21.15.107:0/1845573698 learned_addr learned my addr 172.21.15.107:0/1845573698 (peer_addr_for_me v1:172.21.15.107:0/0) 2024-06-11T13:24:21.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:21.567+0000 7f79af7fe640 1 -- 172.21.15.107:0/1845573698 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3731259334 0 0) 0x7f79b8196990 con 0x7f79b8100600 2024-06-11T13:24:21.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:21.567+0000 7f79af7fe640 1 -- 172.21.15.107:0/1845573698 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f79a4005410 con 0x7f79b8100600 2024-06-11T13:24:21.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:21.567+0000 7f79af7fe640 1 -- 172.21.15.107:0/1845573698 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 557246662 0 0) 0x7f79a4005410 con 0x7f79b8100600 2024-06-11T13:24:21.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:21.567+0000 7f79af7fe640 1 -- 172.21.15.107:0/1845573698 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f79b8196990 con 0x7f79b8100600 2024-06-11T13:24:21.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:21.567+0000 7f79af7fe640 1 -- 172.21.15.107:0/1845573698 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f79b40045d0 con 0x7f79b8100600 2024-06-11T13:24:21.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:21.567+0000 7f79af7fe640 1 -- 172.21.15.107:0/1845573698 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4038609252 0 0) 0x7f79b8196990 con 0x7f79b8100600 2024-06-11T13:24:21.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:21.567+0000 7f79af7fe640 1 -- 172.21.15.107:0/1845573698 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79b8196b60 con 0x7f79b8100600 2024-06-11T13:24:21.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:21.567+0000 7f79c0364640 1 -- 172.21.15.107:0/1845573698 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f79b8196e50 con 0x7f79b8100600 2024-06-11T13:24:21.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:21.567+0000 7f79c0364640 1 -- 172.21.15.107:0/1845573698 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f79b819efb0 con 0x7f79b8100600 2024-06-11T13:24:21.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:21.567+0000 7f79af7fe640 1 -- 172.21.15.107:0/1845573698 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f79b4001c00 con 0x7f79b8100600 2024-06-11T13:24:21.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:21.567+0000 7f79af7fe640 1 -- 172.21.15.107:0/1845573698 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3040948378 0 0) 0x7f79b4004d00 con 0x7f79b8100600 2024-06-11T13:24:21.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:21.568+0000 7f79c0364640 1 -- 172.21.15.107:0/1845573698 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f799c005080 con 0x7f79b8100600 2024-06-11T13:24:21.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:21.568+0000 7f79af7fe640 1 -- 172.21.15.107:0/1845573698 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 13) ==== 46308+0+0 (unknown 2805797583 0 0) 0x7f79b4004ec0 con 0x7f79b8100600 2024-06-11T13:24:21.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:21.568+0000 7f79af7fe640 1 -- 172.21.15.107:0/1845573698 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(4..4 src has 1..4) ==== 1069+0+0 (unknown 1503136288 0 0) 0x7f79b40478a0 con 0x7f79b8100600 2024-06-11T13:24:21.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:21.573+0000 7f79af7fe640 1 -- 172.21.15.107:0/1845573698 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f79b401eff0 con 0x7f79b8100600 2024-06-11T13:24:21.680 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 podman[33125]: 2024-06-11 13:24:21.360729323 +0000 UTC m=+0.018164158 image pull 2d7459ec65a098c35c774154617f2893b0ad051f56ce89b72ab795f65a97fc22 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310 2024-06-11T13:24:21.680 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 podman[33125]: 2024-06-11 13:24:21.686304842 +0000 UTC m=+0.343739657 container create 95117eeb1e103a8bb5e9ff99530bc3442ca96fb5cec22dc29dee295055197180 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mon-b, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.35.3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, RELEASE=squid-3f51ea3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-06-11T13:24:21.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:21.823+0000 7f79c0364640 1 -- 172.21.15.107:0/1845573698 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f799c005370 con 0x7f79b8100600 2024-06-11T13:24:21.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:21.823+0000 7f79af7fe640 1 -- 172.21.15.107:0/1845573698 <== mon.0 v1:172.21.15.57:6789/0 10 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+694 (unknown 2237029548 0 2820755656) 0x7f79b4005580 con 0x7f79b8100600 2024-06-11T13:24:21.818 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:24:21.818 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":1,"fsid":"98a6635e-27f5-11ef-bc9b-c7b262605968","modified":"2024-06-11T13:22:49.372656Z","created":"2024-06-11T13:22:49.372656Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6789","nonce":0}]},"addr":"172.21.15.57:6789/0","public_addr":"172.21.15.57:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-11T13:24:21.819 INFO:teuthology.orchestra.run.smithi107.stderr:dumped monmap epoch 1 2024-06-11T13:24:21.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:21.826+0000 7f79c0364640 1 -- 172.21.15.107:0/1845573698 >> v1:172.21.15.57:6800/2989665238 conn(0x7f79a403a430 legacy=0x7f79a403c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:24:21.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:21.826+0000 7f79c0364640 1 -- 172.21.15.107:0/1845573698 >> v1:172.21.15.57:6789/0 conn(0x7f79b8100600 legacy=0x7f79b8196280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:24:21.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:21.826+0000 7f79c0364640 1 -- 172.21.15.107:0/1845573698 shutdown_connections 2024-06-11T13:24:21.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:21.826+0000 7f79c0364640 1 -- 172.21.15.107:0/1845573698 >> 172.21.15.107:0/1845573698 conn(0x7f79b80fbe90 msgr2=0x7f79b8069390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:24:21.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:21.826+0000 7f79c0364640 1 -- 172.21.15.107:0/1845573698 shutdown_connections 2024-06-11T13:24:21.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:21.826+0000 7f79c0364640 1 -- 172.21.15.107:0/1845573698 wait complete. 2024-06-11T13:24:21.931 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 podman[33125]: 2024-06-11 13:24:21.907078189 +0000 UTC m=+0.564513020 container init 95117eeb1e103a8bb5e9ff99530bc3442ca96fb5cec22dc29dee295055197180 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.35.3, org.label-schema.schema-version=1.0, RELEASE=squid-3f51ea3, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-06-11T13:24:21.931 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 podman[33125]: 2024-06-11 13:24:21.910372899 +0000 UTC m=+0.567807721 container start 95117eeb1e103a8bb5e9ff99530bc3442ca96fb5cec22dc29dee295055197180 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mon-b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-3f51ea3, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, io.buildah.version=1.35.3) 2024-06-11T13:24:21.931 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: set uid:gid to 167:167 (ceph:ceph) 2024-06-11T13:24:21.931 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: ceph version 19.0.0-2976-g3f51ea39 (3f51ea3944d39ca13aa40153c9a4ee298306679d) squid (rc), process ceph-mon, pid 2 2024-06-11T13:24:21.931 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: pidfile_write: ignore empty --pid-file 2024-06-11T13:24:21.931 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: load: jerasure load: lrc 2024-06-11T13:24:21.931 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: RocksDB version: 7.9.2 2024-06-11T13:24:21.931 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Git sha 0 2024-06-11T13:24:21.931 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Compile date 2024-06-10 15:42:49 2024-06-11T13:24:21.931 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: DB SUMMARY 2024-06-11T13:24:21.932 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: DB Session ID: 66YQR8BXPNDE502XM97H 2024-06-11T13:24:21.932 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: CURRENT file: CURRENT 2024-06-11T13:24:21.932 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: IDENTITY file: IDENTITY 2024-06-11T13:24:21.932 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-06-11T13:24:21.932 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-06-11T13:24:21.932 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 476 ; 2024-06-11T13:24:21.932 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.error_if_exists: 0 2024-06-11T13:24:21.932 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.create_if_missing: 0 2024-06-11T13:24:21.932 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.paranoid_checks: 1 2024-06-11T13:24:21.932 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.flush_verify_memtable_count: 1 2024-06-11T13:24:21.932 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-06-11T13:24:21.932 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-06-11T13:24:21.932 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.env: 0x55e8927d5740 2024-06-11T13:24:21.932 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.fs: PosixFileSystem 2024-06-11T13:24:21.932 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.info_log: 0x55e894074f60 2024-06-11T13:24:21.933 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.max_file_opening_threads: 16 2024-06-11T13:24:21.933 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.statistics: (nil) 2024-06-11T13:24:21.933 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.use_fsync: 0 2024-06-11T13:24:21.933 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.max_log_file_size: 0 2024-06-11T13:24:21.933 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-11T13:24:21.933 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.log_file_time_to_roll: 0 2024-06-11T13:24:21.933 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.keep_log_file_num: 1000 2024-06-11T13:24:21.933 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.recycle_log_file_num: 0 2024-06-11T13:24:21.933 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.allow_fallocate: 1 2024-06-11T13:24:21.934 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.allow_mmap_reads: 0 2024-06-11T13:24:21.934 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.allow_mmap_writes: 0 2024-06-11T13:24:21.934 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.use_direct_reads: 0 2024-06-11T13:24:21.934 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-11T13:24:21.934 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.create_missing_column_families: 0 2024-06-11T13:24:21.934 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.db_log_dir: 2024-06-11T13:24:21.934 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.wal_dir: 2024-06-11T13:24:21.934 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.table_cache_numshardbits: 6 2024-06-11T13:24:21.934 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.WAL_ttl_seconds: 0 2024-06-11T13:24:21.934 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.WAL_size_limit_MB: 0 2024-06-11T13:24:21.934 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-11T13:24:21.934 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-11T13:24:21.934 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.is_fd_close_on_exec: 1 2024-06-11T13:24:21.934 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.advise_random_on_open: 1 2024-06-11T13:24:21.935 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.db_write_buffer_size: 0 2024-06-11T13:24:21.935 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.write_buffer_manager: 0x55e8940841e0 2024-06-11T13:24:21.935 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-11T13:24:21.935 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-11T13:24:21.935 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.use_adaptive_mutex: 0 2024-06-11T13:24:21.935 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.rate_limiter: (nil) 2024-06-11T13:24:21.935 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-11T13:24:21.935 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.wal_recovery_mode: 2 2024-06-11T13:24:21.935 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.enable_thread_tracking: 0 2024-06-11T13:24:21.935 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.enable_pipelined_write: 0 2024-06-11T13:24:21.935 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.unordered_write: 0 2024-06-11T13:24:21.935 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-11T13:24:21.935 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-11T13:24:21.935 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-11T13:24:22.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-11T13:24:22.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.row_cache: None 2024-06-11T13:24:22.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.wal_filter: None 2024-06-11T13:24:22.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-11T13:24:22.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.allow_ingest_behind: 0 2024-06-11T13:24:22.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.two_write_queues: 0 2024-06-11T13:24:22.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.manual_wal_flush: 0 2024-06-11T13:24:22.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.wal_compression: 0 2024-06-11T13:24:22.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.atomic_flush: 0 2024-06-11T13:24:22.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-11T13:24:22.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.persist_stats_to_disk: 0 2024-06-11T13:24:22.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.write_dbid_to_manifest: 0 2024-06-11T13:24:22.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.log_readahead_size: 0 2024-06-11T13:24:22.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-06-11T13:24:22.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.best_efforts_recovery: 0 2024-06-11T13:24:22.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-06-11T13:24:22.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-06-11T13:24:22.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.allow_data_in_errors: 0 2024-06-11T13:24:22.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.db_host_id: __hostname__ 2024-06-11T13:24:22.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.enforce_single_del_contracts: true 2024-06-11T13:24:22.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.max_background_jobs: 2 2024-06-11T13:24:22.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.max_background_compactions: -1 2024-06-11T13:24:22.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.max_subcompactions: 1 2024-06-11T13:24:22.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-11T13:24:22.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-11T13:24:22.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.delayed_write_rate : 16777216 2024-06-11T13:24:22.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.max_total_wal_size: 0 2024-06-11T13:24:22.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-11T13:24:22.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.stats_dump_period_sec: 600 2024-06-11T13:24:22.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.stats_persist_period_sec: 600 2024-06-11T13:24:22.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-11T13:24:22.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.max_open_files: -1 2024-06-11T13:24:22.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.bytes_per_sync: 0 2024-06-11T13:24:22.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.wal_bytes_per_sync: 0 2024-06-11T13:24:22.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.strict_bytes_per_sync: 0 2024-06-11T13:24:22.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.compaction_readahead_size: 0 2024-06-11T13:24:22.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.max_background_flushes: -1 2024-06-11T13:24:22.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Compression algorithms supported: 2024-06-11T13:24:22.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: kZSTD supported: 0 2024-06-11T13:24:22.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: kXpressCompression supported: 0 2024-06-11T13:24:22.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: kBZip2Compression supported: 0 2024-06-11T13:24:22.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-11T13:24:22.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: kLZ4Compression supported: 1 2024-06-11T13:24:22.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: kZlibCompression supported: 1 2024-06-11T13:24:22.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: kLZ4HCCompression supported: 1 2024-06-11T13:24:22.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: kSnappyCompression supported: 1 2024-06-11T13:24:22.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Fast CRC32 supported: Supported on x86 2024-06-11T13:24:22.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: DMutex implementation: pthread_mutex_t 2024-06-11T13:24:22.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-06-11T13:24:22.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-06-11T13:24:22.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-11T13:24:22.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.merge_operator: 2024-06-11T13:24:22.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.compaction_filter: None 2024-06-11T13:24:22.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.compaction_filter_factory: None 2024-06-11T13:24:22.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.sst_partitioner_factory: None 2024-06-11T13:24:22.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.memtable_factory: SkipListFactory 2024-06-11T13:24:22.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.table_factory: BlockBasedTable 2024-06-11T13:24:22.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e894074c20) 2024-06-11T13:24:22.300 INFO:journalctl@ceph.mon.b.smithi107.stdout: cache_index_and_filter_blocks: 1 2024-06-11T13:24:22.300 INFO:journalctl@ceph.mon.b.smithi107.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-11T13:24:22.300 INFO:journalctl@ceph.mon.b.smithi107.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-11T13:24:22.300 INFO:journalctl@ceph.mon.b.smithi107.stdout: pin_top_level_index_and_filter: 1 2024-06-11T13:24:22.300 INFO:journalctl@ceph.mon.b.smithi107.stdout: index_type: 0 2024-06-11T13:24:22.300 INFO:journalctl@ceph.mon.b.smithi107.stdout: data_block_index_type: 0 2024-06-11T13:24:22.300 INFO:journalctl@ceph.mon.b.smithi107.stdout: index_shortening: 1 2024-06-11T13:24:22.300 INFO:journalctl@ceph.mon.b.smithi107.stdout: data_block_hash_table_util_ratio: 0.750000 2024-06-11T13:24:22.300 INFO:journalctl@ceph.mon.b.smithi107.stdout: checksum: 4 2024-06-11T13:24:22.300 INFO:journalctl@ceph.mon.b.smithi107.stdout: no_block_cache: 0 2024-06-11T13:24:22.301 INFO:journalctl@ceph.mon.b.smithi107.stdout: block_cache: 0x55e89406b090 2024-06-11T13:24:22.301 INFO:journalctl@ceph.mon.b.smithi107.stdout: block_cache_name: BinnedLRUCache 2024-06-11T13:24:22.301 INFO:journalctl@ceph.mon.b.smithi107.stdout: block_cache_options: 2024-06-11T13:24:22.301 INFO:journalctl@ceph.mon.b.smithi107.stdout: capacity : 536870912 2024-06-11T13:24:22.301 INFO:journalctl@ceph.mon.b.smithi107.stdout: num_shard_bits : 4 2024-06-11T13:24:22.301 INFO:journalctl@ceph.mon.b.smithi107.stdout: strict_capacity_limit : 0 2024-06-11T13:24:22.301 INFO:journalctl@ceph.mon.b.smithi107.stdout: high_pri_pool_ratio: 0.000 2024-06-11T13:24:22.301 INFO:journalctl@ceph.mon.b.smithi107.stdout: block_cache_compressed: (nil) 2024-06-11T13:24:22.301 INFO:journalctl@ceph.mon.b.smithi107.stdout: persistent_cache: (nil) 2024-06-11T13:24:22.301 INFO:journalctl@ceph.mon.b.smithi107.stdout: block_size: 4096 2024-06-11T13:24:22.301 INFO:journalctl@ceph.mon.b.smithi107.stdout: block_size_deviation: 10 2024-06-11T13:24:22.301 INFO:journalctl@ceph.mon.b.smithi107.stdout: block_restart_interval: 16 2024-06-11T13:24:22.301 INFO:journalctl@ceph.mon.b.smithi107.stdout: index_block_restart_interval: 1 2024-06-11T13:24:22.301 INFO:journalctl@ceph.mon.b.smithi107.stdout: metadata_block_size: 4096 2024-06-11T13:24:22.302 INFO:journalctl@ceph.mon.b.smithi107.stdout: partition_filters: 0 2024-06-11T13:24:22.302 INFO:journalctl@ceph.mon.b.smithi107.stdout: use_delta_encoding: 1 2024-06-11T13:24:22.302 INFO:journalctl@ceph.mon.b.smithi107.stdout: filter_policy: bloomfilter 2024-06-11T13:24:22.302 INFO:journalctl@ceph.mon.b.smithi107.stdout: whole_key_filtering: 1 2024-06-11T13:24:22.302 INFO:journalctl@ceph.mon.b.smithi107.stdout: verify_compression: 0 2024-06-11T13:24:22.302 INFO:journalctl@ceph.mon.b.smithi107.stdout: read_amp_bytes_per_bit: 0 2024-06-11T13:24:22.302 INFO:journalctl@ceph.mon.b.smithi107.stdout: format_version: 5 2024-06-11T13:24:22.302 INFO:journalctl@ceph.mon.b.smithi107.stdout: enable_index_compression: 1 2024-06-11T13:24:22.303 INFO:journalctl@ceph.mon.b.smithi107.stdout: block_align: 0 2024-06-11T13:24:22.303 INFO:journalctl@ceph.mon.b.smithi107.stdout: max_auto_readahead_size: 262144 2024-06-11T13:24:22.303 INFO:journalctl@ceph.mon.b.smithi107.stdout: prepopulate_block_cache: 0 2024-06-11T13:24:22.304 INFO:journalctl@ceph.mon.b.smithi107.stdout: initial_auto_readahead_size: 8192 2024-06-11T13:24:22.304 INFO:journalctl@ceph.mon.b.smithi107.stdout: num_file_reads_for_auto_readahead: 2 2024-06-11T13:24:22.304 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.write_buffer_size: 33554432 2024-06-11T13:24:22.304 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.max_write_buffer_number: 2 2024-06-11T13:24:22.304 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.compression: NoCompression 2024-06-11T13:24:22.304 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.bottommost_compression: Disabled 2024-06-11T13:24:22.304 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.prefix_extractor: nullptr 2024-06-11T13:24:22.304 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-11T13:24:22.304 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.num_levels: 7 2024-06-11T13:24:22.304 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-11T13:24:22.304 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-11T13:24:22.304 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-11T13:24:22.304 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-11T13:24:22.304 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-11T13:24:22.305 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-11T13:24:22.305 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-11T13:24:22.305 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-11T13:24:22.305 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-06-11T13:24:22.305 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-11T13:24:22.305 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-06-11T13:24:22.305 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-06-11T13:24:22.305 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.compression_opts.window_bits: -14 2024-06-11T13:24:22.305 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.compression_opts.level: 32767 2024-06-11T13:24:22.305 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.compression_opts.strategy: 0 2024-06-11T13:24:22.305 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-11T13:24:22.305 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-11T13:24:22.305 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-06-11T13:24:22.306 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-06-11T13:24:22.306 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.compression_opts.enabled: false 2024-06-11T13:24:22.306 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-06-11T13:24:22.306 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-11T13:24:22.306 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-11T13:24:22.306 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-11T13:24:22.306 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.target_file_size_base: 67108864 2024-06-11T13:24:22.306 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.target_file_size_multiplier: 1 2024-06-11T13:24:22.306 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-11T13:24:22.306 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-11T13:24:22.306 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-11T13:24:22.306 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-11T13:24:22.306 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-11T13:24:22.306 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-11T13:24:22.307 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-11T13:24:22.307 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-11T13:24:22.307 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-11T13:24:22.307 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-11T13:24:22.307 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-11T13:24:22.307 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-11T13:24:22.307 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-06-11T13:24:22.307 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.arena_block_size: 1048576 2024-06-11T13:24:22.307 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-11T13:24:22.307 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-11T13:24:22.307 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.disable_auto_compactions: 0 2024-06-11T13:24:22.308 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-11T13:24:22.308 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-11T13:24:22.308 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-11T13:24:22.308 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-11T13:24:22.308 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-11T13:24:22.308 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-11T13:24:22.308 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-11T13:24:22.308 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-11T13:24:22.308 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-11T13:24:22.308 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-11T13:24:22.308 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.table_properties_collectors: 2024-06-11T13:24:22.308 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.inplace_update_support: 0 2024-06-11T13:24:22.308 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.inplace_update_num_locks: 10000 2024-06-11T13:24:22.308 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-11T13:24:22.308 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-11T13:24:22.309 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.memtable_huge_page_size: 0 2024-06-11T13:24:22.309 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.bloom_locality: 0 2024-06-11T13:24:22.309 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.max_successive_merges: 0 2024-06-11T13:24:22.309 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.optimize_filters_for_hits: 0 2024-06-11T13:24:22.309 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.paranoid_file_checks: 0 2024-06-11T13:24:22.309 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.force_consistency_checks: 1 2024-06-11T13:24:22.309 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.report_bg_io_stats: 0 2024-06-11T13:24:22.309 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.ttl: 2592000 2024-06-11T13:24:22.309 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.periodic_compaction_seconds: 0 2024-06-11T13:24:22.309 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-06-11T13:24:22.309 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-06-11T13:24:22.309 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.enable_blob_files: false 2024-06-11T13:24:22.309 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.min_blob_size: 0 2024-06-11T13:24:22.309 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.blob_file_size: 268435456 2024-06-11T13:24:22.310 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.blob_compression_type: NoCompression 2024-06-11T13:24:22.310 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.enable_blob_garbage_collection: false 2024-06-11T13:24:22.310 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-06-11T13:24:22.310 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-06-11T13:24:22.310 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-06-11T13:24:22.310 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.blob_file_starting_level: 0 2024-06-11T13:24:22.310 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-06-11T13:24:22.310 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-06-11T13:24:22.310 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-06-11T13:24:22.310 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: a0cff890-a819-4206-9261-9b26646f1851 2024-06-11T13:24:22.310 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: EVENT_LOG_v1 {"time_micros": 1718112261937975, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-06-11T13:24:22.310 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-06-11T13:24:22.310 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: EVENT_LOG_v1 {"time_micros": 1718112261938457, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1587, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 488, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 366, "raw_average_value_size": 73, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1718112261, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a0cff890-a819-4206-9261-9b26646f1851", "db_session_id": "66YQR8BXPNDE502XM97H", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-06-11T13:24:22.311 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: EVENT_LOG_v1 {"time_micros": 1718112261938571, "job": 1, "event": "recovery_finished"} 2024-06-11T13:24:22.311 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-06-11T13:24:22.311 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-06-11T13:24:22.311 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55e894098e00 2024-06-11T13:24:22.311 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: DB pointer 0x55e89418e000 2024-06-11T13:24:22.311 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-06-11T13:24:22.311 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-06-11T13:24:22.311 INFO:journalctl@ceph.mon.b.smithi107.stdout: ** DB Stats ** 2024-06-11T13:24:22.311 INFO:journalctl@ceph.mon.b.smithi107.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-11T13:24:22.311 INFO:journalctl@ceph.mon.b.smithi107.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-11T13:24:22.311 INFO:journalctl@ceph.mon.b.smithi107.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-11T13:24:22.311 INFO:journalctl@ceph.mon.b.smithi107.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-11T13:24:22.311 INFO:journalctl@ceph.mon.b.smithi107.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-11T13:24:22.311 INFO:journalctl@ceph.mon.b.smithi107.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-11T13:24:22.312 INFO:journalctl@ceph.mon.b.smithi107.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-11T13:24:22.312 INFO:journalctl@ceph.mon.b.smithi107.stdout: 2024-06-11T13:24:22.312 INFO:journalctl@ceph.mon.b.smithi107.stdout: ** Compaction Stats [default] ** 2024-06-11T13:24:22.312 INFO:journalctl@ceph.mon.b.smithi107.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-11T13:24:22.312 INFO:journalctl@ceph.mon.b.smithi107.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-06-11T13:24:22.312 INFO:journalctl@ceph.mon.b.smithi107.stdout: L0 1/0 1.55 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-11T13:24:22.312 INFO:journalctl@ceph.mon.b.smithi107.stdout: Sum 1/0 1.55 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-11T13:24:22.312 INFO:journalctl@ceph.mon.b.smithi107.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-11T13:24:22.312 INFO:journalctl@ceph.mon.b.smithi107.stdout: 2024-06-11T13:24:22.312 INFO:journalctl@ceph.mon.b.smithi107.stdout: ** Compaction Stats [default] ** 2024-06-11T13:24:22.312 INFO:journalctl@ceph.mon.b.smithi107.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-11T13:24:22.312 INFO:journalctl@ceph.mon.b.smithi107.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-11T13:24:22.312 INFO:journalctl@ceph.mon.b.smithi107.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-11T13:24:22.313 INFO:journalctl@ceph.mon.b.smithi107.stdout: 2024-06-11T13:24:22.313 INFO:journalctl@ceph.mon.b.smithi107.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-06-11T13:24:22.313 INFO:journalctl@ceph.mon.b.smithi107.stdout: 2024-06-11T13:24:22.313 INFO:journalctl@ceph.mon.b.smithi107.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-11T13:24:22.313 INFO:journalctl@ceph.mon.b.smithi107.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-06-11T13:24:22.313 INFO:journalctl@ceph.mon.b.smithi107.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-11T13:24:22.313 INFO:journalctl@ceph.mon.b.smithi107.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-11T13:24:22.313 INFO:journalctl@ceph.mon.b.smithi107.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-11T13:24:22.313 INFO:journalctl@ceph.mon.b.smithi107.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-11T13:24:22.313 INFO:journalctl@ceph.mon.b.smithi107.stdout: Cumulative compaction: 0.00 GB write, 0.50 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-11T13:24:22.313 INFO:journalctl@ceph.mon.b.smithi107.stdout: Interval compaction: 0.00 GB write, 0.50 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-11T13:24:22.313 INFO:journalctl@ceph.mon.b.smithi107.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-11T13:24:22.313 INFO:journalctl@ceph.mon.b.smithi107.stdout: Block cache BinnedLRUCache@0x55e89406b090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-06-11T13:24:22.314 INFO:journalctl@ceph.mon.b.smithi107.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-06-11T13:24:22.314 INFO:journalctl@ceph.mon.b.smithi107.stdout: 2024-06-11T13:24:22.314 INFO:journalctl@ceph.mon.b.smithi107.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-11T13:24:22.314 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-06-11T13:24:22.314 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: using public_addrv v1:172.21.15.107:6789/0 2024-06-11T13:24:22.314 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: starting mon.b rank -1 at public addrs v1:172.21.15.107:6789/0 at bind addrs v1:172.21.15.107:6789/0 mon_data /var/lib/ceph/mon/ceph-b fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 2024-06-11T13:24:22.314 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: mon.b@-1(???) e0 preinit fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 2024-06-11T13:24:22.314 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: mon.b@-1(synchronizing).mds e1 new map 2024-06-11T13:24:22.314 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: mon.b@-1(synchronizing).mds e1 print_map 2024-06-11T13:24:22.314 INFO:journalctl@ceph.mon.b.smithi107.stdout: e1 2024-06-11T13:24:22.314 INFO:journalctl@ceph.mon.b.smithi107.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-11T13:24:22.314 INFO:journalctl@ceph.mon.b.smithi107.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-11T13:24:22.314 INFO:journalctl@ceph.mon.b.smithi107.stdout: legacy client fscid: -1 2024-06-11T13:24:22.315 INFO:journalctl@ceph.mon.b.smithi107.stdout: 2024-06-11T13:24:22.315 INFO:journalctl@ceph.mon.b.smithi107.stdout: No filesystems configured 2024-06-11T13:24:22.315 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-06-11T13:24:22.315 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-06-11T13:24:22.315 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-06-11T13:24:22.315 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-06-11T13:24:22.315 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-06-11T13:24:22.315 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-06-11T13:24:22.315 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-06-11T13:24:22.315 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-11T13:24:22.315 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-11T13:24:22.315 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-11T13:24:22.315 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:22.316 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:22.316 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: Added host smithi107 2024-06-11T13:24:22.316 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:24:22.316 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:22.316 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:22.316 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:22.316 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:22.316 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: from='client.14178 v1:172.21.15.57:0/2849430365' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-11T13:24:22.316 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:22.316 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:22.316 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:22.316 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:22.316 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:22.316 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi107", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:24:22.317 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:24:22.317 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:24:22.317 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: Updating smithi107:/etc/ceph/ceph.conf 2024-06-11T13:24:22.317 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: Updating smithi107:/var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-11T13:24:22.317 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-06-11T13:24:22.317 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:22.317 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:22.317 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:22.317 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: Updating smithi107:/var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-11T13:24:22.317 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/16036252' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-06-11T13:24:22.317 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:22.317 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/16036252' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-06-11T13:24:22.317 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: osdmap e4: 0 total, 0 up, 0 in 2024-06-11T13:24:22.317 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:22.318 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: from='client.14182 v1:172.21.15.107:0/2413997773' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi057:[v1:172.21.15.57:6789]=a;smithi057:[v1:172.21.15.57:6790]=c;smithi107:[v1:172.21.15.107:6789]=b", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:24:22.318 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: Saving service mon spec with placement smithi057:[v1:172.21.15.57:6789]=a;smithi057:[v1:172.21.15.57:6790]=c;smithi107:[v1:172.21.15.107:6789]=b;count:3 2024-06-11T13:24:22.318 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:22.318 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:24:22.318 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:24:22.318 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:24:22.318 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:22.318 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-11T13:24:22.318 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:24:22.318 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: Deploying daemon mon.b on smithi107 2024-06-11T13:24:22.318 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:22.318 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:22.318 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:21 smithi107 ceph-mon[33178]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-06-11T13:24:22.319 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:22 smithi107 bash[33125]: 95117eeb1e103a8bb5e9ff99530bc3442ca96fb5cec22dc29dee295055197180 2024-06-11T13:24:22.319 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:22 smithi107 systemd[1]: Started Ceph mon.b for 98a6635e-27f5-11ef-bc9b-c7b262605968. 2024-06-11T13:24:23.550 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-11T13:24:23.551 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-11T13:24:23.794 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.b/config 2024-06-11T13:24:26.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:26.991+0000 7fb9cc59b640 1 -- 172.21.15.107:0/1499033666 >> v1:172.21.15.57:6789/0 conn(0x7fb9c4100610 legacy=0x7fb9c41009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:24:26.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:26.991+0000 7fb9cc59b640 1 -- 172.21.15.107:0/1499033666 shutdown_connections 2024-06-11T13:24:26.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:26.991+0000 7fb9cc59b640 1 -- 172.21.15.107:0/1499033666 >> 172.21.15.107:0/1499033666 conn(0x7fb9c40fbee0 msgr2=0x7fb9c40fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:24:26.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:26.991+0000 7fb9cc59b640 1 -- 172.21.15.107:0/1499033666 shutdown_connections 2024-06-11T13:24:26.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:26.991+0000 7fb9cc59b640 1 -- 172.21.15.107:0/1499033666 wait complete. 2024-06-11T13:24:26.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:26.992+0000 7fb9cc59b640 1 Processor -- start 2024-06-11T13:24:26.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:26.992+0000 7fb9cc59b640 1 -- start start 2024-06-11T13:24:26.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:26.992+0000 7fb9cc59b640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fb9c41a25c0 con 0x7fb9c4100610 2024-06-11T13:24:26.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:26.992+0000 7fb9cc59b640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fb9c41a37a0 con 0x7fb9c4196a90 2024-06-11T13:24:26.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:26.992+0000 7fb9c9b36640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7fb9c4196a90 0x7fb9c41a0ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.107:60434/0 (socket says 172.21.15.107:60434) 2024-06-11T13:24:26.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:26.992+0000 7fb9c9b36640 1 -- 172.21.15.107:0/3533684122 learned_addr learned my addr 172.21.15.107:0/3533684122 (peer_addr_for_me v1:172.21.15.107:0/0) 2024-06-11T13:24:26.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:26.993+0000 7fb9b77fe640 1 -- 172.21.15.107:0/3533684122 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4137563954 0 0) 0x7fb9c41a25c0 con 0x7fb9c4100610 2024-06-11T13:24:26.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:26.993+0000 7fb9b77fe640 1 -- 172.21.15.107:0/3533684122 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb9b0003520 con 0x7fb9c4100610 2024-06-11T13:24:26.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:26.993+0000 7fb9b77fe640 1 -- 172.21.15.107:0/3533684122 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 459632447 0 0) 0x7fb9b0003520 con 0x7fb9c4100610 2024-06-11T13:24:26.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:26.993+0000 7fb9b77fe640 1 -- 172.21.15.107:0/3533684122 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb9c41a25c0 con 0x7fb9c4100610 2024-06-11T13:24:26.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:26.993+0000 7fb9b77fe640 1 -- 172.21.15.107:0/3533684122 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 239+0+0 (unknown 4081101537 0 0) 0x7fb9b80014c0 con 0x7fb9c4100610 2024-06-11T13:24:26.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:26.993+0000 7fb9b77fe640 1 -- 172.21.15.107:0/3533684122 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3460231575 0 0) 0x7fb9c41a25c0 con 0x7fb9c4100610 2024-06-11T13:24:26.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:26.993+0000 7fb9b77fe640 1 -- 172.21.15.107:0/3533684122 >> v1:172.21.15.107:6789/0 conn(0x7fb9c4196a90 legacy=0x7fb9c41a0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:24:26.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:26.993+0000 7fb9b77fe640 1 -- 172.21.15.107:0/3533684122 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9c41a4980 con 0x7fb9c4100610 2024-06-11T13:24:26.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:26.993+0000 7fb9cc59b640 1 -- 172.21.15.107:0/3533684122 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fb9c41a2790 con 0x7fb9c4100610 2024-06-11T13:24:26.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:26.993+0000 7fb9cc59b640 1 -- 172.21.15.107:0/3533684122 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fb9c41a2c90 con 0x7fb9c4100610 2024-06-11T13:24:26.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:26.994+0000 7fb9b77fe640 1 -- 172.21.15.107:0/3533684122 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7fb9b80039f0 con 0x7fb9c4100610 2024-06-11T13:24:26.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:26.994+0000 7fb9b77fe640 1 -- 172.21.15.107:0/3533684122 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 239+0+0 (unknown 4081101537 0 0) 0x7fb9b8004bb0 con 0x7fb9c4100610 2024-06-11T13:24:26.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:26.994+0000 7fb9b77fe640 1 -- 172.21.15.107:0/3533684122 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 13) ==== 46308+0+0 (unknown 2805797583 0 0) 0x7fb9b8004d50 con 0x7fb9c4100610 2024-06-11T13:24:26.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:26.995+0000 7fb9cc59b640 1 -- 172.21.15.107:0/3533684122 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9a8005080 con 0x7fb9c4100610 2024-06-11T13:24:26.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:26.995+0000 7fb9b77fe640 1 -- 172.21.15.107:0/3533684122 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(4..4 src has 1..4) ==== 1069+0+0 (unknown 1503136288 0 0) 0x7fb9b80474c0 con 0x7fb9c4100610 2024-06-11T13:24:26.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:27.002+0000 7fb9b77fe640 1 -- 172.21.15.107:0/3533684122 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7fb9b8016210 con 0x7fb9c4100610 2024-06-11T13:24:27.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:27.251+0000 7fb9cc59b640 1 -- 172.21.15.107:0/3533684122 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fb9a8005370 con 0x7fb9c4100610 2024-06-11T13:24:27.247 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:27 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-11T13:24:27.247 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:27 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-11T13:24:27.247 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:27 smithi107 ceph-mon[33178]: mon.a calling monitor election 2024-06-11T13:24:27.247 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:27 smithi107 ceph-mon[33178]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:27.247 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:27 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-11T13:24:27.247 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:27 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-11T13:24:27.247 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:27 smithi107 ceph-mon[33178]: mon.b calling monitor election 2024-06-11T13:24:27.247 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:27 smithi107 ceph-mon[33178]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:27.247 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:27 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-11T13:24:27.247 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:27 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-11T13:24:27.247 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:27 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-11T13:24:27.247 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:27 smithi107 ceph-mon[33178]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-06-11T13:24:27.248 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:27 smithi107 ceph-mon[33178]: monmap epoch 2 2024-06-11T13:24:27.248 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:27 smithi107 ceph-mon[33178]: fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 2024-06-11T13:24:27.248 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:27 smithi107 ceph-mon[33178]: last_changed 2024-06-11T13:24:21.959567+0000 2024-06-11T13:24:27.248 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:27 smithi107 ceph-mon[33178]: created 2024-06-11T13:22:49.372656+0000 2024-06-11T13:24:27.248 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:27 smithi107 ceph-mon[33178]: min_mon_release 19 (squid) 2024-06-11T13:24:27.248 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:27 smithi107 ceph-mon[33178]: election_strategy: 1 2024-06-11T13:24:27.248 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:27 smithi107 ceph-mon[33178]: 0: v1:172.21.15.57:6789/0 mon.a 2024-06-11T13:24:27.248 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:27 smithi107 ceph-mon[33178]: 1: v1:172.21.15.107:6789/0 mon.b 2024-06-11T13:24:27.248 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:27 smithi107 ceph-mon[33178]: fsmap 2024-06-11T13:24:27.248 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:27 smithi107 ceph-mon[33178]: osdmap e4: 0 total, 0 up, 0 in 2024-06-11T13:24:27.248 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:27 smithi107 ceph-mon[33178]: mgrmap e13: y(active, since 40s) 2024-06-11T13:24:27.248 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:27 smithi107 ceph-mon[33178]: overall HEALTH_OK 2024-06-11T13:24:27.248 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:27 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:27.248 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:27 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:27.248 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:27 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:27.249 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:27 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-11T13:24:27.249 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:27 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:24:27.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:27.252+0000 7fb9b77fe640 1 -- 172.21.15.107:0/3533684122 <== mon.0 v1:172.21.15.57:6789/0 10 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+912 (unknown 3557084514 0 3482497693) 0x7fb9b8015b10 con 0x7fb9c4100610 2024-06-11T13:24:27.249 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:24:27.249 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":2,"fsid":"98a6635e-27f5-11ef-bc9b-c7b262605968","modified":"2024-06-11T13:24:21.959567Z","created":"2024-06-11T13:22:49.372656Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6789","nonce":0}]},"addr":"172.21.15.57:6789/0","public_addr":"172.21.15.57:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-06-11T13:24:27.250 INFO:teuthology.orchestra.run.smithi107.stderr:dumped monmap epoch 2 2024-06-11T13:24:27.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:27.254+0000 7fb9cc59b640 1 -- 172.21.15.107:0/3533684122 >> v1:172.21.15.57:6800/2989665238 conn(0x7fb9b003a470 legacy=0x7fb9b003c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:24:27.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:27.254+0000 7fb9cc59b640 1 -- 172.21.15.107:0/3533684122 >> v1:172.21.15.57:6789/0 conn(0x7fb9c4100610 legacy=0x7fb9c4196380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:24:27.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:27.254+0000 7fb9cc59b640 1 -- 172.21.15.107:0/3533684122 shutdown_connections 2024-06-11T13:24:27.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:27.254+0000 7fb9cc59b640 1 -- 172.21.15.107:0/3533684122 >> 172.21.15.107:0/3533684122 conn(0x7fb9c40fbee0 msgr2=0x7fb9c4107670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:24:27.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:27.254+0000 7fb9cc59b640 1 -- 172.21.15.107:0/3533684122 shutdown_connections 2024-06-11T13:24:27.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:27.254+0000 7fb9cc59b640 1 -- 172.21.15.107:0/3533684122 wait complete. 2024-06-11T13:24:27.432 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:27 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-11T13:24:27.432 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:27 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-11T13:24:27.432 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:27 smithi057 ceph-mon[29805]: mon.a calling monitor election 2024-06-11T13:24:27.432 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:27 smithi057 ceph-mon[29805]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:27.432 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:27 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-11T13:24:27.432 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:27 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-11T13:24:27.432 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:27 smithi057 ceph-mon[29805]: mon.b calling monitor election 2024-06-11T13:24:27.432 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:27 smithi057 ceph-mon[29805]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:27.433 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:27 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-11T13:24:27.433 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:27 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-11T13:24:27.433 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:27 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-11T13:24:27.433 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:27 smithi057 ceph-mon[29805]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-06-11T13:24:27.433 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:27 smithi057 ceph-mon[29805]: monmap epoch 2 2024-06-11T13:24:27.433 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:27 smithi057 ceph-mon[29805]: fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 2024-06-11T13:24:27.433 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:27 smithi057 ceph-mon[29805]: last_changed 2024-06-11T13:24:21.959567+0000 2024-06-11T13:24:27.433 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:27 smithi057 ceph-mon[29805]: created 2024-06-11T13:22:49.372656+0000 2024-06-11T13:24:27.433 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:27 smithi057 ceph-mon[29805]: min_mon_release 19 (squid) 2024-06-11T13:24:27.433 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:27 smithi057 ceph-mon[29805]: election_strategy: 1 2024-06-11T13:24:27.433 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:27 smithi057 ceph-mon[29805]: 0: v1:172.21.15.57:6789/0 mon.a 2024-06-11T13:24:27.433 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:27 smithi057 ceph-mon[29805]: 1: v1:172.21.15.107:6789/0 mon.b 2024-06-11T13:24:27.433 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:27 smithi057 ceph-mon[29805]: fsmap 2024-06-11T13:24:27.433 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:27 smithi057 ceph-mon[29805]: osdmap e4: 0 total, 0 up, 0 in 2024-06-11T13:24:27.434 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:27 smithi057 ceph-mon[29805]: mgrmap e13: y(active, since 40s) 2024-06-11T13:24:27.434 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:27 smithi057 ceph-mon[29805]: overall HEALTH_OK 2024-06-11T13:24:27.434 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:27 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:27.434 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:27 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:27.434 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:27 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:27.434 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:27 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-11T13:24:27.434 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:27 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:24:28.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:28 smithi107 ceph-mon[33178]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:28.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:28 smithi107 ceph-mon[33178]: Deploying daemon mon.c on smithi057 2024-06-11T13:24:28.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:28 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.107:0/3533684122' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-11T13:24:28.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:28 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-11T13:24:28.298 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:28 smithi057 ceph-mon[29805]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:28.298 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:28 smithi057 ceph-mon[29805]: Deploying daemon mon.c on smithi057 2024-06-11T13:24:28.299 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:28 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.107:0/3533684122' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-11T13:24:28.299 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:28 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-11T13:24:28.801 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-11T13:24:28.802 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-11T13:24:29.046 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.b/config 2024-06-11T13:24:30.066 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:30 smithi107 ceph-mon[33178]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:30.222 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:30 smithi057 ceph-mon[29805]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:30.781 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:30 smithi057 systemd[1]: Started Ceph mon.c for 98a6635e-27f5-11ef-bc9b-c7b262605968. 2024-06-11T13:24:35.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:35.424+0000 7fb67e09a640 1 -- 172.21.15.107:0/3517972526 >> v1:172.21.15.57:6789/0 conn(0x7fb664003560 legacy=0x7fb6640059f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:24:35.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:35.424+0000 7fb67e09a640 1 -- 172.21.15.107:0/3517972526 shutdown_connections 2024-06-11T13:24:35.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:35.424+0000 7fb67e09a640 1 -- 172.21.15.107:0/3517972526 >> 172.21.15.107:0/3517972526 conn(0x7fb6780fbee0 msgr2=0x7fb6780fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:24:35.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:35.424+0000 7fb67e09a640 1 -- 172.21.15.107:0/3517972526 shutdown_connections 2024-06-11T13:24:35.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:35.424+0000 7fb67e09a640 1 -- 172.21.15.107:0/3517972526 wait complete. 2024-06-11T13:24:35.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:35.425+0000 7fb67e09a640 1 Processor -- start 2024-06-11T13:24:35.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:35.426+0000 7fb67e09a640 1 -- start start 2024-06-11T13:24:35.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:35.426+0000 7fb67e09a640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fb6781a61e0 con 0x7fb664003560 2024-06-11T13:24:35.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:35.426+0000 7fb67e09a640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fb6781a73e0 con 0x7fb6781a2540 2024-06-11T13:24:35.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:35.426+0000 7fb67e09a640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fb6781a8620 con 0x7fb678196a10 2024-06-11T13:24:35.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:35.426+0000 7fb677fff640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7fb6781a2540 0x7fb6781a4930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.107:57366/0 (socket says 172.21.15.107:57366) 2024-06-11T13:24:35.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:35.426+0000 7fb677fff640 1 -- 172.21.15.107:0/3012984839 learned_addr learned my addr 172.21.15.107:0/3012984839 (peer_addr_for_me v1:172.21.15.107:0/0) 2024-06-11T13:24:35.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:35.428+0000 7fb674ff9640 1 -- 172.21.15.107:0/3012984839 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3912476678 0 0) 0x7fb6781a61e0 con 0x7fb664003560 2024-06-11T13:24:35.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:35.428+0000 7fb674ff9640 1 -- 172.21.15.107:0/3012984839 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb660003b60 con 0x7fb664003560 2024-06-11T13:24:35.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:35.428+0000 7fb674ff9640 1 -- 172.21.15.107:0/3012984839 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3962952158 0 0) 0x7fb660003b60 con 0x7fb664003560 2024-06-11T13:24:35.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:35.428+0000 7fb674ff9640 1 -- 172.21.15.107:0/3012984839 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb6781a61e0 con 0x7fb664003560 2024-06-11T13:24:35.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:35.429+0000 7fb674ff9640 1 -- 172.21.15.107:0/3012984839 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fb6680019f0 con 0x7fb664003560 2024-06-11T13:24:35.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:35.429+0000 7fb674ff9640 1 -- 172.21.15.107:0/3012984839 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2347899342 0 0) 0x7fb6781a61e0 con 0x7fb664003560 2024-06-11T13:24:35.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:35.429+0000 7fb674ff9640 1 -- 172.21.15.107:0/3012984839 >> v1:172.21.15.57:6790/0 conn(0x7fb678196a10 legacy=0x7fb6781a0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:24:35.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:35.429+0000 7fb674ff9640 1 -- 172.21.15.107:0/3012984839 >> v1:172.21.15.107:6789/0 conn(0x7fb6781a2540 legacy=0x7fb6781a4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:24:35.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:35.429+0000 7fb674ff9640 1 -- 172.21.15.107:0/3012984839 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6781a9860 con 0x7fb664003560 2024-06-11T13:24:35.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:35.429+0000 7fb67e09a640 1 -- 172.21.15.107:0/3012984839 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fb6781a6410 con 0x7fb664003560 2024-06-11T13:24:35.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:35.429+0000 7fb67e09a640 1 -- 172.21.15.107:0/3012984839 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fb6781a6910 con 0x7fb664003560 2024-06-11T13:24:35.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:35.429+0000 7fb674ff9640 1 -- 172.21.15.107:0/3012984839 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7fb6680011e0 con 0x7fb664003560 2024-06-11T13:24:35.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:35.429+0000 7fb674ff9640 1 -- 172.21.15.107:0/3012984839 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fb668004b20 con 0x7fb664003560 2024-06-11T13:24:35.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:35.430+0000 7fb67e09a640 1 -- 172.21.15.107:0/3012984839 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb664007b50 con 0x7fb664003560 2024-06-11T13:24:35.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:35.431+0000 7fb674ff9640 1 -- 172.21.15.107:0/3012984839 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 13) ==== 46308+0+0 (unknown 2805797583 0 0) 0x7fb6680101b0 con 0x7fb664003560 2024-06-11T13:24:35.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:35.431+0000 7fb674ff9640 1 -- 172.21.15.107:0/3012984839 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(4..4 src has 1..4) ==== 1069+0+0 (unknown 1503136288 0 0) 0x7fb6680464f0 con 0x7fb664003560 2024-06-11T13:24:35.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:35.440+0000 7fb674ff9640 1 -- 172.21.15.107:0/3012984839 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7fb6680161c0 con 0x7fb664003560 2024-06-11T13:24:35.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:35.698+0000 7fb67e09a640 1 -- 172.21.15.107:0/3012984839 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fb664007e40 con 0x7fb664003560 2024-06-11T13:24:35.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:35.699+0000 7fb674ff9640 1 -- 172.21.15.107:0/3012984839 <== mon.0 v1:172.21.15.57:6789/0 10 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1127 (unknown 409899127 0 2890400141) 0x7fb668015ac0 con 0x7fb664003560 2024-06-11T13:24:35.694 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-11T13:24:35.694 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":3,"fsid":"98a6635e-27f5-11ef-bc9b-c7b262605968","modified":"2024-06-11T13:24:30.404524Z","created":"2024-06-11T13:22:49.372656Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6789","nonce":0}]},"addr":"172.21.15.57:6789/0","public_addr":"172.21.15.57:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6790","nonce":0}]},"addr":"172.21.15.57:6790/0","public_addr":"172.21.15.57:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-06-11T13:24:35.695 INFO:teuthology.orchestra.run.smithi107.stderr:dumped monmap epoch 3 2024-06-11T13:24:35.695 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:35 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-11T13:24:35.696 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:35 smithi107 ceph-mon[33178]: mon.a calling monitor election 2024-06-11T13:24:35.696 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:35 smithi107 ceph-mon[33178]: mon.b calling monitor election 2024-06-11T13:24:35.696 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:35 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-11T13:24:35.696 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:35 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-11T13:24:35.696 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:35 smithi107 ceph-mon[33178]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:35.696 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:35 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-11T13:24:35.697 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:35 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-11T13:24:35.697 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:35 smithi107 ceph-mon[33178]: mon.c calling monitor election 2024-06-11T13:24:35.697 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:35 smithi107 ceph-mon[33178]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:35.697 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:35 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-11T13:24:35.697 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:35 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-11T13:24:35.697 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:35 smithi107 ceph-mon[33178]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:35.697 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:35 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-11T13:24:35.697 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:35 smithi107 ceph-mon[33178]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-06-11T13:24:35.697 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:35 smithi107 ceph-mon[33178]: monmap epoch 3 2024-06-11T13:24:35.698 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:35 smithi107 ceph-mon[33178]: fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 2024-06-11T13:24:35.698 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:35 smithi107 ceph-mon[33178]: last_changed 2024-06-11T13:24:30.404524+0000 2024-06-11T13:24:35.698 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:35 smithi107 ceph-mon[33178]: created 2024-06-11T13:22:49.372656+0000 2024-06-11T13:24:35.698 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:35 smithi107 ceph-mon[33178]: min_mon_release 19 (squid) 2024-06-11T13:24:35.698 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:35 smithi107 ceph-mon[33178]: election_strategy: 1 2024-06-11T13:24:35.698 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:35 smithi107 ceph-mon[33178]: 0: v1:172.21.15.57:6789/0 mon.a 2024-06-11T13:24:35.698 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:35 smithi107 ceph-mon[33178]: 1: v1:172.21.15.107:6789/0 mon.b 2024-06-11T13:24:35.698 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:35 smithi107 ceph-mon[33178]: 2: v1:172.21.15.57:6790/0 mon.c 2024-06-11T13:24:35.698 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:35 smithi107 ceph-mon[33178]: fsmap 2024-06-11T13:24:35.699 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:35 smithi107 ceph-mon[33178]: osdmap e4: 0 total, 0 up, 0 in 2024-06-11T13:24:35.699 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:35 smithi107 ceph-mon[33178]: mgrmap e13: y(active, since 48s) 2024-06-11T13:24:35.699 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:35 smithi107 ceph-mon[33178]: overall HEALTH_OK 2024-06-11T13:24:35.699 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:35 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:35.699 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:35 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:35.699 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:35 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:35.699 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:35 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:35.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:35.701+0000 7fb67e09a640 1 -- 172.21.15.107:0/3012984839 >> v1:172.21.15.57:6800/2989665238 conn(0x7fb66003a1b0 legacy=0x7fb66003c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:24:35.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:35.701+0000 7fb67e09a640 1 -- 172.21.15.107:0/3012984839 >> v1:172.21.15.57:6789/0 conn(0x7fb664003560 legacy=0x7fb678196300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:24:35.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:35.701+0000 7fb67e09a640 1 -- 172.21.15.107:0/3012984839 shutdown_connections 2024-06-11T13:24:35.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:35.701+0000 7fb67e09a640 1 -- 172.21.15.107:0/3012984839 >> 172.21.15.107:0/3012984839 conn(0x7fb6780fbee0 msgr2=0x7fb6780fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:24:35.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:35.701+0000 7fb67e09a640 1 -- 172.21.15.107:0/3012984839 shutdown_connections 2024-06-11T13:24:35.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:35.701+0000 7fb67e09a640 1 -- 172.21.15.107:0/3012984839 wait complete. 2024-06-11T13:24:35.782 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-11T13:24:35.782 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[40588]: mon.a calling monitor election 2024-06-11T13:24:35.782 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[40588]: mon.b calling monitor election 2024-06-11T13:24:35.782 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-11T13:24:35.782 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-11T13:24:35.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[40588]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:35.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-11T13:24:35.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-11T13:24:35.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[40588]: mon.c calling monitor election 2024-06-11T13:24:35.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[40588]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:35.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-11T13:24:35.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-11T13:24:35.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[40588]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:35.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-11T13:24:35.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[40588]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-06-11T13:24:35.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[40588]: monmap epoch 3 2024-06-11T13:24:35.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[40588]: fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 2024-06-11T13:24:35.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[40588]: last_changed 2024-06-11T13:24:30.404524+0000 2024-06-11T13:24:35.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[40588]: created 2024-06-11T13:22:49.372656+0000 2024-06-11T13:24:35.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[40588]: min_mon_release 19 (squid) 2024-06-11T13:24:35.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[40588]: election_strategy: 1 2024-06-11T13:24:35.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[40588]: 0: v1:172.21.15.57:6789/0 mon.a 2024-06-11T13:24:35.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[40588]: 1: v1:172.21.15.107:6789/0 mon.b 2024-06-11T13:24:35.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[40588]: 2: v1:172.21.15.57:6790/0 mon.c 2024-06-11T13:24:35.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[40588]: fsmap 2024-06-11T13:24:35.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[40588]: osdmap e4: 0 total, 0 up, 0 in 2024-06-11T13:24:35.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[40588]: mgrmap e13: y(active, since 48s) 2024-06-11T13:24:35.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[40588]: overall HEALTH_OK 2024-06-11T13:24:35.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:35.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:35.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:35.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:35.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-11T13:24:35.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[29805]: mon.a calling monitor election 2024-06-11T13:24:35.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[29805]: mon.b calling monitor election 2024-06-11T13:24:35.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-11T13:24:35.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-11T13:24:35.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[29805]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:35.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-11T13:24:35.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-11T13:24:35.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[29805]: mon.c calling monitor election 2024-06-11T13:24:35.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[29805]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:35.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-11T13:24:35.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-11T13:24:35.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[29805]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:35.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-11T13:24:35.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[29805]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-06-11T13:24:35.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[29805]: monmap epoch 3 2024-06-11T13:24:35.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[29805]: fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 2024-06-11T13:24:35.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[29805]: last_changed 2024-06-11T13:24:30.404524+0000 2024-06-11T13:24:35.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[29805]: created 2024-06-11T13:22:49.372656+0000 2024-06-11T13:24:35.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[29805]: min_mon_release 19 (squid) 2024-06-11T13:24:35.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[29805]: election_strategy: 1 2024-06-11T13:24:35.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[29805]: 0: v1:172.21.15.57:6789/0 mon.a 2024-06-11T13:24:35.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[29805]: 1: v1:172.21.15.107:6789/0 mon.b 2024-06-11T13:24:35.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[29805]: 2: v1:172.21.15.57:6790/0 mon.c 2024-06-11T13:24:35.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[29805]: fsmap 2024-06-11T13:24:35.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[29805]: osdmap e4: 0 total, 0 up, 0 in 2024-06-11T13:24:35.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[29805]: mgrmap e13: y(active, since 48s) 2024-06-11T13:24:35.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[29805]: overall HEALTH_OK 2024-06-11T13:24:35.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:35.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:35.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:35.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:35 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:36.214 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-06-11T13:24:36.214 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph config generate-minimal-conf 2024-06-11T13:24:36.481 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.a/config 2024-06-11T13:24:36.758 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:36 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:24:36.758 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:36 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.107:0/3012984839' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-11T13:24:36.758 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:36 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-11T13:24:36.758 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:36 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:36.758 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:36 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:36.758 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:36 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:36.781 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:36 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:24:36.781 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:36 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.107:0/3012984839' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-11T13:24:36.781 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:36 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-11T13:24:36.781 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:36 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:36.781 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:36 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:36.782 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:36 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:36.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:36 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:24:36.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:36 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.107:0/3012984839' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-11T13:24:36.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:36 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-11T13:24:36.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:36 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:36.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:36 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:36.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:36 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:38.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:38.144+0000 7f493ac64640 1 -- 172.21.15.57:0/1146143292 >> v1:172.21.15.57:6789/0 conn(0x7f49341000f0 legacy=0x7f49341004d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:24:38.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:38.144+0000 7f493ac64640 1 -- 172.21.15.57:0/1146143292 shutdown_connections 2024-06-11T13:24:38.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:38.144+0000 7f493ac64640 1 -- 172.21.15.57:0/1146143292 >> 172.21.15.57:0/1146143292 conn(0x7f49340fbdf0 msgr2=0x7f49340fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:24:38.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:38.144+0000 7f493ac64640 1 -- 172.21.15.57:0/1146143292 shutdown_connections 2024-06-11T13:24:38.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:38.144+0000 7f493ac64640 1 -- 172.21.15.57:0/1146143292 wait complete. 2024-06-11T13:24:38.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:38.145+0000 7f493ac64640 1 Processor -- start 2024-06-11T13:24:38.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:38.145+0000 7f493ac64640 1 -- start start 2024-06-11T13:24:38.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:38.145+0000 7f493ac64640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f49341a85a0 con 0x7f49341000f0 2024-06-11T13:24:38.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:38.145+0000 7f493ac64640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f49341a97a0 con 0x7f4934198cf0 2024-06-11T13:24:38.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:38.145+0000 7f493ac64640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f49341aa9e0 con 0x7f49341a4870 2024-06-11T13:24:38.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:38.145+0000 7f4938a00640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f49341000f0 0x7f49341985e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:32974/0 (socket says 172.21.15.57:32974) 2024-06-11T13:24:38.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:38.145+0000 7f4938a00640 1 -- 172.21.15.57:0/1929702754 learned_addr learned my addr 172.21.15.57:0/1929702754 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:24:38.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:38.145+0000 7f4931ffb640 1 -- 172.21.15.57:0/1929702754 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2689625011 0 0) 0x7f49341a85a0 con 0x7f49341000f0 2024-06-11T13:24:38.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:38.145+0000 7f4931ffb640 1 -- 172.21.15.57:0/1929702754 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4920003b60 con 0x7f49341000f0 2024-06-11T13:24:38.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:38.146+0000 7f4931ffb640 1 -- 172.21.15.57:0/1929702754 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2145507037 0 0) 0x7f49341aa9e0 con 0x7f49341a4870 2024-06-11T13:24:38.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:38.146+0000 7f4931ffb640 1 -- 172.21.15.57:0/1929702754 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f49341a85a0 con 0x7f49341a4870 2024-06-11T13:24:38.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:38.146+0000 7f4931ffb640 1 -- 172.21.15.57:0/1929702754 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2647462054 0 0) 0x7f4920003b60 con 0x7f49341000f0 2024-06-11T13:24:38.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:38.146+0000 7f4931ffb640 1 -- 172.21.15.57:0/1929702754 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f49341aa9e0 con 0x7f49341000f0 2024-06-11T13:24:38.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:38.146+0000 7f4931ffb640 1 -- 172.21.15.57:0/1929702754 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f4924001cc0 con 0x7f49341000f0 2024-06-11T13:24:38.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:38.146+0000 7f4931ffb640 1 -- 172.21.15.57:0/1929702754 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2399225746 0 0) 0x7f49341a97a0 con 0x7f4934198cf0 2024-06-11T13:24:38.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:38.146+0000 7f4931ffb640 1 -- 172.21.15.57:0/1929702754 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4920003b60 con 0x7f4934198cf0 2024-06-11T13:24:38.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:38.146+0000 7f4931ffb640 1 -- 172.21.15.57:0/1929702754 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1421266458 0 0) 0x7f49341a85a0 con 0x7f49341a4870 2024-06-11T13:24:38.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:38.146+0000 7f4931ffb640 1 -- 172.21.15.57:0/1929702754 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f49341a97a0 con 0x7f49341a4870 2024-06-11T13:24:38.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:38.146+0000 7f4931ffb640 1 -- 172.21.15.57:0/1929702754 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f492c001810 con 0x7f49341a4870 2024-06-11T13:24:38.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:38.146+0000 7f4931ffb640 1 -- 172.21.15.57:0/1929702754 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3150323694 0 0) 0x7f49341aa9e0 con 0x7f49341000f0 2024-06-11T13:24:38.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:38.146+0000 7f4931ffb640 1 -- 172.21.15.57:0/1929702754 >> v1:172.21.15.57:6790/0 conn(0x7f49341a4870 legacy=0x7f49341a6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:24:38.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:38.146+0000 7f4931ffb640 1 -- 172.21.15.57:0/1929702754 >> v1:172.21.15.107:6789/0 conn(0x7f4934198cf0 legacy=0x7f49341a3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:24:38.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:38.146+0000 7f4931ffb640 1 -- 172.21.15.57:0/1929702754 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49341abc20 con 0x7f49341000f0 2024-06-11T13:24:38.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:38.146+0000 7f493ac64640 1 -- 172.21.15.57:0/1929702754 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f49341aac10 con 0x7f49341000f0 2024-06-11T13:24:38.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:38.146+0000 7f4931ffb640 1 -- 172.21.15.57:0/1929702754 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f4924001fb0 con 0x7f49341000f0 2024-06-11T13:24:38.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:38.146+0000 7f4931ffb640 1 -- 172.21.15.57:0/1929702754 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f49240053d0 con 0x7f49341000f0 2024-06-11T13:24:38.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:38.146+0000 7f493ac64640 1 -- 172.21.15.57:0/1929702754 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f49341ab0d0 con 0x7f49341000f0 2024-06-11T13:24:38.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:38.147+0000 7f4931ffb640 1 -- 172.21.15.57:0/1929702754 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 13) ==== 46308+0+0 (unknown 2805797583 0 0) 0x7f4924005590 con 0x7f49341000f0 2024-06-11T13:24:38.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:38.147+0000 7f4931ffb640 1 -- 172.21.15.57:0/1929702754 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(4..4 src has 1..4) ==== 1069+0+0 (unknown 1503136288 0 0) 0x7f4924047bf0 con 0x7f49341000f0 2024-06-11T13:24:38.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:38.147+0000 7f493ac64640 1 -- 172.21.15.57:0/1929702754 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4934063000 con 0x7f49341000f0 2024-06-11T13:24:38.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:38.152+0000 7f4931ffb640 1 -- 172.21.15.57:0/1929702754 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f4924017800 con 0x7f49341000f0 2024-06-11T13:24:38.244 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:38 smithi057 ceph-mon[29805]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:38.244 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:38 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:38.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:38 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:38.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:38 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:24:38.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:38 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:24:38.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:38 smithi057 ceph-mon[29805]: Updating smithi057:/etc/ceph/ceph.conf 2024-06-11T13:24:38.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:38 smithi057 ceph-mon[29805]: Updating smithi107:/etc/ceph/ceph.conf 2024-06-11T13:24:38.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:38 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:38.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:38 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:38.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:38 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:38.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:38 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:38.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:38 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:38.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:38 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:38.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:38 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:38.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:38 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:38.245 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:38 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:38.246 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:38 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-11T13:24:38.246 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:38 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-11T13:24:38.246 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:38 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:24:38.246 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:38 smithi057 ceph-mon[40588]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:38.246 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:38 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:38.246 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:38 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:38.246 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:38 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:24:38.246 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:38 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:24:38.246 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:38 smithi057 ceph-mon[40588]: Updating smithi057:/etc/ceph/ceph.conf 2024-06-11T13:24:38.246 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:38 smithi057 ceph-mon[40588]: Updating smithi107:/etc/ceph/ceph.conf 2024-06-11T13:24:38.246 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:38 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:38.246 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:38 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:38.247 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:38 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:38.247 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:38 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:38.247 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:38 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:38.247 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:38 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:38.247 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:38 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:38.247 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:38 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:38.247 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:38 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:38.247 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:38 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-11T13:24:38.247 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:38 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-11T13:24:38.247 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:38 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:24:38.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:38 smithi107 ceph-mon[33178]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:38.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:38 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:38.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:38 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:38.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:38 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:24:38.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:38 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:24:38.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:38 smithi107 ceph-mon[33178]: Updating smithi057:/etc/ceph/ceph.conf 2024-06-11T13:24:38.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:38 smithi107 ceph-mon[33178]: Updating smithi107:/etc/ceph/ceph.conf 2024-06-11T13:24:38.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:38 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:38.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:38 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:38.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:38 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:38.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:38 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:38.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:38 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:38.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:38 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:38.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:38 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:38.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:38 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:38.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:38 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:38.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:38 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-11T13:24:38.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:38 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-11T13:24:38.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:38 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:24:38.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:38.341+0000 7f493ac64640 1 -- 172.21.15.57:0/1929702754 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f4934199a60 con 0x7f49341000f0 2024-06-11T13:24:38.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:38.341+0000 7f4931ffb640 1 -- 172.21.15.57:0/1929702754 <== mon.0 v1:172.21.15.57:6789/0 10 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+191 (unknown 2189217731 0 2943132970) 0x7f4924017100 con 0x7f49341000f0 2024-06-11T13:24:38.336 INFO:teuthology.orchestra.run.smithi057.stdout:# minimal ceph.conf for 98a6635e-27f5-11ef-bc9b-c7b262605968 2024-06-11T13:24:38.336 INFO:teuthology.orchestra.run.smithi057.stdout:[global] 2024-06-11T13:24:38.337 INFO:teuthology.orchestra.run.smithi057.stdout: fsid = 98a6635e-27f5-11ef-bc9b-c7b262605968 2024-06-11T13:24:38.337 INFO:teuthology.orchestra.run.smithi057.stdout: mon_host = 172.21.15.57:6789/0 172.21.15.107:6789/0 v1:172.21.15.57:6790/0 2024-06-11T13:24:38.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:38.343+0000 7f493ac64640 1 -- 172.21.15.57:0/1929702754 >> v1:172.21.15.57:6800/2989665238 conn(0x7f492003a3d0 legacy=0x7f492003c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:24:38.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:38.343+0000 7f493ac64640 1 -- 172.21.15.57:0/1929702754 >> v1:172.21.15.57:6789/0 conn(0x7f49341000f0 legacy=0x7f49341985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:24:38.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:38.343+0000 7f493ac64640 1 -- 172.21.15.57:0/1929702754 shutdown_connections 2024-06-11T13:24:38.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:38.344+0000 7f493ac64640 1 -- 172.21.15.57:0/1929702754 >> 172.21.15.57:0/1929702754 conn(0x7f49340fbdf0 msgr2=0x7f49340fc660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:24:38.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:38.344+0000 7f493ac64640 1 -- 172.21.15.57:0/1929702754 shutdown_connections 2024-06-11T13:24:38.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:38.344+0000 7f493ac64640 1 -- 172.21.15.57:0/1929702754 wait complete. 2024-06-11T13:24:38.901 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-06-11T13:24:38.901 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-06-11T13:24:38.901 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/etc/ceph/ceph.conf 2024-06-11T13:24:38.931 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-06-11T13:24:38.931 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-11T13:24:38.999 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-06-11T13:24:38.999 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.conf 2024-06-11T13:24:39.031 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-06-11T13:24:39.032 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-11T13:24:39.100 INFO:tasks.cephadm:Adding mgr.y on smithi057 2024-06-11T13:24:39.101 INFO:tasks.cephadm:Adding mgr.x on smithi107 2024-06-11T13:24:39.101 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph orch apply mgr '2;smithi057=y;smithi107=x' 2024-06-11T13:24:39.181 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:39 smithi057 ceph-mon[29805]: Updating smithi057:/var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-11T13:24:39.181 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:39 smithi057 ceph-mon[29805]: Updating smithi107:/var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-11T13:24:39.181 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:39 smithi057 ceph-mon[29805]: Reconfiguring mon.a (unknown last config time)... 2024-06-11T13:24:39.181 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:39 smithi057 ceph-mon[29805]: Reconfiguring daemon mon.a on smithi057 2024-06-11T13:24:39.181 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:39 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1929702754' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:24:39.181 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:39 smithi057 ceph-mon[40588]: Updating smithi057:/var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-11T13:24:39.181 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:39 smithi057 ceph-mon[40588]: Updating smithi107:/var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-11T13:24:39.181 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:39 smithi057 ceph-mon[40588]: Reconfiguring mon.a (unknown last config time)... 2024-06-11T13:24:39.182 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:39 smithi057 ceph-mon[40588]: Reconfiguring daemon mon.a on smithi057 2024-06-11T13:24:39.182 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:39 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1929702754' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:24:39.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:39 smithi107 ceph-mon[33178]: Updating smithi057:/var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-11T13:24:39.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:39 smithi107 ceph-mon[33178]: Updating smithi107:/var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-11T13:24:39.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:39 smithi107 ceph-mon[33178]: Reconfiguring mon.a (unknown last config time)... 2024-06-11T13:24:39.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:39 smithi107 ceph-mon[33178]: Reconfiguring daemon mon.a on smithi057 2024-06-11T13:24:39.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:39 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1929702754' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:24:39.373 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.b/config 2024-06-11T13:24:40.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:40 smithi107 ceph-mon[33178]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:40.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:40 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:40.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:40 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:40.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:40 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-11T13:24:40.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:40 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-11T13:24:40.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:40 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:24:40.315 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:40 smithi057 ceph-mon[29805]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:40.315 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:40 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:40.315 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:40 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:40.315 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:40 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-11T13:24:40.315 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:40 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-11T13:24:40.315 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:40 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:24:40.315 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:40 smithi057 ceph-mon[40588]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:40.316 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:40 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:40.316 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:40 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:40.316 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:40 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-11T13:24:40.316 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:40 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-11T13:24:40.316 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:40 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:24:40.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:40.852+0000 7f3541973640 1 -- 172.21.15.107:0/455932189 >> v1:172.21.15.57:6789/0 conn(0x7f353c102080 legacy=0x7f353c104470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:24:40.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:40.852+0000 7f3541973640 1 -- 172.21.15.107:0/455932189 shutdown_connections 2024-06-11T13:24:40.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:40.852+0000 7f3541973640 1 -- 172.21.15.107:0/455932189 >> 172.21.15.107:0/455932189 conn(0x7f353c0fba10 msgr2=0x7f353c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:24:40.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:40.852+0000 7f3541973640 1 -- 172.21.15.107:0/455932189 shutdown_connections 2024-06-11T13:24:40.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:40.852+0000 7f3541973640 1 -- 172.21.15.107:0/455932189 wait complete. 2024-06-11T13:24:40.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:40.853+0000 7f3541973640 1 Processor -- start 2024-06-11T13:24:40.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:40.853+0000 7f3541973640 1 -- start start 2024-06-11T13:24:40.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:40.853+0000 7f3541973640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f353c1a1770 con 0x7f353c196610 2024-06-11T13:24:40.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:40.853+0000 7f3541973640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f353c1a29b0 con 0x7f353c102080 2024-06-11T13:24:40.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:40.853+0000 7f3541973640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f353c1a3bf0 con 0x7f353c19db00 2024-06-11T13:24:40.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:40.853+0000 7f353affd640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f353c102080 0x7f353c195f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.107:58680/0 (socket says 172.21.15.107:58680) 2024-06-11T13:24:40.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:40.853+0000 7f353affd640 1 -- 172.21.15.107:0/3844865838 learned_addr learned my addr 172.21.15.107:0/3844865838 (peer_addr_for_me v1:172.21.15.107:0/0) 2024-06-11T13:24:40.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:40.853+0000 7f3540971640 1 -- 172.21.15.107:0/3844865838 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 288764589 0 0) 0x7f353c1a29b0 con 0x7f353c102080 2024-06-11T13:24:40.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:40.854+0000 7f3540971640 1 -- 172.21.15.107:0/3844865838 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3528003b60 con 0x7f353c102080 2024-06-11T13:24:40.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:40.854+0000 7f3540971640 1 -- 172.21.15.107:0/3844865838 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4169675118 0 0) 0x7f353c1a1770 con 0x7f353c196610 2024-06-11T13:24:40.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:40.854+0000 7f3540971640 1 -- 172.21.15.107:0/3844865838 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f353c1a29b0 con 0x7f353c196610 2024-06-11T13:24:40.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:40.854+0000 7f3540971640 1 -- 172.21.15.107:0/3844865838 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2151858851 0 0) 0x7f353c1a3bf0 con 0x7f353c19db00 2024-06-11T13:24:40.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:40.854+0000 7f3540971640 1 -- 172.21.15.107:0/3844865838 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f353c1a1770 con 0x7f353c19db00 2024-06-11T13:24:40.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:40.854+0000 7f3540971640 1 -- 172.21.15.107:0/3844865838 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 98864242 0 0) 0x7f3528003b60 con 0x7f353c102080 2024-06-11T13:24:40.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:40.854+0000 7f3540971640 1 -- 172.21.15.107:0/3844865838 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f353c1a3bf0 con 0x7f353c102080 2024-06-11T13:24:40.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:40.854+0000 7f3540971640 1 -- 172.21.15.107:0/3844865838 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f352c003880 con 0x7f353c102080 2024-06-11T13:24:40.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:40.854+0000 7f3540971640 1 -- 172.21.15.107:0/3844865838 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1966645409 0 0) 0x7f353c1a29b0 con 0x7f353c196610 2024-06-11T13:24:40.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:40.854+0000 7f3540971640 1 -- 172.21.15.107:0/3844865838 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f3528003b60 con 0x7f353c196610 2024-06-11T13:24:40.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:40.854+0000 7f3540971640 1 -- 172.21.15.107:0/3844865838 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f3524001810 con 0x7f353c196610 2024-06-11T13:24:40.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:40.854+0000 7f3540971640 1 -- 172.21.15.107:0/3844865838 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3168928111 0 0) 0x7f353c1a1770 con 0x7f353c19db00 2024-06-11T13:24:40.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:40.854+0000 7f3540971640 1 -- 172.21.15.107:0/3844865838 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f353c1a29b0 con 0x7f353c19db00 2024-06-11T13:24:40.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:40.854+0000 7f3540971640 1 -- 172.21.15.107:0/3844865838 <== mon.1 v1:172.21.15.107:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2090725205 0 0) 0x7f353c1a3bf0 con 0x7f353c102080 2024-06-11T13:24:40.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:40.854+0000 7f3540971640 1 -- 172.21.15.107:0/3844865838 >> v1:172.21.15.57:6790/0 conn(0x7f353c19db00 legacy=0x7f353c19fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:24:40.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:40.854+0000 7f3540971640 1 -- 172.21.15.107:0/3844865838 >> v1:172.21.15.57:6789/0 conn(0x7f353c196610 legacy=0x7f353c19c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:24:40.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:40.854+0000 7f3540971640 1 -- 172.21.15.107:0/3844865838 --> v1:172.21.15.107:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f353c1a4e30 con 0x7f353c102080 2024-06-11T13:24:40.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:40.854+0000 7f3541973640 1 -- 172.21.15.107:0/3844865838 --> v1:172.21.15.107:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f353c1a19a0 con 0x7f353c102080 2024-06-11T13:24:40.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:40.854+0000 7f3541973640 1 -- 172.21.15.107:0/3844865838 --> v1:172.21.15.107:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f353c1a1ee0 con 0x7f353c102080 2024-06-11T13:24:40.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:40.854+0000 7f3540971640 1 -- 172.21.15.107:0/3844865838 <== mon.1 v1:172.21.15.107:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f352c003b80 con 0x7f353c102080 2024-06-11T13:24:40.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:40.854+0000 7f3540971640 1 -- 172.21.15.107:0/3844865838 <== mon.1 v1:172.21.15.107:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f352c004fe0 con 0x7f353c102080 2024-06-11T13:24:40.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:40.855+0000 7f3540971640 1 -- 172.21.15.107:0/3844865838 <== mon.1 v1:172.21.15.107:6789/0 7 ==== mgrmap(e 13) ==== 46308+0+0 (unknown 2805797583 0 0) 0x7f352c005180 con 0x7f353c102080 2024-06-11T13:24:40.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:40.855+0000 7f3541973640 1 -- 172.21.15.107:0/3844865838 --> v1:172.21.15.107:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f351c0050b0 con 0x7f353c102080 2024-06-11T13:24:40.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:40.856+0000 7f3540971640 1 -- 172.21.15.107:0/3844865838 <== mon.1 v1:172.21.15.107:6789/0 8 ==== osd_map(4..4 src has 1..4) ==== 1069+0+0 (unknown 1503136288 0 0) 0x7f352c047870 con 0x7f353c102080 2024-06-11T13:24:40.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:40.860+0000 7f3540971640 1 -- 172.21.15.107:0/3844865838 <== mon.1 v1:172.21.15.107:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f352c0175a0 con 0x7f353c102080 2024-06-11T13:24:41.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:41.045+0000 7f3541973640 1 -- 172.21.15.107:0/3844865838 --> v1:172.21.15.57:6800/2989665238 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi057=y;smithi107=x", "target": ["mon-mgr", ""]}) -- 0x7f351c000bc0 con 0x7f352803a410 2024-06-11T13:24:41.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:41.056+0000 7f3540971640 1 -- 172.21.15.107:0/3844865838 <== mgr.14150 v1:172.21.15.57:6800/2989665238 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (unknown 0 0 325935098) 0x7f351c000bc0 con 0x7f352803a410 2024-06-11T13:24:41.051 INFO:teuthology.orchestra.run.smithi107.stdout:Scheduled mgr update... 2024-06-11T13:24:41.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:41.059+0000 7f3541973640 1 -- 172.21.15.107:0/3844865838 >> v1:172.21.15.57:6800/2989665238 conn(0x7f352803a410 legacy=0x7f352803c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:24:41.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:41.059+0000 7f3541973640 1 -- 172.21.15.107:0/3844865838 >> v1:172.21.15.107:6789/0 conn(0x7f353c102080 legacy=0x7f353c195f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:24:41.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:41.059+0000 7f3541973640 1 -- 172.21.15.107:0/3844865838 shutdown_connections 2024-06-11T13:24:41.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:41.059+0000 7f3541973640 1 -- 172.21.15.107:0/3844865838 >> 172.21.15.107:0/3844865838 conn(0x7f353c0fba10 msgr2=0x7f353c1046b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:24:41.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:41.059+0000 7f3541973640 1 -- 172.21.15.107:0/3844865838 shutdown_connections 2024-06-11T13:24:41.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:24:41.059+0000 7f3541973640 1 -- 172.21.15.107:0/3844865838 wait complete. 2024-06-11T13:24:41.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:41 smithi057 ceph-mon[29805]: Reconfiguring mon.c (monmap changed)... 2024-06-11T13:24:41.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:41 smithi057 ceph-mon[29805]: Reconfiguring daemon mon.c on smithi057 2024-06-11T13:24:41.282 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:41 smithi057 ceph-mon[40588]: Reconfiguring mon.c (monmap changed)... 2024-06-11T13:24:41.282 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:41 smithi057 ceph-mon[40588]: Reconfiguring daemon mon.c on smithi057 2024-06-11T13:24:41.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:41 smithi107 ceph-mon[33178]: Reconfiguring mon.c (monmap changed)... 2024-06-11T13:24:41.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:41 smithi107 ceph-mon[33178]: Reconfiguring daemon mon.c on smithi057 2024-06-11T13:24:41.624 DEBUG:teuthology.orchestra.run.smithi107:mgr.x> sudo journalctl -f -n 0 -u ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@mgr.x.service 2024-06-11T13:24:41.626 INFO:tasks.cephadm:Deploying OSDs... 2024-06-11T13:24:41.626 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-06-11T13:24:41.626 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/scratch_devs of=/dev/stdout 2024-06-11T13:24:41.645 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-11T13:24:41.645 DEBUG:teuthology.orchestra.run.smithi057:> stat /dev/vg_nvme/lv_1 2024-06-11T13:24:41.705 INFO:teuthology.orchestra.run.smithi057.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-11T13:24:41.705 INFO:teuthology.orchestra.run.smithi057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-11T13:24:41.705 INFO:teuthology.orchestra.run.smithi057.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-06-11T13:24:41.705 INFO:teuthology.orchestra.run.smithi057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-11T13:24:41.705 INFO:teuthology.orchestra.run.smithi057.stdout:Context: system_u:object_r:device_t:s0 2024-06-11T13:24:41.705 INFO:teuthology.orchestra.run.smithi057.stdout:Access: 2024-06-11 13:24:29.817337585 +0000 2024-06-11T13:24:41.705 INFO:teuthology.orchestra.run.smithi057.stdout:Modify: 2024-06-11 13:18:51.956250195 +0000 2024-06-11T13:24:41.705 INFO:teuthology.orchestra.run.smithi057.stdout:Change: 2024-06-11 13:18:51.956250195 +0000 2024-06-11T13:24:41.705 INFO:teuthology.orchestra.run.smithi057.stdout: Birth: 2024-06-11 13:18:51.956250195 +0000 2024-06-11T13:24:41.706 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-11T13:24:41.773 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records in 2024-06-11T13:24:41.773 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records out 2024-06-11T13:24:41.774 INFO:teuthology.orchestra.run.smithi057.stderr:512 bytes copied, 0.000279555 s, 1.8 MB/s 2024-06-11T13:24:41.775 DEBUG:teuthology.orchestra.run.smithi057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-11T13:24:41.833 DEBUG:teuthology.orchestra.run.smithi057:> stat /dev/vg_nvme/lv_2 2024-06-11T13:24:41.891 INFO:teuthology.orchestra.run.smithi057.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-11T13:24:41.891 INFO:teuthology.orchestra.run.smithi057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-11T13:24:41.891 INFO:teuthology.orchestra.run.smithi057.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-06-11T13:24:41.891 INFO:teuthology.orchestra.run.smithi057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-11T13:24:41.891 INFO:teuthology.orchestra.run.smithi057.stdout:Context: system_u:object_r:device_t:s0 2024-06-11T13:24:41.891 INFO:teuthology.orchestra.run.smithi057.stdout:Access: 2024-06-11 13:24:29.817337585 +0000 2024-06-11T13:24:41.891 INFO:teuthology.orchestra.run.smithi057.stdout:Modify: 2024-06-11 13:18:52.310245100 +0000 2024-06-11T13:24:41.892 INFO:teuthology.orchestra.run.smithi057.stdout:Change: 2024-06-11 13:18:52.310245100 +0000 2024-06-11T13:24:41.892 INFO:teuthology.orchestra.run.smithi057.stdout: Birth: 2024-06-11 13:18:52.310245100 +0000 2024-06-11T13:24:41.892 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-11T13:24:41.958 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records in 2024-06-11T13:24:41.958 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records out 2024-06-11T13:24:41.958 INFO:teuthology.orchestra.run.smithi057.stderr:512 bytes copied, 0.000300911 s, 1.7 MB/s 2024-06-11T13:24:41.959 DEBUG:teuthology.orchestra.run.smithi057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-11T13:24:42.018 DEBUG:teuthology.orchestra.run.smithi057:> stat /dev/vg_nvme/lv_3 2024-06-11T13:24:42.075 INFO:teuthology.orchestra.run.smithi057.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-11T13:24:42.076 INFO:teuthology.orchestra.run.smithi057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-11T13:24:42.076 INFO:teuthology.orchestra.run.smithi057.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-06-11T13:24:42.076 INFO:teuthology.orchestra.run.smithi057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-11T13:24:42.076 INFO:teuthology.orchestra.run.smithi057.stdout:Context: system_u:object_r:device_t:s0 2024-06-11T13:24:42.076 INFO:teuthology.orchestra.run.smithi057.stdout:Access: 2024-06-11 13:24:29.818337571 +0000 2024-06-11T13:24:42.076 INFO:teuthology.orchestra.run.smithi057.stdout:Modify: 2024-06-11 13:18:52.644240293 +0000 2024-06-11T13:24:42.076 INFO:teuthology.orchestra.run.smithi057.stdout:Change: 2024-06-11 13:18:52.644240293 +0000 2024-06-11T13:24:42.076 INFO:teuthology.orchestra.run.smithi057.stdout: Birth: 2024-06-11 13:18:52.644240293 +0000 2024-06-11T13:24:42.076 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-11T13:24:42.141 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records in 2024-06-11T13:24:42.141 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records out 2024-06-11T13:24:42.141 INFO:teuthology.orchestra.run.smithi057.stderr:512 bytes copied, 0.000279173 s, 1.8 MB/s 2024-06-11T13:24:42.142 DEBUG:teuthology.orchestra.run.smithi057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-11T13:24:42.199 DEBUG:teuthology.orchestra.run.smithi057:> stat /dev/vg_nvme/lv_4 2024-06-11T13:24:42.255 INFO:teuthology.orchestra.run.smithi057.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-11T13:24:42.255 INFO:teuthology.orchestra.run.smithi057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-11T13:24:42.255 INFO:teuthology.orchestra.run.smithi057.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-06-11T13:24:42.256 INFO:teuthology.orchestra.run.smithi057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-11T13:24:42.256 INFO:teuthology.orchestra.run.smithi057.stdout:Context: system_u:object_r:device_t:s0 2024-06-11T13:24:42.256 INFO:teuthology.orchestra.run.smithi057.stdout:Access: 2024-06-11 13:24:29.818337571 +0000 2024-06-11T13:24:42.256 INFO:teuthology.orchestra.run.smithi057.stdout:Modify: 2024-06-11 13:18:52.993235270 +0000 2024-06-11T13:24:42.256 INFO:teuthology.orchestra.run.smithi057.stdout:Change: 2024-06-11 13:18:52.993235270 +0000 2024-06-11T13:24:42.256 INFO:teuthology.orchestra.run.smithi057.stdout: Birth: 2024-06-11 13:18:52.993235270 +0000 2024-06-11T13:24:42.256 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-11T13:24:42.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:42 smithi107 ceph-mon[33178]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:42.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:42 smithi107 ceph-mon[33178]: from='client.24103 v1:172.21.15.107:0/3844865838' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi057=y;smithi107=x", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:24:42.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:42 smithi107 ceph-mon[33178]: Saving service mgr spec with placement smithi057=y;smithi107=x;count:2 2024-06-11T13:24:42.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:42 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:42.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:42 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:42.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:42 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:42.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:42 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-11T13:24:42.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:42 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-11T13:24:42.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:42 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:24:42.310 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:42 smithi057 ceph-mon[40588]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:42.310 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:42 smithi057 ceph-mon[40588]: from='client.24103 v1:172.21.15.107:0/3844865838' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi057=y;smithi107=x", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:24:42.310 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:42 smithi057 ceph-mon[40588]: Saving service mgr spec with placement smithi057=y;smithi107=x;count:2 2024-06-11T13:24:42.310 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:42 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:42.310 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:42 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:42.310 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:42 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:42.311 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:42 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-11T13:24:42.311 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:42 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-11T13:24:42.311 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:42 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:24:42.311 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:42 smithi057 ceph-mon[29805]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:42.311 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:42 smithi057 ceph-mon[29805]: from='client.24103 v1:172.21.15.107:0/3844865838' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi057=y;smithi107=x", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:24:42.311 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:42 smithi057 ceph-mon[29805]: Saving service mgr spec with placement smithi057=y;smithi107=x;count:2 2024-06-11T13:24:42.311 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:42 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:42.311 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:42 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:42.311 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:42 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:42.311 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:42 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-11T13:24:42.311 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:42 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-11T13:24:42.311 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:42 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:24:42.320 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records in 2024-06-11T13:24:42.320 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records out 2024-06-11T13:24:42.320 INFO:teuthology.orchestra.run.smithi057.stderr:512 bytes copied, 0.000290333 s, 1.8 MB/s 2024-06-11T13:24:42.322 DEBUG:teuthology.orchestra.run.smithi057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-11T13:24:42.380 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-06-11T13:24:42.380 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/scratch_devs of=/dev/stdout 2024-06-11T13:24:42.397 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-11T13:24:42.398 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_1 2024-06-11T13:24:42.454 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-11T13:24:42.455 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-11T13:24:42.455 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-06-11T13:24:42.455 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-11T13:24:42.455 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-06-11T13:24:42.455 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-06-11 13:24:20.831738473 +0000 2024-06-11T13:24:42.455 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-06-11 13:18:44.411907636 +0000 2024-06-11T13:24:42.455 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-06-11 13:18:44.411907636 +0000 2024-06-11T13:24:42.455 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-06-11 13:18:44.411907636 +0000 2024-06-11T13:24:42.455 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-11T13:24:42.519 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-06-11T13:24:42.519 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-06-11T13:24:42.519 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000194067 s, 2.6 MB/s 2024-06-11T13:24:42.520 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-11T13:24:42.577 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_2 2024-06-11T13:24:42.632 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-11T13:24:42.632 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-11T13:24:42.632 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-06-11T13:24:42.632 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-11T13:24:42.632 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-06-11T13:24:42.633 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-06-11 13:24:20.831738473 +0000 2024-06-11T13:24:42.633 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-06-11 13:18:44.750901895 +0000 2024-06-11T13:24:42.633 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-06-11 13:18:44.750901895 +0000 2024-06-11T13:24:42.633 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-06-11 13:18:44.750901895 +0000 2024-06-11T13:24:42.633 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-11T13:24:42.695 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-06-11T13:24:42.695 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-06-11T13:24:42.695 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000178781 s, 2.9 MB/s 2024-06-11T13:24:42.696 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-11T13:24:42.752 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_3 2024-06-11T13:24:42.809 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-11T13:24:42.809 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-11T13:24:42.810 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-06-11T13:24:42.810 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-11T13:24:42.810 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-06-11T13:24:42.810 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-06-11 13:24:20.832738457 +0000 2024-06-11T13:24:42.810 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-06-11 13:18:45.093896087 +0000 2024-06-11T13:24:42.810 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-06-11 13:18:45.093896087 +0000 2024-06-11T13:24:42.810 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-06-11 13:18:45.093896087 +0000 2024-06-11T13:24:42.810 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-11T13:24:42.874 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-06-11T13:24:42.874 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-06-11T13:24:42.874 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000215424 s, 2.4 MB/s 2024-06-11T13:24:42.875 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-11T13:24:42.931 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_4 2024-06-11T13:24:42.987 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-11T13:24:42.987 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-11T13:24:42.987 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-06-11T13:24:42.987 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-11T13:24:42.987 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-06-11T13:24:42.987 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-06-11 13:24:20.832738457 +0000 2024-06-11T13:24:42.987 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-06-11 13:18:45.438890244 +0000 2024-06-11T13:24:42.988 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-06-11 13:18:45.438890244 +0000 2024-06-11T13:24:42.988 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-06-11 13:18:45.438890244 +0000 2024-06-11T13:24:42.988 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-11T13:24:43.051 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-06-11T13:24:43.051 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-06-11T13:24:43.051 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000170416 s, 3.0 MB/s 2024-06-11T13:24:43.052 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-11T13:24:43.108 INFO:tasks.cephadm:Deploying osd.0 on smithi057 with /dev/vg_nvme/lv_4... 2024-06-11T13:24:43.108 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-11T13:24:43.203 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:43 smithi107 ceph-mon[33178]: Reconfiguring mon.b (monmap changed)... 2024-06-11T13:24:43.204 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:43 smithi107 ceph-mon[33178]: Reconfiguring daemon mon.b on smithi107 2024-06-11T13:24:43.204 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:43 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:43.204 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:43 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:43.204 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:43 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:24:43.204 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:43 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:24:43.204 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:43 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:24:43.204 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:43 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:43.204 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:43 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-11T13:24:43.204 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:43 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-11T13:24:43.204 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:43 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-11T13:24:43.204 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:43 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:24:43.313 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:43 smithi057 ceph-mon[40588]: Reconfiguring mon.b (monmap changed)... 2024-06-11T13:24:43.313 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:43 smithi057 ceph-mon[40588]: Reconfiguring daemon mon.b on smithi107 2024-06-11T13:24:43.313 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:43 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:43.313 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:43 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:43.313 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:43 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:24:43.313 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:43 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:24:43.313 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:43 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:24:43.313 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:43 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:43.313 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:43 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-11T13:24:43.313 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:43 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-11T13:24:43.313 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:43 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-11T13:24:43.314 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:43 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:24:43.314 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:43 smithi057 ceph-mon[29805]: Reconfiguring mon.b (monmap changed)... 2024-06-11T13:24:43.314 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:43 smithi057 ceph-mon[29805]: Reconfiguring daemon mon.b on smithi107 2024-06-11T13:24:43.314 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:43 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:43.314 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:43 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:43.314 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:43 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:24:43.314 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:43 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:24:43.314 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:43 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:24:43.314 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:43 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:43.314 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:43 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-11T13:24:43.314 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:43 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-11T13:24:43.315 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:43 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-11T13:24:43.315 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:43 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:24:43.354 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.a/config 2024-06-11T13:24:44.169 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:44 smithi107 ceph-mon[33178]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:44.169 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:44 smithi107 ceph-mon[33178]: Deploying daemon mgr.x on smithi107 2024-06-11T13:24:44.338 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:44 smithi057 ceph-mon[29805]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:44.338 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:44 smithi057 ceph-mon[29805]: Deploying daemon mgr.x on smithi107 2024-06-11T13:24:44.338 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:44 smithi057 ceph-mon[40588]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:44.338 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:44 smithi057 ceph-mon[40588]: Deploying daemon mgr.x on smithi107 2024-06-11T13:24:44.795 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:24:44 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:24:44.744+0000 7f73ab6a4040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-11T13:24:45.100 INFO:teuthology.orchestra.run.smithi057.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-11T13:24:45.200 INFO:teuthology.orchestra.run.smithi057.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-11T13:24:45.213 INFO:teuthology.orchestra.run.smithi057.stderr: stderr: 10+0 records in 2024-06-11T13:24:45.213 INFO:teuthology.orchestra.run.smithi057.stderr:10+0 records out 2024-06-11T13:24:45.213 INFO:teuthology.orchestra.run.smithi057.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113856 s, 921 MB/s 2024-06-11T13:24:45.213 INFO:teuthology.orchestra.run.smithi057.stderr:--> Zapping successful for: 2024-06-11T13:24:45.262 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:24:44 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:24:44.909+0000 7f73ab6a4040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-11T13:24:45.262 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:24:44 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:24:44.983+0000 7f73ab6a4040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-11T13:24:45.263 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:24:45 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:24:45.267+0000 7f73ab6a4040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-11T13:24:45.545 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:24:45 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:24:45.328+0000 7f73ab6a4040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-11T13:24:45.749 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi057:vg_nvme/lv_4 2024-06-11T13:24:45.999 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.a/config 2024-06-11T13:24:46.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:45 smithi057 ceph-mon[29805]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:46.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:45 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:46.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:45 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:46.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:45 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:46.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:45 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:46.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:45 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:24:46.032 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:45 smithi057 ceph-mon[40588]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:46.032 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:45 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:46.032 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:45 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:46.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:45 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:46.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:45 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:46.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:45 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:24:46.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:45 smithi107 ceph-mon[33178]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:46.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:45 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:46.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:45 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:46.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:45 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:46.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:45 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:46.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:45 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:24:46.680 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:24:46 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:24:46.301+0000 7f73ab6a4040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-11T13:24:46.680 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:24:46 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:24:46.415+0000 7f73ab6a4040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-11T13:24:46.680 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:24:46 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:24:46.473+0000 7f73ab6a4040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-11T13:24:46.680 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:24:46 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:24:46.685+0000 7f73ab6a4040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-11T13:24:46.956 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:24:46 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:24:46.746+0000 7f73ab6a4040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-11T13:24:46.956 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:24:46 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:24:46.813+0000 7f73ab6a4040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-11T13:24:46.956 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:24:46 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:24:46.961+0000 7f73ab6a4040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-11T13:24:46.956 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:46 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:46.956 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:46 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:46.957 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:46 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:24:46.957 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:46 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:24:46.957 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:46 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:46.957 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:46 smithi107 ceph-mon[33178]: Reconfiguring mgr.y (unknown last config time)... 2024-06-11T13:24:46.957 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:46 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-11T13:24:46.957 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:46 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-11T13:24:46.958 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:46 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:24:46.958 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:46 smithi107 ceph-mon[33178]: Reconfiguring daemon mgr.y on smithi057 2024-06-11T13:24:46.958 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:46 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:47.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:46 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:47.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:46 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:47.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:46 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:24:47.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:46 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:24:47.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:46 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:47.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:46 smithi057 ceph-mon[29805]: Reconfiguring mgr.y (unknown last config time)... 2024-06-11T13:24:47.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:46 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-11T13:24:47.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:46 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-11T13:24:47.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:46 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:24:47.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:46 smithi057 ceph-mon[29805]: Reconfiguring daemon mgr.y on smithi057 2024-06-11T13:24:47.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:46 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:47.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:46 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:47.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:46 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:47.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:46 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:24:47.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:46 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:24:47.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:46 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:47.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:46 smithi057 ceph-mon[40588]: Reconfiguring mgr.y (unknown last config time)... 2024-06-11T13:24:47.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:46 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-11T13:24:47.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:46 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-11T13:24:47.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:46 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:24:47.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:46 smithi057 ceph-mon[40588]: Reconfiguring daemon mgr.y on smithi057 2024-06-11T13:24:47.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:46 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:47.694 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:24:47 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:24:47.380+0000 7f73ab6a4040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-11T13:24:47.694 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:24:47 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:24:47.699+0000 7f73ab6a4040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-11T13:24:48.045 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:24:47 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:24:47.768+0000 7f73ab6a4040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-11T13:24:48.045 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:24:47 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:24:47.923+0000 7f73ab6a4040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-11T13:24:48.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:47 smithi107 ceph-mon[33178]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:48.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:47 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:48.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:47 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:48.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:47 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:24:48.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:47 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:24:48.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:47 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:24:48.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:47 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:48.199 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:47 smithi057 ceph-mon[29805]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:48.199 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:47 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:48.199 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:47 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:48.199 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:47 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:24:48.199 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:47 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:24:48.199 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:47 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:24:48.199 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:47 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:48.200 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:47 smithi057 ceph-mon[40588]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:48.200 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:47 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:48.200 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:47 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:48.200 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:47 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:24:48.200 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:47 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:24:48.200 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:47 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:24:48.200 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:47 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:24:48.482 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:24:48 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:24:48.236+0000 7f73ab6a4040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-11T13:24:48.482 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:24:48 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:24:48.299+0000 7f73ab6a4040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-11T13:24:48.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:48.509+0000 7f29e5a33640 1 -- 172.21.15.57:0/1451039299 >> v1:172.21.15.57:6789/0 conn(0x7f29e01088e0 legacy=0x7f29e010ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:24:48.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:48.509+0000 7f29e5a33640 1 -- 172.21.15.57:0/1451039299 shutdown_connections 2024-06-11T13:24:48.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:48.509+0000 7f29e5a33640 1 -- 172.21.15.57:0/1451039299 >> 172.21.15.57:0/1451039299 conn(0x7f29e00fbd10 msgr2=0x7f29e00fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:24:48.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:48.509+0000 7f29e5a33640 1 -- 172.21.15.57:0/1451039299 shutdown_connections 2024-06-11T13:24:48.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:48.509+0000 7f29e5a33640 1 -- 172.21.15.57:0/1451039299 wait complete. 2024-06-11T13:24:48.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:48.510+0000 7f29e5a33640 1 Processor -- start 2024-06-11T13:24:48.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:48.510+0000 7f29e5a33640 1 -- start start 2024-06-11T13:24:48.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:48.510+0000 7f29e5a33640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f29e0197790 con 0x7f29e01088e0 2024-06-11T13:24:48.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:48.510+0000 7f29e5a33640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f29e01a2fe0 con 0x7f29e0100770 2024-06-11T13:24:48.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:48.510+0000 7f29e5a33640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f29e01a41c0 con 0x7f29e0104ba0 2024-06-11T13:24:48.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:48.510+0000 7f29de7fc640 1 --1- >> v1:172.21.15.57:6790/0 conn(0x7f29e0104ba0 0x7f29e019d180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6790/0 says I am v1:172.21.15.57:48220/0 (socket says 172.21.15.57:48220) 2024-06-11T13:24:48.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:48.510+0000 7f29de7fc640 1 -- 172.21.15.57:0/2516750687 learned_addr learned my addr 172.21.15.57:0/2516750687 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:24:48.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:48.510+0000 7f29df7fe640 1 --1- 172.21.15.57:0/2516750687 >> v1:172.21.15.57:6789/0 conn(0x7f29e01088e0 0x7f29e01a08b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:34496/0 (socket says 172.21.15.57:34496) 2024-06-11T13:24:48.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:48.510+0000 7f29e4a31640 1 -- 172.21.15.57:0/2516750687 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3424442484 0 0) 0x7f29e0197790 con 0x7f29e01088e0 2024-06-11T13:24:48.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:48.511+0000 7f29e4a31640 1 -- 172.21.15.57:0/2516750687 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f29c8003520 con 0x7f29e01088e0 2024-06-11T13:24:48.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:48.511+0000 7f29e4a31640 1 -- 172.21.15.57:0/2516750687 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2028067002 0 0) 0x7f29e01a41c0 con 0x7f29e0104ba0 2024-06-11T13:24:48.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:48.511+0000 7f29e4a31640 1 -- 172.21.15.57:0/2516750687 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f29e0197790 con 0x7f29e0104ba0 2024-06-11T13:24:48.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:48.511+0000 7f29e4a31640 1 -- 172.21.15.57:0/2516750687 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3903534599 0 0) 0x7f29e01a2fe0 con 0x7f29e0100770 2024-06-11T13:24:48.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:48.511+0000 7f29e4a31640 1 -- 172.21.15.57:0/2516750687 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f29e01a41c0 con 0x7f29e0100770 2024-06-11T13:24:48.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:48.511+0000 7f29e4a31640 1 -- 172.21.15.57:0/2516750687 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3526098620 0 0) 0x7f29c8003520 con 0x7f29e01088e0 2024-06-11T13:24:48.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:48.511+0000 7f29e4a31640 1 -- 172.21.15.57:0/2516750687 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f29e01a2fe0 con 0x7f29e01088e0 2024-06-11T13:24:48.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:48.511+0000 7f29e4a31640 1 -- 172.21.15.57:0/2516750687 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f29d80046a0 con 0x7f29e01088e0 2024-06-11T13:24:48.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:48.511+0000 7f29e4a31640 1 -- 172.21.15.57:0/2516750687 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3140138325 0 0) 0x7f29e0197790 con 0x7f29e0104ba0 2024-06-11T13:24:48.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:48.511+0000 7f29e4a31640 1 -- 172.21.15.57:0/2516750687 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f29c8003520 con 0x7f29e0104ba0 2024-06-11T13:24:48.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:48.511+0000 7f29e4a31640 1 -- 172.21.15.57:0/2516750687 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f29d4001830 con 0x7f29e0104ba0 2024-06-11T13:24:48.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:48.511+0000 7f29e4a31640 1 -- 172.21.15.57:0/2516750687 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 918621262 0 0) 0x7f29e01a41c0 con 0x7f29e0100770 2024-06-11T13:24:48.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:48.511+0000 7f29e4a31640 1 -- 172.21.15.57:0/2516750687 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f29e0197790 con 0x7f29e0100770 2024-06-11T13:24:48.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:48.511+0000 7f29e4a31640 1 -- 172.21.15.57:0/2516750687 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f29d0001af0 con 0x7f29e0100770 2024-06-11T13:24:48.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:48.511+0000 7f29e4a31640 1 -- 172.21.15.57:0/2516750687 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2116722283 0 0) 0x7f29e01a2fe0 con 0x7f29e01088e0 2024-06-11T13:24:48.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:48.511+0000 7f29e4a31640 1 -- 172.21.15.57:0/2516750687 >> v1:172.21.15.57:6790/0 conn(0x7f29e0104ba0 legacy=0x7f29e019d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:24:48.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:48.511+0000 7f29e4a31640 1 -- 172.21.15.57:0/2516750687 >> v1:172.21.15.107:6789/0 conn(0x7f29e0100770 legacy=0x7f29e0196ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:24:48.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:48.511+0000 7f29e4a31640 1 -- 172.21.15.57:0/2516750687 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e01a53a0 con 0x7f29e01088e0 2024-06-11T13:24:48.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:48.511+0000 7f29e5a33640 1 -- 172.21.15.57:0/2516750687 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f29e01a43f0 con 0x7f29e01088e0 2024-06-11T13:24:48.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:48.511+0000 7f29e5a33640 1 -- 172.21.15.57:0/2516750687 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f29e01a4930 con 0x7f29e01088e0 2024-06-11T13:24:48.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:48.511+0000 7f29e4a31640 1 -- 172.21.15.57:0/2516750687 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f29d8004940 con 0x7f29e01088e0 2024-06-11T13:24:48.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:48.511+0000 7f29e4a31640 1 -- 172.21.15.57:0/2516750687 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f29d8005000 con 0x7f29e01088e0 2024-06-11T13:24:48.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:48.512+0000 7f29e4a31640 1 -- 172.21.15.57:0/2516750687 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 13) ==== 46308+0+0 (unknown 2805797583 0 0) 0x7f29d8004fe0 con 0x7f29e01088e0 2024-06-11T13:24:48.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:48.512+0000 7f29e5a33640 1 -- 172.21.15.57:0/2516750687 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29cc005080 con 0x7f29e01088e0 2024-06-11T13:24:48.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:48.512+0000 7f29e4a31640 1 -- 172.21.15.57:0/2516750687 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(4..4 src has 1..4) ==== 1069+0+0 (unknown 1503136288 0 0) 0x7f29d8047870 con 0x7f29e01088e0 2024-06-11T13:24:48.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:48.518+0000 7f29e4a31640 1 -- 172.21.15.57:0/2516750687 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f29d8017550 con 0x7f29e01088e0 2024-06-11T13:24:48.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:48.711+0000 7f29e5a33640 1 -- 172.21.15.57:0/2516750687 --> v1:172.21.15.57:6800/2989665238 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f29cc000b90 con 0x7f29c803a3b0 2024-06-11T13:24:48.779 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:24:48 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:24:48.487+0000 7f73ab6a4040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-11T13:24:48.779 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:24:48 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:24:48.635+0000 7f73ab6a4040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-11T13:24:48.779 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:24:48 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:24:48.701+0000 7f73ab6a4040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-11T13:24:48.780 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:24:48 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:24:48.784+0000 7f73ab6a4040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-11T13:24:49.045 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:24:48 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:24:48.844+0000 7f73ab6a4040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-11T13:24:49.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:48 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-11T13:24:49.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:48 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-11T13:24:49.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:48 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:24:49.210 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:48 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-11T13:24:49.210 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:48 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-11T13:24:49.210 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:48 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:24:49.210 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:48 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-11T13:24:49.210 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:48 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-11T13:24:49.210 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:48 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:24:49.795 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:24:49 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:24:49.388+0000 7f73ab6a4040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-11T13:24:49.795 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:24:49 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:24:49.550+0000 7f73ab6a4040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-11T13:24:49.916 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:24:49.922+0000 7f29e4a31640 1 -- 172.21.15.57:0/2516750687 <== mon.0 v1:172.21.15.57:6789/0 10 ==== mgrmap(e 14) ==== 92025+0+0 (unknown 3311910117 0 0) 0x7f29d8001e70 con 0x7f29e01088e0 2024-06-11T13:24:50.228 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:49 smithi057 ceph-mon[29805]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:50.228 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:49 smithi057 ceph-mon[29805]: from='client.14214 v1:172.21.15.57:0/2516750687' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:24:50.228 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:49 smithi057 ceph-mon[29805]: Standby manager daemon x started 2024-06-11T13:24:50.228 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:49 smithi057 ceph-mon[29805]: from='mgr.? v1:172.21.15.107:0/1128925806' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-06-11T13:24:50.228 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:49 smithi057 ceph-mon[29805]: from='mgr.? v1:172.21.15.107:0/1128925806' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-11T13:24:50.228 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:49 smithi057 ceph-mon[29805]: from='mgr.? v1:172.21.15.107:0/1128925806' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-06-11T13:24:50.228 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:49 smithi057 ceph-mon[29805]: from='mgr.? v1:172.21.15.107:0/1128925806' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-11T13:24:50.229 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:49 smithi057 ceph-mon[40588]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:50.229 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:49 smithi057 ceph-mon[40588]: from='client.14214 v1:172.21.15.57:0/2516750687' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:24:50.229 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:49 smithi057 ceph-mon[40588]: Standby manager daemon x started 2024-06-11T13:24:50.229 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:49 smithi057 ceph-mon[40588]: from='mgr.? v1:172.21.15.107:0/1128925806' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-06-11T13:24:50.229 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:49 smithi057 ceph-mon[40588]: from='mgr.? v1:172.21.15.107:0/1128925806' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-11T13:24:50.229 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:49 smithi057 ceph-mon[40588]: from='mgr.? v1:172.21.15.107:0/1128925806' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-06-11T13:24:50.229 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:49 smithi057 ceph-mon[40588]: from='mgr.? v1:172.21.15.107:0/1128925806' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-11T13:24:50.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:49 smithi107 ceph-mon[33178]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:50.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:49 smithi107 ceph-mon[33178]: from='client.14214 v1:172.21.15.57:0/2516750687' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:24:50.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:49 smithi107 ceph-mon[33178]: Standby manager daemon x started 2024-06-11T13:24:50.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:49 smithi107 ceph-mon[33178]: from='mgr.? v1:172.21.15.107:0/1128925806' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-06-11T13:24:50.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:49 smithi107 ceph-mon[33178]: from='mgr.? v1:172.21.15.107:0/1128925806' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-11T13:24:50.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:49 smithi107 ceph-mon[33178]: from='mgr.? v1:172.21.15.107:0/1128925806' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-06-11T13:24:50.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:49 smithi107 ceph-mon[33178]: from='mgr.? v1:172.21.15.107:0/1128925806' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-11T13:24:51.236 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:50 smithi057 ceph-mon[29805]: mgrmap e14: y(active, since 63s), standbys: x 2024-06-11T13:24:51.236 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:50 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-11T13:24:51.236 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:50 smithi057 ceph-mon[40588]: mgrmap e14: y(active, since 63s), standbys: x 2024-06-11T13:24:51.236 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:50 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-11T13:24:51.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:50 smithi107 ceph-mon[33178]: mgrmap e14: y(active, since 63s), standbys: x 2024-06-11T13:24:51.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:50 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-11T13:24:52.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:51 smithi057 ceph-mon[29805]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:52.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:51 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1627697260' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b167b692-f22f-4806-ad75-e94e0ed1e57a"}]: dispatch 2024-06-11T13:24:52.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:51 smithi057 ceph-mon[29805]: from='client.34118 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b167b692-f22f-4806-ad75-e94e0ed1e57a"}]: dispatch 2024-06-11T13:24:52.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:51 smithi057 ceph-mon[29805]: from='client.34118 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b167b692-f22f-4806-ad75-e94e0ed1e57a"}]': finished 2024-06-11T13:24:52.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:51 smithi057 ceph-mon[29805]: osdmap e5: 1 total, 0 up, 1 in 2024-06-11T13:24:52.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:51 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-11T13:24:52.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:51 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2877061985' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-11T13:24:52.282 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:51 smithi057 ceph-mon[40588]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:52.282 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:51 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1627697260' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b167b692-f22f-4806-ad75-e94e0ed1e57a"}]: dispatch 2024-06-11T13:24:52.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:51 smithi057 ceph-mon[40588]: from='client.34118 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b167b692-f22f-4806-ad75-e94e0ed1e57a"}]: dispatch 2024-06-11T13:24:52.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:51 smithi057 ceph-mon[40588]: from='client.34118 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b167b692-f22f-4806-ad75-e94e0ed1e57a"}]': finished 2024-06-11T13:24:52.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:51 smithi057 ceph-mon[40588]: osdmap e5: 1 total, 0 up, 1 in 2024-06-11T13:24:52.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:51 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-11T13:24:52.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:51 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2877061985' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-11T13:24:52.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:51 smithi107 ceph-mon[33178]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:52.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:51 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1627697260' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b167b692-f22f-4806-ad75-e94e0ed1e57a"}]: dispatch 2024-06-11T13:24:52.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:51 smithi107 ceph-mon[33178]: from='client.34118 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b167b692-f22f-4806-ad75-e94e0ed1e57a"}]: dispatch 2024-06-11T13:24:52.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:51 smithi107 ceph-mon[33178]: from='client.34118 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b167b692-f22f-4806-ad75-e94e0ed1e57a"}]': finished 2024-06-11T13:24:52.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:51 smithi107 ceph-mon[33178]: osdmap e5: 1 total, 0 up, 1 in 2024-06-11T13:24:52.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:51 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-11T13:24:52.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:51 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2877061985' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-11T13:24:54.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:53 smithi057 ceph-mon[29805]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:54.032 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:53 smithi057 ceph-mon[40588]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:54.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:53 smithi107 ceph-mon[33178]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:56.150 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:55 smithi057 ceph-mon[29805]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:56.150 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:55 smithi057 ceph-mon[40588]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:56.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:55 smithi107 ceph-mon[33178]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:58.282 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:57 smithi057 ceph-mon[40588]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:58.282 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:57 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-11T13:24:58.282 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:57 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:24:58.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:57 smithi057 ceph-mon[29805]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:58.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:57 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-11T13:24:58.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:57 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:24:58.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:57 smithi107 ceph-mon[33178]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:24:58.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:57 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-11T13:24:58.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:57 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:24:59.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:58 smithi057 ceph-mon[29805]: Deploying daemon osd.0 on smithi057 2024-06-11T13:24:59.032 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:58 smithi057 ceph-mon[40588]: Deploying daemon osd.0 on smithi057 2024-06-11T13:24:59.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:58 smithi107 ceph-mon[33178]: Deploying daemon osd.0 on smithi057 2024-06-11T13:25:00.171 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:24:59 smithi057 ceph-mon[40588]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:25:00.171 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:24:59 smithi057 ceph-mon[29805]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:25:00.292 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:24:59 smithi107 ceph-mon[33178]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:25:02.099 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:01 smithi057 ceph-mon[29805]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:25:02.099 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:01 smithi057 ceph-mon[40588]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:25:02.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:01 smithi107 ceph-mon[33178]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:25:03.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:02 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:25:03.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:02 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:03.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:02 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:03.032 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:02 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:25:03.032 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:02 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:03.032 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:02 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:03.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:02 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:25:03.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:02 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:03.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:02 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:04.282 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:04 smithi057 ceph-mon[40588]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:25:04.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:04 smithi057 ceph-mon[29805]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:25:04.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:04 smithi107 ceph-mon[33178]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:25:05.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:05.786+0000 7f29e4a31640 1 -- 172.21.15.57:0/2516750687 <== mgr.14150 v1:172.21.15.57:6800/2989665238 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (unknown 0 0 3456505939) 0x7f29cc000b90 con 0x7f29c803a3b0 2024-06-11T13:25:05.782 INFO:teuthology.orchestra.run.smithi057.stdout:Created osd(s) 0 on host 'smithi057' 2024-06-11T13:25:05.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:05.789+0000 7f29e5a33640 1 -- 172.21.15.57:0/2516750687 >> v1:172.21.15.57:6800/2989665238 conn(0x7f29c803a3b0 legacy=0x7f29c803c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:25:05.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:05.789+0000 7f29e5a33640 1 -- 172.21.15.57:0/2516750687 >> v1:172.21.15.57:6789/0 conn(0x7f29e01088e0 legacy=0x7f29e01a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:25:05.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:05.789+0000 7f29e5a33640 1 -- 172.21.15.57:0/2516750687 shutdown_connections 2024-06-11T13:25:05.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:05.789+0000 7f29e5a33640 1 -- 172.21.15.57:0/2516750687 >> 172.21.15.57:0/2516750687 conn(0x7f29e00fbd10 msgr2=0x7f29e0069bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:25:05.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:05.789+0000 7f29e5a33640 1 -- 172.21.15.57:0/2516750687 shutdown_connections 2024-06-11T13:25:05.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:05.789+0000 7f29e5a33640 1 -- 172.21.15.57:0/2516750687 wait complete. 2024-06-11T13:25:06.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:05 smithi057 ceph-mon[29805]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:25:06.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:05 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:06.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:05 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:06.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:05 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:25:06.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:05 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:25:06.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:05 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:06.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:05 smithi057 ceph-mon[29805]: from='osd.0 v1:172.21.15.57:6801/978735223' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-11T13:25:06.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:05 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:25:06.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:05 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:06.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:05 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:06.032 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:05 smithi057 ceph-mon[40588]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:25:06.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:05 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:06.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:05 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:06.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:05 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:25:06.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:05 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:25:06.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:05 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:06.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:05 smithi057 ceph-mon[40588]: from='osd.0 v1:172.21.15.57:6801/978735223' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-11T13:25:06.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:05 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:25:06.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:05 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:06.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:05 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:06.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:05 smithi107 ceph-mon[33178]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:25:06.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:05 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:06.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:05 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:06.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:05 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:25:06.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:05 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:25:06.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:05 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:06.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:05 smithi107 ceph-mon[33178]: from='osd.0 v1:172.21.15.57:6801/978735223' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-11T13:25:06.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:05 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:25:06.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:05 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:06.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:05 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:06.399 DEBUG:teuthology.orchestra.run.smithi057:osd.0> sudo journalctl -f -n 0 -u ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@osd.0.service 2024-06-11T13:25:06.401 INFO:tasks.cephadm:Deploying osd.1 on smithi057 with /dev/vg_nvme/lv_3... 2024-06-11T13:25:06.402 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-11T13:25:07.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:07 smithi057 ceph-mon[29805]: from='osd.0 v1:172.21.15.57:6801/978735223' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-11T13:25:07.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:07 smithi057 ceph-mon[29805]: osdmap e6: 1 total, 0 up, 1 in 2024-06-11T13:25:07.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:07 smithi057 ceph-mon[29805]: from='osd.0 v1:172.21.15.57:6801/978735223' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-06-11T13:25:07.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:07 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-11T13:25:07.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:07 smithi057 ceph-mon[29805]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:25:07.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:07 smithi057 ceph-mon[40588]: from='osd.0 v1:172.21.15.57:6801/978735223' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-11T13:25:07.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:07 smithi057 ceph-mon[40588]: osdmap e6: 1 total, 0 up, 1 in 2024-06-11T13:25:07.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:07 smithi057 ceph-mon[40588]: from='osd.0 v1:172.21.15.57:6801/978735223' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-06-11T13:25:07.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:07 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-11T13:25:07.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:07 smithi057 ceph-mon[40588]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:25:07.533 INFO:journalctl@ceph.osd.0.smithi057.stdout:Jun 11 13:25:07 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-0[44455]: 2024-06-11T13:25:07.259+0000 7f4adb48b640 -1 osd.0 0 waiting for initial osdmap 2024-06-11T13:25:07.533 INFO:journalctl@ceph.osd.0.smithi057.stdout:Jun 11 13:25:07 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-0[44455]: 2024-06-11T13:25:07.264+0000 7f4ad72b3640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-11T13:25:07.545 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:07 smithi107 ceph-mon[33178]: from='osd.0 v1:172.21.15.57:6801/978735223' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-11T13:25:07.545 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:07 smithi107 ceph-mon[33178]: osdmap e6: 1 total, 0 up, 1 in 2024-06-11T13:25:07.545 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:07 smithi107 ceph-mon[33178]: from='osd.0 v1:172.21.15.57:6801/978735223' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-06-11T13:25:07.545 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:07 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-11T13:25:07.545 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:07 smithi107 ceph-mon[33178]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:25:07.691 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.a/config 2024-06-11T13:25:08.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:08 smithi057 ceph-mon[29805]: from='osd.0 v1:172.21.15.57:6801/978735223' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2024-06-11T13:25:08.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:08 smithi057 ceph-mon[29805]: osdmap e7: 1 total, 0 up, 1 in 2024-06-11T13:25:08.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:08 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-11T13:25:08.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:08 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-11T13:25:08.532 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:08 smithi057 ceph-mon[40588]: from='osd.0 v1:172.21.15.57:6801/978735223' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2024-06-11T13:25:08.532 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:08 smithi057 ceph-mon[40588]: osdmap e7: 1 total, 0 up, 1 in 2024-06-11T13:25:08.532 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:08 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-11T13:25:08.532 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:08 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-11T13:25:08.545 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:08 smithi107 ceph-mon[33178]: from='osd.0 v1:172.21.15.57:6801/978735223' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2024-06-11T13:25:08.545 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:08 smithi107 ceph-mon[33178]: osdmap e7: 1 total, 0 up, 1 in 2024-06-11T13:25:08.545 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:08 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-11T13:25:08.545 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:08 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-11T13:25:09.508 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:09 smithi057 ceph-mon[29805]: purged_snaps scrub starts 2024-06-11T13:25:09.508 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:09 smithi057 ceph-mon[29805]: purged_snaps scrub ok 2024-06-11T13:25:09.509 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:09 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-11T13:25:09.509 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:09 smithi057 ceph-mon[29805]: osd.0 v1:172.21.15.57:6801/978735223 boot 2024-06-11T13:25:09.509 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:09 smithi057 ceph-mon[29805]: osdmap e8: 1 total, 1 up, 1 in 2024-06-11T13:25:09.509 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:09 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-11T13:25:09.509 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:09 smithi057 ceph-mon[29805]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:25:09.509 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:09 smithi057 ceph-mon[40588]: purged_snaps scrub starts 2024-06-11T13:25:09.509 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:09 smithi057 ceph-mon[40588]: purged_snaps scrub ok 2024-06-11T13:25:09.509 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:09 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-11T13:25:09.509 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:09 smithi057 ceph-mon[40588]: osd.0 v1:172.21.15.57:6801/978735223 boot 2024-06-11T13:25:09.509 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:09 smithi057 ceph-mon[40588]: osdmap e8: 1 total, 1 up, 1 in 2024-06-11T13:25:09.509 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:09 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-11T13:25:09.510 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:09 smithi057 ceph-mon[40588]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:25:09.545 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:09 smithi107 ceph-mon[33178]: purged_snaps scrub starts 2024-06-11T13:25:09.545 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:09 smithi107 ceph-mon[33178]: purged_snaps scrub ok 2024-06-11T13:25:09.545 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:09 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-11T13:25:09.545 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:09 smithi107 ceph-mon[33178]: osd.0 v1:172.21.15.57:6801/978735223 boot 2024-06-11T13:25:09.545 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:09 smithi107 ceph-mon[33178]: osdmap e8: 1 total, 1 up, 1 in 2024-06-11T13:25:09.545 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:09 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-11T13:25:09.545 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:09 smithi107 ceph-mon[33178]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-11T13:25:09.834 INFO:teuthology.orchestra.run.smithi057.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-11T13:25:09.922 INFO:teuthology.orchestra.run.smithi057.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-11T13:25:09.940 INFO:teuthology.orchestra.run.smithi057.stderr: stderr: 10+0 records in 2024-06-11T13:25:09.940 INFO:teuthology.orchestra.run.smithi057.stderr:10+0 records out 2024-06-11T13:25:09.940 INFO:teuthology.orchestra.run.smithi057.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.016044 s, 654 MB/s 2024-06-11T13:25:09.940 INFO:teuthology.orchestra.run.smithi057.stderr:--> Zapping successful for: 2024-06-11T13:25:10.520 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:10 smithi057 ceph-mon[29805]: osdmap e9: 1 total, 1 up, 1 in 2024-06-11T13:25:10.520 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:10 smithi057 ceph-mon[40588]: osdmap e9: 1 total, 1 up, 1 in 2024-06-11T13:25:10.795 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:10 smithi107 ceph-mon[33178]: osdmap e9: 1 total, 1 up, 1 in 2024-06-11T13:25:10.850 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi057:vg_nvme/lv_3 2024-06-11T13:25:11.429 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.a/config 2024-06-11T13:25:11.552 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:11 smithi057 ceph-mon[29805]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-11T13:25:11.552 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:11 smithi057 ceph-mon[40588]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-11T13:25:11.795 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:11 smithi107 ceph-mon[33178]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-11T13:25:12.465 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:12 smithi057 ceph-mon[40588]: Detected new or changed devices on smithi057 2024-06-11T13:25:12.466 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:12 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:12.466 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:12 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:12.466 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:12 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:25:12.466 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:12 smithi057 ceph-mon[40588]: Adjusting osd_memory_target on smithi057 to 16068M 2024-06-11T13:25:12.466 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:12 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:12.466 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:12 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:25:12.466 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:12 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:25:12.466 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:12 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:12.466 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:12 smithi057 ceph-mon[29805]: Detected new or changed devices on smithi057 2024-06-11T13:25:12.466 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:12 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:12.466 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:12 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:12.467 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:12 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:25:12.467 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:12 smithi057 ceph-mon[29805]: Adjusting osd_memory_target on smithi057 to 16068M 2024-06-11T13:25:12.467 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:12 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:12.467 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:12 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:25:12.467 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:12 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:25:12.467 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:12 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:12.795 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:12 smithi107 ceph-mon[33178]: Detected new or changed devices on smithi057 2024-06-11T13:25:12.795 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:12 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:12.795 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:12 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:12.795 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:12 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:25:12.795 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:12 smithi107 ceph-mon[33178]: Adjusting osd_memory_target on smithi057 to 16068M 2024-06-11T13:25:12.795 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:12 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:12.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:12 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:25:12.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:12 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:25:12.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:12 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:12.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:12.965+0000 7f8225c40640 1 -- 172.21.15.57:0/634838059 >> v1:172.21.15.57:6789/0 conn(0x7f82201088e0 legacy=0x7f822010ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:25:12.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:12.966+0000 7f8225c40640 1 -- 172.21.15.57:0/634838059 shutdown_connections 2024-06-11T13:25:12.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:12.966+0000 7f8225c40640 1 -- 172.21.15.57:0/634838059 >> 172.21.15.57:0/634838059 conn(0x7f82200fbd10 msgr2=0x7f82200fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:25:12.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:12.966+0000 7f8225c40640 1 -- 172.21.15.57:0/634838059 shutdown_connections 2024-06-11T13:25:12.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:12.966+0000 7f8225c40640 1 -- 172.21.15.57:0/634838059 wait complete. 2024-06-11T13:25:12.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:12.966+0000 7f8225c40640 1 Processor -- start 2024-06-11T13:25:12.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:12.966+0000 7f8225c40640 1 -- start start 2024-06-11T13:25:12.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:12.967+0000 7f8225c40640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f822006a270 con 0x7f82201088e0 2024-06-11T13:25:12.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:12.967+0000 7f8225c40640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f82201a2ea0 con 0x7f8220100770 2024-06-11T13:25:12.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:12.967+0000 7f8225c40640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f82201a4080 con 0x7f8220104ba0 2024-06-11T13:25:12.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:12.967+0000 7f821ffff640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f82201088e0 0x7f82201a0770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:42620/0 (socket says 172.21.15.57:42620) 2024-06-11T13:25:12.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:12.967+0000 7f821effd640 1 --1- >> v1:172.21.15.57:6790/0 conn(0x7f8220104ba0 0x7f822019d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6790/0 says I am v1:172.21.15.57:32934/0 (socket says 172.21.15.57:32934) 2024-06-11T13:25:12.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:12.967+0000 7f821ffff640 1 -- 172.21.15.57:0/2817319484 learned_addr learned my addr 172.21.15.57:0/2817319484 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:25:12.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:12.967+0000 7f821cff9640 1 -- 172.21.15.57:0/2817319484 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1116298086 0 0) 0x7f822006a270 con 0x7f82201088e0 2024-06-11T13:25:12.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:12.967+0000 7f821cff9640 1 -- 172.21.15.57:0/2817319484 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f820c0051b0 con 0x7f82201088e0 2024-06-11T13:25:12.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:12.967+0000 7f821cff9640 1 -- 172.21.15.57:0/2817319484 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 871335091 0 0) 0x7f82201a4080 con 0x7f8220104ba0 2024-06-11T13:25:12.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:12.967+0000 7f821cff9640 1 -- 172.21.15.57:0/2817319484 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f822006a270 con 0x7f8220104ba0 2024-06-11T13:25:12.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:12.967+0000 7f821cff9640 1 -- 172.21.15.57:0/2817319484 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3847162232 0 0) 0x7f82201a2ea0 con 0x7f8220100770 2024-06-11T13:25:12.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:12.967+0000 7f821cff9640 1 -- 172.21.15.57:0/2817319484 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f82201a4080 con 0x7f8220100770 2024-06-11T13:25:12.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:12.967+0000 7f821cff9640 1 -- 172.21.15.57:0/2817319484 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2894280766 0 0) 0x7f820c0051b0 con 0x7f82201088e0 2024-06-11T13:25:12.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:12.967+0000 7f821cff9640 1 -- 172.21.15.57:0/2817319484 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f82201a2ea0 con 0x7f82201088e0 2024-06-11T13:25:12.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:12.967+0000 7f821cff9640 1 -- 172.21.15.57:0/2817319484 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f82180042d0 con 0x7f82201088e0 2024-06-11T13:25:12.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:12.967+0000 7f821cff9640 1 -- 172.21.15.57:0/2817319484 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1600402848 0 0) 0x7f822006a270 con 0x7f8220104ba0 2024-06-11T13:25:12.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:12.967+0000 7f821cff9640 1 -- 172.21.15.57:0/2817319484 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f820c0051b0 con 0x7f8220104ba0 2024-06-11T13:25:12.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:12.967+0000 7f821cff9640 1 -- 172.21.15.57:0/2817319484 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f8208001890 con 0x7f8220104ba0 2024-06-11T13:25:12.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:12.968+0000 7f821cff9640 1 -- 172.21.15.57:0/2817319484 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3888223392 0 0) 0x7f82201a4080 con 0x7f8220100770 2024-06-11T13:25:12.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:12.968+0000 7f821cff9640 1 -- 172.21.15.57:0/2817319484 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f822006a270 con 0x7f8220100770 2024-06-11T13:25:12.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:12.968+0000 7f821cff9640 1 -- 172.21.15.57:0/2817319484 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f8210001b70 con 0x7f8220100770 2024-06-11T13:25:12.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:12.968+0000 7f821cff9640 1 -- 172.21.15.57:0/2817319484 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2110829918 0 0) 0x7f82201a2ea0 con 0x7f82201088e0 2024-06-11T13:25:12.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:12.968+0000 7f821cff9640 1 -- 172.21.15.57:0/2817319484 >> v1:172.21.15.57:6790/0 conn(0x7f8220104ba0 legacy=0x7f822019d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:25:12.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:12.968+0000 7f821cff9640 1 -- 172.21.15.57:0/2817319484 >> v1:172.21.15.107:6789/0 conn(0x7f8220100770 legacy=0x7f8220199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:25:12.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:12.968+0000 7f821cff9640 1 -- 172.21.15.57:0/2817319484 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82201a52c0 con 0x7f82201088e0 2024-06-11T13:25:12.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:12.968+0000 7f8225c40640 1 -- 172.21.15.57:0/2817319484 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f82201a42b0 con 0x7f82201088e0 2024-06-11T13:25:12.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:12.968+0000 7f8225c40640 1 -- 172.21.15.57:0/2817319484 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f82201a47f0 con 0x7f82201088e0 2024-06-11T13:25:12.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:12.968+0000 7f821cff9640 1 -- 172.21.15.57:0/2817319484 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f8218001ac0 con 0x7f82201088e0 2024-06-11T13:25:12.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:12.968+0000 7f821cff9640 1 -- 172.21.15.57:0/2817319484 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f8218004e20 con 0x7f82201088e0 2024-06-11T13:25:12.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:12.969+0000 7f8225c40640 1 -- 172.21.15.57:0/2817319484 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82000050b0 con 0x7f82201088e0 2024-06-11T13:25:12.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:12.969+0000 7f821cff9640 1 -- 172.21.15.57:0/2817319484 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 14) ==== 92025+0+0 (unknown 3311910117 0 0) 0x7f8218005080 con 0x7f82201088e0 2024-06-11T13:25:12.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:12.970+0000 7f821cff9640 1 -- 172.21.15.57:0/2817319484 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(9..9 src has 1..9) ==== 1541+0+0 (unknown 774836669 0 0) 0x7f8218088220 con 0x7f82201088e0 2024-06-11T13:25:12.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:12.974+0000 7f821cff9640 1 -- 172.21.15.57:0/2817319484 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f8218004e20 con 0x7f82201088e0 2024-06-11T13:25:13.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:13.169+0000 7f8225c40640 1 -- 172.21.15.57:0/2817319484 --> v1:172.21.15.57:6800/2989665238 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f8200000bc0 con 0x7f820c06ed90 2024-06-11T13:25:13.454 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:13 smithi057 ceph-mon[29805]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-11T13:25:13.454 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:13 smithi057 ceph-mon[29805]: from='client.14241 v1:172.21.15.57:0/2817319484' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:25:13.454 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:13 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-11T13:25:13.454 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:13 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-11T13:25:13.454 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:13 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:25:13.454 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:13 smithi057 ceph-mon[40588]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-11T13:25:13.454 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:13 smithi057 ceph-mon[40588]: from='client.14241 v1:172.21.15.57:0/2817319484' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:25:13.455 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:13 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-11T13:25:13.455 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:13 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-11T13:25:13.455 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:13 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:25:13.795 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:13 smithi107 ceph-mon[33178]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-11T13:25:13.795 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:13 smithi107 ceph-mon[33178]: from='client.14241 v1:172.21.15.57:0/2817319484' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:25:13.795 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:13 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-11T13:25:13.795 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:13 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-11T13:25:13.795 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:13 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:25:15.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:15 smithi057 ceph-mon[29805]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-11T13:25:15.782 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:15 smithi057 ceph-mon[40588]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-11T13:25:15.795 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:15 smithi107 ceph-mon[33178]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-11T13:25:16.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:16 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/71713030' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e153beb-619d-47ba-ac9f-12d2fbc69b1e"}]: dispatch 2024-06-11T13:25:16.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:16 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/71713030' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3e153beb-619d-47ba-ac9f-12d2fbc69b1e"}]': finished 2024-06-11T13:25:16.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:16 smithi057 ceph-mon[29805]: osdmap e10: 2 total, 1 up, 2 in 2024-06-11T13:25:16.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:16 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-11T13:25:16.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:16 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/605062340' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-11T13:25:16.532 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:16 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/71713030' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e153beb-619d-47ba-ac9f-12d2fbc69b1e"}]: dispatch 2024-06-11T13:25:16.532 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:16 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/71713030' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3e153beb-619d-47ba-ac9f-12d2fbc69b1e"}]': finished 2024-06-11T13:25:16.532 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:16 smithi057 ceph-mon[40588]: osdmap e10: 2 total, 1 up, 2 in 2024-06-11T13:25:16.532 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:16 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-11T13:25:16.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:16 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/605062340' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-11T13:25:16.795 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:16 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/71713030' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e153beb-619d-47ba-ac9f-12d2fbc69b1e"}]: dispatch 2024-06-11T13:25:16.795 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:16 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/71713030' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3e153beb-619d-47ba-ac9f-12d2fbc69b1e"}]': finished 2024-06-11T13:25:16.795 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:16 smithi107 ceph-mon[33178]: osdmap e10: 2 total, 1 up, 2 in 2024-06-11T13:25:16.795 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:16 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-11T13:25:16.795 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:16 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/605062340' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-11T13:25:18.032 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:17 smithi057 ceph-mon[40588]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-11T13:25:18.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:17 smithi057 ceph-mon[29805]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-11T13:25:18.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:17 smithi107 ceph-mon[33178]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-11T13:25:19.848 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:19 smithi057 ceph-mon[29805]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-11T13:25:19.848 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:19 smithi057 ceph-mon[40588]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-11T13:25:20.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:19 smithi107 ceph-mon[33178]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-11T13:25:21.776 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:21 smithi057 ceph-mon[29805]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-11T13:25:21.777 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:21 smithi057 ceph-mon[40588]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-11T13:25:22.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:21 smithi107 ceph-mon[33178]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-11T13:25:22.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:22 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-11T13:25:22.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:22 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:25:22.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:22 smithi057 ceph-mon[29805]: Deploying daemon osd.1 on smithi057 2024-06-11T13:25:22.782 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:22 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-11T13:25:22.782 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:22 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:25:22.782 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:22 smithi057 ceph-mon[40588]: Deploying daemon osd.1 on smithi057 2024-06-11T13:25:23.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:22 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-11T13:25:23.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:22 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:25:23.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:22 smithi107 ceph-mon[33178]: Deploying daemon osd.1 on smithi057 2024-06-11T13:25:23.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:23 smithi057 ceph-mon[29805]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-11T13:25:23.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:23 smithi057 ceph-mon[40588]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-11T13:25:24.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:23 smithi107 ceph-mon[33178]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-11T13:25:26.030 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:25 smithi057 ceph-mon[29805]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-11T13:25:26.030 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:25 smithi057 ceph-mon[40588]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-11T13:25:26.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:25 smithi107 ceph-mon[33178]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-11T13:25:27.891 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:27 smithi057 ceph-mon[40588]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-11T13:25:27.891 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:27 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:25:27.891 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:27 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:27.891 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:27 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:27.891 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:27 smithi057 ceph-mon[29805]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-11T13:25:27.891 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:27 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:25:27.891 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:27 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:27.891 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:27 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:28.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:27 smithi107 ceph-mon[33178]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-11T13:25:28.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:27 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:25:28.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:27 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:28.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:27 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:30.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:29 smithi057 ceph-mon[29805]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-11T13:25:30.032 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:29 smithi057 ceph-mon[40588]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-11T13:25:30.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:29 smithi107 ceph-mon[33178]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-11T13:25:30.775 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:30.779+0000 7f821cff9640 1 -- 172.21.15.57:0/2817319484 <== mgr.14150 v1:172.21.15.57:6800/2989665238 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (unknown 0 0 2450781927) 0x7f8200000bc0 con 0x7f820c06ed90 2024-06-11T13:25:30.775 INFO:teuthology.orchestra.run.smithi057.stdout:Created osd(s) 1 on host 'smithi057' 2024-06-11T13:25:30.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:30.782+0000 7f8225c40640 1 -- 172.21.15.57:0/2817319484 >> v1:172.21.15.57:6800/2989665238 conn(0x7f820c06ed90 legacy=0x7f820c071250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:25:30.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:30.782+0000 7f8225c40640 1 -- 172.21.15.57:0/2817319484 >> v1:172.21.15.57:6789/0 conn(0x7f82201088e0 legacy=0x7f82201a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:25:30.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:30.782+0000 7f8225c40640 1 -- 172.21.15.57:0/2817319484 shutdown_connections 2024-06-11T13:25:30.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:30.782+0000 7f8225c40640 1 -- 172.21.15.57:0/2817319484 >> 172.21.15.57:0/2817319484 conn(0x7f82200fbd10 msgr2=0x7f822010b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:25:30.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:30.782+0000 7f8225c40640 1 -- 172.21.15.57:0/2817319484 shutdown_connections 2024-06-11T13:25:30.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:30.782+0000 7f8225c40640 1 -- 172.21.15.57:0/2817319484 wait complete. 2024-06-11T13:25:31.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:30 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:31.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:30 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:31.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:30 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:25:31.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:30 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:25:31.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:30 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:31.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:30 smithi057 ceph-mon[29805]: from='osd.1 v1:172.21.15.57:6805/137889538' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-11T13:25:31.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:30 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:25:31.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:30 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:31.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:30 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:31.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:30 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:31.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:30 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:31.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:30 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:25:31.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:30 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:25:31.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:30 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:31.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:30 smithi057 ceph-mon[40588]: from='osd.1 v1:172.21.15.57:6805/137889538' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-11T13:25:31.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:30 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:25:31.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:30 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:31.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:30 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:31.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:30 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:31.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:30 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:31.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:30 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:25:31.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:30 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:25:31.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:30 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:31.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:30 smithi107 ceph-mon[33178]: from='osd.1 v1:172.21.15.57:6805/137889538' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-11T13:25:31.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:30 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:25:31.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:30 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:31.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:30 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:31.441 DEBUG:teuthology.orchestra.run.smithi057:osd.1> sudo journalctl -f -n 0 -u ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@osd.1.service 2024-06-11T13:25:31.444 INFO:tasks.cephadm:Deploying osd.2 on smithi057 with /dev/vg_nvme/lv_2... 2024-06-11T13:25:31.444 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-11T13:25:32.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:32 smithi057 ceph-mon[29805]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-11T13:25:32.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:32 smithi057 ceph-mon[29805]: from='osd.1 v1:172.21.15.57:6805/137889538' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-11T13:25:32.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:32 smithi057 ceph-mon[29805]: osdmap e11: 2 total, 1 up, 2 in 2024-06-11T13:25:32.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:32 smithi057 ceph-mon[29805]: from='osd.1 v1:172.21.15.57:6805/137889538' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-06-11T13:25:32.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:32 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-11T13:25:32.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:32 smithi057 ceph-mon[40588]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-11T13:25:32.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:32 smithi057 ceph-mon[40588]: from='osd.1 v1:172.21.15.57:6805/137889538' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-11T13:25:32.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:32 smithi057 ceph-mon[40588]: osdmap e11: 2 total, 1 up, 2 in 2024-06-11T13:25:32.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:32 smithi057 ceph-mon[40588]: from='osd.1 v1:172.21.15.57:6805/137889538' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-06-11T13:25:32.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:32 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-11T13:25:32.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:32 smithi107 ceph-mon[33178]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-11T13:25:32.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:32 smithi107 ceph-mon[33178]: from='osd.1 v1:172.21.15.57:6805/137889538' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-11T13:25:32.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:32 smithi107 ceph-mon[33178]: osdmap e11: 2 total, 1 up, 2 in 2024-06-11T13:25:32.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:32 smithi107 ceph-mon[33178]: from='osd.1 v1:172.21.15.57:6805/137889538' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-06-11T13:25:32.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:32 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-11T13:25:32.594 INFO:journalctl@ceph.osd.1.smithi057.stdout:Jun 11 13:25:32 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-1[50184]: 2024-06-11T13:25:32.398+0000 7fc02ff80640 -1 osd.1 0 waiting for initial osdmap 2024-06-11T13:25:32.594 INFO:journalctl@ceph.osd.1.smithi057.stdout:Jun 11 13:25:32 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-1[50184]: 2024-06-11T13:25:32.405+0000 7fc02758c640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-11T13:25:33.128 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.a/config 2024-06-11T13:25:33.233 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:33 smithi057 ceph-mon[29805]: from='osd.1 v1:172.21.15.57:6805/137889538' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2024-06-11T13:25:33.234 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:33 smithi057 ceph-mon[29805]: osdmap e12: 2 total, 1 up, 2 in 2024-06-11T13:25:33.234 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:33 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-11T13:25:33.234 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:33 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-11T13:25:33.234 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:33 smithi057 ceph-mon[29805]: from='osd.1 v1:172.21.15.57:6805/137889538' entity='osd.1' 2024-06-11T13:25:33.234 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:33 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-11T13:25:33.234 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:33 smithi057 ceph-mon[40588]: from='osd.1 v1:172.21.15.57:6805/137889538' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2024-06-11T13:25:33.234 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:33 smithi057 ceph-mon[40588]: osdmap e12: 2 total, 1 up, 2 in 2024-06-11T13:25:33.234 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:33 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-11T13:25:33.234 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:33 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-11T13:25:33.234 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:33 smithi057 ceph-mon[40588]: from='osd.1 v1:172.21.15.57:6805/137889538' entity='osd.1' 2024-06-11T13:25:33.234 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:33 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-11T13:25:33.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:33 smithi107 ceph-mon[33178]: from='osd.1 v1:172.21.15.57:6805/137889538' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2024-06-11T13:25:33.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:33 smithi107 ceph-mon[33178]: osdmap e12: 2 total, 1 up, 2 in 2024-06-11T13:25:33.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:33 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-11T13:25:33.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:33 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-11T13:25:33.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:33 smithi107 ceph-mon[33178]: from='osd.1 v1:172.21.15.57:6805/137889538' entity='osd.1' 2024-06-11T13:25:33.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:33 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-11T13:25:34.282 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:34 smithi057 ceph-mon[40588]: purged_snaps scrub starts 2024-06-11T13:25:34.282 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:34 smithi057 ceph-mon[40588]: purged_snaps scrub ok 2024-06-11T13:25:34.282 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:34 smithi057 ceph-mon[40588]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-11T13:25:34.282 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:34 smithi057 ceph-mon[40588]: osd.1 v1:172.21.15.57:6805/137889538 boot 2024-06-11T13:25:34.282 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:34 smithi057 ceph-mon[40588]: osdmap e13: 2 total, 2 up, 2 in 2024-06-11T13:25:34.282 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:34 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-11T13:25:34.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:34 smithi057 ceph-mon[29805]: purged_snaps scrub starts 2024-06-11T13:25:34.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:34 smithi057 ceph-mon[29805]: purged_snaps scrub ok 2024-06-11T13:25:34.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:34 smithi057 ceph-mon[29805]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-11T13:25:34.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:34 smithi057 ceph-mon[29805]: osd.1 v1:172.21.15.57:6805/137889538 boot 2024-06-11T13:25:34.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:34 smithi057 ceph-mon[29805]: osdmap e13: 2 total, 2 up, 2 in 2024-06-11T13:25:34.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:34 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-11T13:25:34.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:34 smithi107 ceph-mon[33178]: purged_snaps scrub starts 2024-06-11T13:25:34.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:34 smithi107 ceph-mon[33178]: purged_snaps scrub ok 2024-06-11T13:25:34.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:34 smithi107 ceph-mon[33178]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-11T13:25:34.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:34 smithi107 ceph-mon[33178]: osd.1 v1:172.21.15.57:6805/137889538 boot 2024-06-11T13:25:34.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:34 smithi107 ceph-mon[33178]: osdmap e13: 2 total, 2 up, 2 in 2024-06-11T13:25:34.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:34 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-11T13:25:35.458 INFO:teuthology.orchestra.run.smithi057.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-11T13:25:35.542 INFO:teuthology.orchestra.run.smithi057.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-11T13:25:35.550 INFO:teuthology.orchestra.run.smithi057.stderr: stderr: 10+0 records in 2024-06-11T13:25:35.550 INFO:teuthology.orchestra.run.smithi057.stderr:10+0 records out 2024-06-11T13:25:35.550 INFO:teuthology.orchestra.run.smithi057.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0072324 s, 1.4 GB/s 2024-06-11T13:25:35.551 INFO:teuthology.orchestra.run.smithi057.stderr:--> Zapping successful for: 2024-06-11T13:25:36.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:35 smithi057 ceph-mon[29805]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-11T13:25:36.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:35 smithi057 ceph-mon[29805]: Detected new or changed devices on smithi057 2024-06-11T13:25:36.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:35 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:36.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:35 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:36.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:35 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:25:36.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:35 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:25:36.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:35 smithi057 ceph-mon[29805]: Adjusting osd_memory_target on smithi057 to 8034M 2024-06-11T13:25:36.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:35 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:36.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:35 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:25:36.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:35 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:25:36.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:35 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:36.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:35 smithi057 ceph-mon[29805]: osdmap e14: 2 total, 2 up, 2 in 2024-06-11T13:25:36.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:35 smithi057 ceph-mon[40588]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-11T13:25:36.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:35 smithi057 ceph-mon[40588]: Detected new or changed devices on smithi057 2024-06-11T13:25:36.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:35 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:36.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:35 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:36.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:35 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:25:36.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:35 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:25:36.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:35 smithi057 ceph-mon[40588]: Adjusting osd_memory_target on smithi057 to 8034M 2024-06-11T13:25:36.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:35 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:36.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:35 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:25:36.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:35 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:25:36.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:35 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:36.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:35 smithi057 ceph-mon[40588]: osdmap e14: 2 total, 2 up, 2 in 2024-06-11T13:25:36.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:35 smithi107 ceph-mon[33178]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-11T13:25:36.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:35 smithi107 ceph-mon[33178]: Detected new or changed devices on smithi057 2024-06-11T13:25:36.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:35 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:36.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:35 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:36.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:35 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:25:36.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:35 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:25:36.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:35 smithi107 ceph-mon[33178]: Adjusting osd_memory_target on smithi057 to 8034M 2024-06-11T13:25:36.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:35 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:36.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:35 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:25:36.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:35 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:25:36.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:35 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:36.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:35 smithi107 ceph-mon[33178]: osdmap e14: 2 total, 2 up, 2 in 2024-06-11T13:25:36.203 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi057:vg_nvme/lv_2 2024-06-11T13:25:36.456 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.a/config 2024-06-11T13:25:37.851 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:37 smithi057 ceph-mon[29805]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-11T13:25:37.851 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:37 smithi057 ceph-mon[40588]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-11T13:25:38.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:37 smithi107 ceph-mon[33178]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-11T13:25:38.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:38.200+0000 7f9432f8e640 1 -- 172.21.15.57:0/264762384 >> v1:172.21.15.57:6790/0 conn(0x7f942c104bd0 legacy=0x7f942c107020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:25:38.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:38.200+0000 7f9432f8e640 1 -- 172.21.15.57:0/264762384 shutdown_connections 2024-06-11T13:25:38.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:38.200+0000 7f9432f8e640 1 -- 172.21.15.57:0/264762384 >> 172.21.15.57:0/264762384 conn(0x7f942c0fbd40 msgr2=0x7f942c0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:25:38.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:38.200+0000 7f9432f8e640 1 -- 172.21.15.57:0/264762384 shutdown_connections 2024-06-11T13:25:38.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:38.200+0000 7f9432f8e640 1 -- 172.21.15.57:0/264762384 wait complete. 2024-06-11T13:25:38.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:38.201+0000 7f9432f8e640 1 Processor -- start 2024-06-11T13:25:38.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:38.201+0000 7f9432f8e640 1 -- start start 2024-06-11T13:25:38.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:38.201+0000 7f9432f8e640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f942c197780 con 0x7f942c1007a0 2024-06-11T13:25:38.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:38.201+0000 7f9432f8e640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f942c1a2fd0 con 0x7f942c104bd0 2024-06-11T13:25:38.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:38.201+0000 7f9432f8e640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f942c1a41b0 con 0x7f942c108910 2024-06-11T13:25:38.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:38.201+0000 7f9430d2a640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f942c1007a0 0x7f942c196c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:49220/0 (socket says 172.21.15.57:49220) 2024-06-11T13:25:38.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:38.201+0000 7f9430d2a640 1 -- 172.21.15.57:0/2578331459 learned_addr learned my addr 172.21.15.57:0/2578331459 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:25:38.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:38.202+0000 7f9429ffb640 1 -- 172.21.15.57:0/2578331459 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2925513290 0 0) 0x7f942c1a41b0 con 0x7f942c108910 2024-06-11T13:25:38.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:38.202+0000 7f9429ffb640 1 -- 172.21.15.57:0/2578331459 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f941c003520 con 0x7f942c108910 2024-06-11T13:25:38.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:38.202+0000 7f9429ffb640 1 -- 172.21.15.57:0/2578331459 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2719594282 0 0) 0x7f942c197780 con 0x7f942c1007a0 2024-06-11T13:25:38.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:38.202+0000 7f9429ffb640 1 -- 172.21.15.57:0/2578331459 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f942c1a41b0 con 0x7f942c1007a0 2024-06-11T13:25:38.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:38.202+0000 7f9429ffb640 1 -- 172.21.15.57:0/2578331459 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2363544514 0 0) 0x7f942c1a2fd0 con 0x7f942c104bd0 2024-06-11T13:25:38.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:38.202+0000 7f9429ffb640 1 -- 172.21.15.57:0/2578331459 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f942c197780 con 0x7f942c104bd0 2024-06-11T13:25:38.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:38.202+0000 7f9429ffb640 1 -- 172.21.15.57:0/2578331459 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 618656224 0 0) 0x7f941c003520 con 0x7f942c108910 2024-06-11T13:25:38.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:38.202+0000 7f9429ffb640 1 -- 172.21.15.57:0/2578331459 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f942c1a2fd0 con 0x7f942c108910 2024-06-11T13:25:38.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:38.202+0000 7f9429ffb640 1 -- 172.21.15.57:0/2578331459 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f9424001b70 con 0x7f942c108910 2024-06-11T13:25:38.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:38.202+0000 7f9429ffb640 1 -- 172.21.15.57:0/2578331459 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3561264787 0 0) 0x7f942c1a41b0 con 0x7f942c1007a0 2024-06-11T13:25:38.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:38.202+0000 7f9429ffb640 1 -- 172.21.15.57:0/2578331459 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f941c003520 con 0x7f942c1007a0 2024-06-11T13:25:38.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:38.202+0000 7f9429ffb640 1 -- 172.21.15.57:0/2578331459 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f9420001810 con 0x7f942c1007a0 2024-06-11T13:25:38.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:38.202+0000 7f9429ffb640 1 -- 172.21.15.57:0/2578331459 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2467349654 0 0) 0x7f942c197780 con 0x7f942c104bd0 2024-06-11T13:25:38.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:38.202+0000 7f9429ffb640 1 -- 172.21.15.57:0/2578331459 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f942c1a41b0 con 0x7f942c104bd0 2024-06-11T13:25:38.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:38.202+0000 7f9429ffb640 1 -- 172.21.15.57:0/2578331459 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f94180036f0 con 0x7f942c104bd0 2024-06-11T13:25:38.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:38.202+0000 7f9429ffb640 1 -- 172.21.15.57:0/2578331459 <== mon.2 v1:172.21.15.57:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3521699894 0 0) 0x7f942c1a2fd0 con 0x7f942c108910 2024-06-11T13:25:38.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:38.202+0000 7f9429ffb640 1 -- 172.21.15.57:0/2578331459 >> v1:172.21.15.107:6789/0 conn(0x7f942c104bd0 legacy=0x7f942c19d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:25:38.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:38.202+0000 7f9429ffb640 1 -- 172.21.15.57:0/2578331459 >> v1:172.21.15.57:6789/0 conn(0x7f942c1007a0 legacy=0x7f942c196c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:25:38.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:38.202+0000 7f9429ffb640 1 -- 172.21.15.57:0/2578331459 --> v1:172.21.15.57:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f942c1a5390 con 0x7f942c108910 2024-06-11T13:25:38.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:38.202+0000 7f9432f8e640 1 -- 172.21.15.57:0/2578331459 --> v1:172.21.15.57:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f942c1a43e0 con 0x7f942c108910 2024-06-11T13:25:38.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:38.202+0000 7f9429ffb640 1 -- 172.21.15.57:0/2578331459 <== mon.2 v1:172.21.15.57:6790/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f9424003e00 con 0x7f942c108910 2024-06-11T13:25:38.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:38.202+0000 7f9429ffb640 1 -- 172.21.15.57:0/2578331459 <== mon.2 v1:172.21.15.57:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f9424004d30 con 0x7f942c108910 2024-06-11T13:25:38.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:38.203+0000 7f9432f8e640 1 -- 172.21.15.57:0/2578331459 --> v1:172.21.15.57:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f942c1a4920 con 0x7f942c108910 2024-06-11T13:25:38.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:38.204+0000 7f9432f8e640 1 -- 172.21.15.57:0/2578331459 --> v1:172.21.15.57:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f942c068360 con 0x7f942c108910 2024-06-11T13:25:38.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:38.204+0000 7f9429ffb640 1 -- 172.21.15.57:0/2578331459 <== mon.2 v1:172.21.15.57:6790/0 7 ==== mgrmap(e 14) ==== 92025+0+0 (unknown 3311910117 0 0) 0x7f9424003970 con 0x7f942c108910 2024-06-11T13:25:38.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:38.205+0000 7f9429ffb640 1 -- 172.21.15.57:0/2578331459 <== mon.2 v1:172.21.15.57:6790/0 8 ==== osd_map(14..14 src has 1..14) ==== 1833+0+0 (unknown 54408697 0 0) 0x7f9424087370 con 0x7f942c108910 2024-06-11T13:25:38.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:38.209+0000 7f9429ffb640 1 -- 172.21.15.57:0/2578331459 <== mon.2 v1:172.21.15.57:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f9424055ca0 con 0x7f942c108910 2024-06-11T13:25:38.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:38.403+0000 7f9432f8e640 1 -- 172.21.15.57:0/2578331459 --> v1:172.21.15.57:6800/2989665238 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f942c1a4d20 con 0x7f941c06eda0 2024-06-11T13:25:39.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:38 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-11T13:25:39.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:38 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-11T13:25:39.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:38 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:25:39.093 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:38 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-11T13:25:39.094 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:38 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-11T13:25:39.094 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:38 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:25:39.094 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:38 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-11T13:25:39.094 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:38 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-11T13:25:39.094 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:38 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:25:40.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:39 smithi057 ceph-mon[29805]: from='client.34166 v1:172.21.15.57:0/2578331459' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:25:40.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:39 smithi057 ceph-mon[29805]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-11T13:25:40.032 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:39 smithi057 ceph-mon[40588]: from='client.34166 v1:172.21.15.57:0/2578331459' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:25:40.032 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:39 smithi057 ceph-mon[40588]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-11T13:25:40.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:39 smithi107 ceph-mon[33178]: from='client.34166 v1:172.21.15.57:0/2578331459' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:25:40.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:39 smithi107 ceph-mon[33178]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-11T13:25:42.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:41 smithi057 ceph-mon[29805]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-11T13:25:42.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:41 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1817273168' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0083f7a4-f96d-4efe-a2e9-4c9a4a66c0fa"}]: dispatch 2024-06-11T13:25:42.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:41 smithi057 ceph-mon[29805]: from='client.34172 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0083f7a4-f96d-4efe-a2e9-4c9a4a66c0fa"}]: dispatch 2024-06-11T13:25:42.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:41 smithi057 ceph-mon[29805]: from='client.34172 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0083f7a4-f96d-4efe-a2e9-4c9a4a66c0fa"}]': finished 2024-06-11T13:25:42.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:41 smithi057 ceph-mon[29805]: osdmap e15: 3 total, 2 up, 3 in 2024-06-11T13:25:42.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:41 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-11T13:25:42.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:41 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3817604737' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-11T13:25:42.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:41 smithi057 ceph-mon[40588]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-11T13:25:42.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:41 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1817273168' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0083f7a4-f96d-4efe-a2e9-4c9a4a66c0fa"}]: dispatch 2024-06-11T13:25:42.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:41 smithi057 ceph-mon[40588]: from='client.34172 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0083f7a4-f96d-4efe-a2e9-4c9a4a66c0fa"}]: dispatch 2024-06-11T13:25:42.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:41 smithi057 ceph-mon[40588]: from='client.34172 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0083f7a4-f96d-4efe-a2e9-4c9a4a66c0fa"}]': finished 2024-06-11T13:25:42.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:41 smithi057 ceph-mon[40588]: osdmap e15: 3 total, 2 up, 3 in 2024-06-11T13:25:42.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:41 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-11T13:25:42.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:41 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3817604737' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-11T13:25:42.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:41 smithi107 ceph-mon[33178]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-11T13:25:42.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:41 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1817273168' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0083f7a4-f96d-4efe-a2e9-4c9a4a66c0fa"}]: dispatch 2024-06-11T13:25:42.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:41 smithi107 ceph-mon[33178]: from='client.34172 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0083f7a4-f96d-4efe-a2e9-4c9a4a66c0fa"}]: dispatch 2024-06-11T13:25:42.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:41 smithi107 ceph-mon[33178]: from='client.34172 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0083f7a4-f96d-4efe-a2e9-4c9a4a66c0fa"}]': finished 2024-06-11T13:25:42.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:41 smithi107 ceph-mon[33178]: osdmap e15: 3 total, 2 up, 3 in 2024-06-11T13:25:42.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:41 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-11T13:25:42.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:41 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3817604737' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-11T13:25:44.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:43 smithi057 ceph-mon[29805]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-11T13:25:44.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:43 smithi057 ceph-mon[40588]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-11T13:25:44.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:43 smithi107 ceph-mon[33178]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-11T13:25:46.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:45 smithi057 ceph-mon[29805]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-11T13:25:46.032 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:45 smithi057 ceph-mon[40588]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-11T13:25:46.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:45 smithi107 ceph-mon[33178]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-11T13:25:48.172 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:47 smithi057 ceph-mon[29805]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-11T13:25:48.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:47 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-11T13:25:48.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:47 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:25:48.173 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:47 smithi057 ceph-mon[29805]: Deploying daemon osd.2 on smithi057 2024-06-11T13:25:48.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:47 smithi057 ceph-mon[40588]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-11T13:25:48.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:47 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-11T13:25:48.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:47 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:25:48.173 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:47 smithi057 ceph-mon[40588]: Deploying daemon osd.2 on smithi057 2024-06-11T13:25:48.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:47 smithi107 ceph-mon[33178]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-11T13:25:48.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:47 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-11T13:25:48.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:47 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:25:48.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:47 smithi107 ceph-mon[33178]: Deploying daemon osd.2 on smithi057 2024-06-11T13:25:50.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:49 smithi057 ceph-mon[29805]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-11T13:25:50.032 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:49 smithi057 ceph-mon[40588]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-11T13:25:50.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:49 smithi107 ceph-mon[33178]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-11T13:25:52.127 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:51 smithi057 ceph-mon[29805]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-11T13:25:52.128 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:51 smithi057 ceph-mon[40588]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-11T13:25:52.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:51 smithi107 ceph-mon[33178]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-11T13:25:53.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:52 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:25:53.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:52 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:53.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:52 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:53.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:52 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:25:53.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:52 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:53.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:52 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:53.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:52 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:25:53.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:52 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:53.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:52 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:54.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:53 smithi057 ceph-mon[29805]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-11T13:25:54.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:53 smithi057 ceph-mon[40588]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-11T13:25:54.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:53 smithi107 ceph-mon[33178]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-11T13:25:55.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:55.936+0000 7f9429ffb640 1 -- 172.21.15.57:0/2578331459 <== mgr.14150 v1:172.21.15.57:6800/2989665238 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (unknown 0 0 1982711099) 0x7f942c1a4d20 con 0x7f941c06eda0 2024-06-11T13:25:55.932 INFO:teuthology.orchestra.run.smithi057.stdout:Created osd(s) 2 on host 'smithi057' 2024-06-11T13:25:55.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:55.939+0000 7f9432f8e640 1 -- 172.21.15.57:0/2578331459 >> v1:172.21.15.57:6800/2989665238 conn(0x7f941c06eda0 legacy=0x7f941c071260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:25:55.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:55.939+0000 7f9432f8e640 1 -- 172.21.15.57:0/2578331459 >> v1:172.21.15.57:6790/0 conn(0x7f942c108910 legacy=0x7f942c1a08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:25:55.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:55.939+0000 7f9432f8e640 1 -- 172.21.15.57:0/2578331459 shutdown_connections 2024-06-11T13:25:55.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:55.939+0000 7f9432f8e640 1 -- 172.21.15.57:0/2578331459 >> 172.21.15.57:0/2578331459 conn(0x7f942c0fbd40 msgr2=0x7f942c06a570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:25:55.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:55.939+0000 7f9432f8e640 1 -- 172.21.15.57:0/2578331459 shutdown_connections 2024-06-11T13:25:55.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:25:55.939+0000 7f9432f8e640 1 -- 172.21.15.57:0/2578331459 wait complete. 2024-06-11T13:25:56.242 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:56 smithi057 ceph-mon[40588]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-11T13:25:56.242 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:56 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:56.242 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:56 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:56.242 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:56 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:25:56.242 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:56 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:25:56.242 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:56 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:56.242 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:56 smithi057 ceph-mon[40588]: from='osd.2 v1:172.21.15.57:6809/385321295' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-11T13:25:56.243 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:56 smithi057 ceph-mon[40588]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-11T13:25:56.243 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:56 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:25:56.243 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:56 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:56.243 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:56 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:56.243 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:56 smithi057 ceph-mon[29805]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-11T13:25:56.243 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:56 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:56.243 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:56 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:56.243 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:56 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:25:56.243 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:56 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:25:56.243 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:56 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:56.243 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:56 smithi057 ceph-mon[29805]: from='osd.2 v1:172.21.15.57:6809/385321295' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-11T13:25:56.243 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:56 smithi057 ceph-mon[29805]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-11T13:25:56.244 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:56 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:25:56.244 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:56 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:56.244 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:56 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:56.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:56 smithi107 ceph-mon[33178]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-11T13:25:56.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:56 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:56.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:56 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:56.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:56 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:25:56.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:56 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:25:56.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:56 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:56.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:56 smithi107 ceph-mon[33178]: from='osd.2 v1:172.21.15.57:6809/385321295' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-11T13:25:56.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:56 smithi107 ceph-mon[33178]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-11T13:25:56.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:56 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:25:56.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:56 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:56.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:56 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:25:56.545 DEBUG:teuthology.orchestra.run.smithi057:osd.2> sudo journalctl -f -n 0 -u ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@osd.2.service 2024-06-11T13:25:56.548 INFO:tasks.cephadm:Deploying osd.3 on smithi057 with /dev/vg_nvme/lv_1... 2024-06-11T13:25:56.548 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-11T13:25:57.318 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.a/config 2024-06-11T13:25:57.371 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:57 smithi057 ceph-mon[40588]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-11T13:25:57.371 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:57 smithi057 ceph-mon[40588]: osdmap e16: 3 total, 2 up, 3 in 2024-06-11T13:25:57.371 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:57 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-11T13:25:57.371 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:57 smithi057 ceph-mon[40588]: from='osd.2 v1:172.21.15.57:6809/385321295' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-06-11T13:25:57.371 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:57 smithi057 ceph-mon[40588]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-06-11T13:25:57.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:57 smithi057 ceph-mon[29805]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-11T13:25:57.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:57 smithi057 ceph-mon[29805]: osdmap e16: 3 total, 2 up, 3 in 2024-06-11T13:25:57.372 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:57 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-11T13:25:57.372 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:57 smithi057 ceph-mon[29805]: from='osd.2 v1:172.21.15.57:6809/385321295' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-06-11T13:25:57.372 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:57 smithi057 ceph-mon[29805]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-06-11T13:25:57.545 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:57 smithi107 ceph-mon[33178]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-11T13:25:57.545 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:57 smithi107 ceph-mon[33178]: osdmap e16: 3 total, 2 up, 3 in 2024-06-11T13:25:57.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:57 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-11T13:25:57.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:57 smithi107 ceph-mon[33178]: from='osd.2 v1:172.21.15.57:6809/385321295' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-06-11T13:25:57.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:57 smithi107 ceph-mon[33178]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-06-11T13:25:57.733 INFO:journalctl@ceph.osd.2.smithi057.stdout:Jun 11 13:25:57 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-2[56152]: 2024-06-11T13:25:57.374+0000 7fc96e10d640 -1 osd.2 0 waiting for initial osdmap 2024-06-11T13:25:57.733 INFO:journalctl@ceph.osd.2.smithi057.stdout:Jun 11 13:25:57 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-2[56152]: 2024-06-11T13:25:57.381+0000 7fc965f2d640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-11T13:25:58.262 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:58 smithi057 ceph-mon[29805]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-11T13:25:58.263 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:58 smithi057 ceph-mon[29805]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2024-06-11T13:25:58.263 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:58 smithi057 ceph-mon[29805]: osdmap e17: 3 total, 2 up, 3 in 2024-06-11T13:25:58.263 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:58 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-11T13:25:58.263 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:58 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-11T13:25:58.263 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:58 smithi057 ceph-mon[29805]: from='osd.2 ' entity='osd.2' 2024-06-11T13:25:58.263 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:58 smithi057 ceph-mon[40588]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-11T13:25:58.263 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:58 smithi057 ceph-mon[40588]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2024-06-11T13:25:58.263 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:58 smithi057 ceph-mon[40588]: osdmap e17: 3 total, 2 up, 3 in 2024-06-11T13:25:58.263 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:58 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-11T13:25:58.263 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:58 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-11T13:25:58.263 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:58 smithi057 ceph-mon[40588]: from='osd.2 ' entity='osd.2' 2024-06-11T13:25:58.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:58 smithi107 ceph-mon[33178]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-11T13:25:58.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:58 smithi107 ceph-mon[33178]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2024-06-11T13:25:58.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:58 smithi107 ceph-mon[33178]: osdmap e17: 3 total, 2 up, 3 in 2024-06-11T13:25:58.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:58 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-11T13:25:58.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:58 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-11T13:25:58.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:58 smithi107 ceph-mon[33178]: from='osd.2 ' entity='osd.2' 2024-06-11T13:25:59.271 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:59 smithi057 ceph-mon[29805]: purged_snaps scrub starts 2024-06-11T13:25:59.271 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:59 smithi057 ceph-mon[29805]: purged_snaps scrub ok 2024-06-11T13:25:59.271 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:59 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-11T13:25:59.271 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:59 smithi057 ceph-mon[29805]: osd.2 v1:172.21.15.57:6809/385321295 boot 2024-06-11T13:25:59.271 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:59 smithi057 ceph-mon[29805]: osdmap e18: 3 total, 3 up, 3 in 2024-06-11T13:25:59.271 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:59 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-11T13:25:59.271 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:25:59 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:25:59.271 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:59 smithi057 ceph-mon[40588]: purged_snaps scrub starts 2024-06-11T13:25:59.271 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:59 smithi057 ceph-mon[40588]: purged_snaps scrub ok 2024-06-11T13:25:59.272 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:59 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-11T13:25:59.272 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:59 smithi057 ceph-mon[40588]: osd.2 v1:172.21.15.57:6809/385321295 boot 2024-06-11T13:25:59.272 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:59 smithi057 ceph-mon[40588]: osdmap e18: 3 total, 3 up, 3 in 2024-06-11T13:25:59.272 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:59 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-11T13:25:59.272 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:25:59 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:25:59.446 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:59 smithi107 ceph-mon[33178]: purged_snaps scrub starts 2024-06-11T13:25:59.447 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:59 smithi107 ceph-mon[33178]: purged_snaps scrub ok 2024-06-11T13:25:59.447 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:59 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-11T13:25:59.447 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:59 smithi107 ceph-mon[33178]: osd.2 v1:172.21.15.57:6809/385321295 boot 2024-06-11T13:25:59.447 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:59 smithi107 ceph-mon[33178]: osdmap e18: 3 total, 3 up, 3 in 2024-06-11T13:25:59.447 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:59 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-11T13:25:59.447 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:25:59 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:26:00.215 INFO:teuthology.orchestra.run.smithi057.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-11T13:26:00.301 INFO:teuthology.orchestra.run.smithi057.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-11T13:26:00.317 INFO:teuthology.orchestra.run.smithi057.stderr: stderr: 10+0 records in 2024-06-11T13:26:00.317 INFO:teuthology.orchestra.run.smithi057.stderr:10+0 records out 2024-06-11T13:26:00.317 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:00 smithi057 ceph-mon[40588]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-11T13:26:00.317 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:00 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-11T13:26:00.318 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:00 smithi057 ceph-mon[40588]: osdmap e19: 3 total, 3 up, 3 in 2024-06-11T13:26:00.318 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:00 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:26:00.318 INFO:teuthology.orchestra.run.smithi057.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.013876 s, 756 MB/s 2024-06-11T13:26:00.318 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:00 smithi057 ceph-mon[29805]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-11T13:26:00.318 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:00 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-11T13:26:00.318 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:00 smithi057 ceph-mon[29805]: osdmap e19: 3 total, 3 up, 3 in 2024-06-11T13:26:00.318 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:00 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:26:00.319 INFO:teuthology.orchestra.run.smithi057.stderr:--> Zapping successful for: 2024-06-11T13:26:00.545 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:00 smithi107 ceph-mon[33178]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-06-11T13:26:00.545 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:00 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-11T13:26:00.545 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:00 smithi107 ceph-mon[33178]: osdmap e19: 3 total, 3 up, 3 in 2024-06-11T13:26:00.545 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:00 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:26:01.032 INFO:journalctl@ceph.osd.0.smithi057.stdout:Jun 11 13:26:00 smithi057 sudo[59756]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-11T13:26:01.032 INFO:journalctl@ceph.osd.0.smithi057.stdout:Jun 11 13:26:00 smithi057 sudo[59756]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-11T13:26:01.032 INFO:journalctl@ceph.osd.0.smithi057.stdout:Jun 11 13:26:00 smithi057 sudo[59756]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-06-11T13:26:01.032 INFO:journalctl@ceph.osd.0.smithi057.stdout:Jun 11 13:26:00 smithi057 sudo[59756]: pam_unix(sudo:session): session closed for user root 2024-06-11T13:26:01.050 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi057:vg_nvme/lv_1 2024-06-11T13:26:01.317 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.a/config 2024-06-11T13:26:01.687 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:01 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-11T13:26:01.687 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:01 smithi057 ceph-mon[29805]: osdmap e20: 3 total, 3 up, 3 in 2024-06-11T13:26:01.687 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:01 smithi057 ceph-mon[29805]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-06-11T13:26:01.687 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:01 smithi057 ceph-mon[29805]: Detected new or changed devices on smithi057 2024-06-11T13:26:01.687 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:01 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:01.687 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:01 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:01.687 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:01 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:26:01.687 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:01 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:26:01.687 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:01 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:26:01.688 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:01 smithi057 ceph-mon[29805]: Adjusting osd_memory_target on smithi057 to 5356M 2024-06-11T13:26:01.688 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:01 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:01.688 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:01 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:26:01.688 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:01 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:26:01.688 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:01 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:01.688 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:01 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-11T13:26:01.688 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:01 smithi057 ceph-mon[40588]: osdmap e20: 3 total, 3 up, 3 in 2024-06-11T13:26:01.688 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:01 smithi057 ceph-mon[40588]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-06-11T13:26:01.688 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:01 smithi057 ceph-mon[40588]: Detected new or changed devices on smithi057 2024-06-11T13:26:01.688 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:01 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:01.688 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:01 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:01.689 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:01 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:26:01.689 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:01 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:26:01.689 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:01 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:26:01.689 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:01 smithi057 ceph-mon[40588]: Adjusting osd_memory_target on smithi057 to 5356M 2024-06-11T13:26:01.689 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:01 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:01.689 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:01 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:26:01.689 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:01 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:26:01.689 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:01 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:01.689 INFO:journalctl@ceph.osd.0.smithi057.stdout:Jun 11 13:26:01 smithi057 sudo[59830]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-11T13:26:01.689 INFO:journalctl@ceph.osd.0.smithi057.stdout:Jun 11 13:26:01 smithi057 sudo[59830]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-11T13:26:01.689 INFO:journalctl@ceph.osd.0.smithi057.stdout:Jun 11 13:26:01 smithi057 sudo[59830]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-06-11T13:26:01.689 INFO:journalctl@ceph.osd.0.smithi057.stdout:Jun 11 13:26:01 smithi057 sudo[59830]: pam_unix(sudo:session): session closed for user root 2024-06-11T13:26:01.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:01 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-11T13:26:01.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:01 smithi107 ceph-mon[33178]: osdmap e20: 3 total, 3 up, 3 in 2024-06-11T13:26:01.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:01 smithi107 ceph-mon[33178]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-06-11T13:26:01.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:01 smithi107 ceph-mon[33178]: Detected new or changed devices on smithi057 2024-06-11T13:26:01.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:01 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:01.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:01 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:01.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:01 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:26:01.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:01 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:26:01.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:01 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:26:01.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:01 smithi107 ceph-mon[33178]: Adjusting osd_memory_target on smithi057 to 5356M 2024-06-11T13:26:01.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:01 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:01.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:01 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:26:01.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:01 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:26:01.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:01 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:02.032 INFO:journalctl@ceph.osd.1.smithi057.stdout:Jun 11 13:26:01 smithi057 sudo[59860]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-11T13:26:02.032 INFO:journalctl@ceph.osd.1.smithi057.stdout:Jun 11 13:26:01 smithi057 sudo[59860]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-11T13:26:02.032 INFO:journalctl@ceph.osd.1.smithi057.stdout:Jun 11 13:26:01 smithi057 sudo[59860]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-06-11T13:26:02.032 INFO:journalctl@ceph.osd.1.smithi057.stdout:Jun 11 13:26:01 smithi057 sudo[59860]: pam_unix(sudo:session): session closed for user root 2024-06-11T13:26:02.411 INFO:journalctl@ceph.osd.1.smithi057.stdout:Jun 11 13:26:02 smithi057 sudo[59866]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-11T13:26:02.411 INFO:journalctl@ceph.osd.1.smithi057.stdout:Jun 11 13:26:02 smithi057 sudo[59866]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-11T13:26:02.411 INFO:journalctl@ceph.osd.1.smithi057.stdout:Jun 11 13:26:02 smithi057 sudo[59866]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-06-11T13:26:02.411 INFO:journalctl@ceph.osd.1.smithi057.stdout:Jun 11 13:26:02 smithi057 sudo[59866]: pam_unix(sudo:session): session closed for user root 2024-06-11T13:26:02.717 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:02 smithi057 ceph-mon[40588]: mgrmap e15: y(active, since 2m), standbys: x 2024-06-11T13:26:02.717 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:02 smithi057 ceph-mon[40588]: osdmap e21: 3 total, 3 up, 3 in 2024-06-11T13:26:02.718 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:02 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:02.718 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:02 smithi057 ceph-mon[29805]: mgrmap e15: y(active, since 2m), standbys: x 2024-06-11T13:26:02.718 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:02 smithi057 ceph-mon[29805]: osdmap e21: 3 total, 3 up, 3 in 2024-06-11T13:26:02.718 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:02 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:02.795 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:02 smithi107 ceph-mon[33178]: mgrmap e15: y(active, since 2m), standbys: x 2024-06-11T13:26:02.795 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:02 smithi107 ceph-mon[33178]: osdmap e21: 3 total, 3 up, 3 in 2024-06-11T13:26:02.795 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:02 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:02.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:26:02.925+0000 7ff4778a2640 1 -- 172.21.15.57:0/371702813 >> v1:172.21.15.57:6790/0 conn(0x7ff4701000f0 legacy=0x7ff470100510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:26:02.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:26:02.925+0000 7ff4778a2640 1 -- 172.21.15.57:0/371702813 shutdown_connections 2024-06-11T13:26:02.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:26:02.925+0000 7ff4778a2640 1 -- 172.21.15.57:0/371702813 >> 172.21.15.57:0/371702813 conn(0x7ff4700fbd10 msgr2=0x7ff4700fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:26:02.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:26:02.925+0000 7ff4778a2640 1 -- 172.21.15.57:0/371702813 shutdown_connections 2024-06-11T13:26:02.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:26:02.925+0000 7ff4778a2640 1 -- 172.21.15.57:0/371702813 wait complete. 2024-06-11T13:26:02.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:26:02.925+0000 7ff4778a2640 1 Processor -- start 2024-06-11T13:26:02.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:26:02.926+0000 7ff4778a2640 1 -- start start 2024-06-11T13:26:02.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:26:02.926+0000 7ff4778a2640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ff470199910 con 0x7ff47010be50 2024-06-11T13:26:02.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:26:02.926+0000 7ff4778a2640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ff4701a5160 con 0x7ff470101c30 2024-06-11T13:26:02.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:26:02.926+0000 7ff4778a2640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ff4701a6340 con 0x7ff4701000f0 2024-06-11T13:26:02.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:26:02.926+0000 7ff47563e640 1 --1- >> v1:172.21.15.57:6790/0 conn(0x7ff4701000f0 0x7ff470198e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6790/0 says I am v1:172.21.15.57:37964/0 (socket says 172.21.15.57:37964) 2024-06-11T13:26:02.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:26:02.926+0000 7ff47563e640 1 -- 172.21.15.57:0/2575242374 learned_addr learned my addr 172.21.15.57:0/2575242374 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:26:02.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:26:02.926+0000 7ff4667fc640 1 -- 172.21.15.57:0/2575242374 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2140091854 0 0) 0x7ff470199910 con 0x7ff47010be50 2024-06-11T13:26:02.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:26:02.926+0000 7ff4667fc640 1 -- 172.21.15.57:0/2575242374 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff45c0051b0 con 0x7ff47010be50 2024-06-11T13:26:02.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:26:02.926+0000 7ff4667fc640 1 -- 172.21.15.57:0/2575242374 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 864713562 0 0) 0x7ff4701a6340 con 0x7ff4701000f0 2024-06-11T13:26:02.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:26:02.926+0000 7ff4667fc640 1 -- 172.21.15.57:0/2575242374 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff470199910 con 0x7ff4701000f0 2024-06-11T13:26:02.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:26:02.926+0000 7ff4667fc640 1 -- 172.21.15.57:0/2575242374 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3854504433 0 0) 0x7ff45c0051b0 con 0x7ff47010be50 2024-06-11T13:26:02.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:26:02.926+0000 7ff4667fc640 1 -- 172.21.15.57:0/2575242374 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff4701a6340 con 0x7ff47010be50 2024-06-11T13:26:02.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:26:02.926+0000 7ff4667fc640 1 -- 172.21.15.57:0/2575242374 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 937884709 0 0) 0x7ff470199910 con 0x7ff4701000f0 2024-06-11T13:26:02.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:26:02.926+0000 7ff4667fc640 1 -- 172.21.15.57:0/2575242374 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff45c0051b0 con 0x7ff4701000f0 2024-06-11T13:26:02.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:26:02.926+0000 7ff4667fc640 1 -- 172.21.15.57:0/2575242374 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4077043121 0 0) 0x7ff4701a5160 con 0x7ff470101c30 2024-06-11T13:26:02.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:26:02.926+0000 7ff4667fc640 1 -- 172.21.15.57:0/2575242374 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff470199910 con 0x7ff470101c30 2024-06-11T13:26:02.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:26:02.926+0000 7ff4667fc640 1 -- 172.21.15.57:0/2575242374 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7ff468001c30 con 0x7ff47010be50 2024-06-11T13:26:02.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:26:02.926+0000 7ff4667fc640 1 -- 172.21.15.57:0/2575242374 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7ff46c004390 con 0x7ff4701000f0 2024-06-11T13:26:02.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:26:02.927+0000 7ff4667fc640 1 -- 172.21.15.57:0/2575242374 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1718379367 0 0) 0x7ff4701a6340 con 0x7ff47010be50 2024-06-11T13:26:02.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:26:02.927+0000 7ff4667fc640 1 -- 172.21.15.57:0/2575242374 >> v1:172.21.15.57:6790/0 conn(0x7ff4701000f0 legacy=0x7ff470198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:26:02.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:26:02.927+0000 7ff4667fc640 1 -- 172.21.15.57:0/2575242374 >> v1:172.21.15.107:6789/0 conn(0x7ff470101c30 legacy=0x7ff47019f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:26:02.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:26:02.927+0000 7ff4667fc640 1 -- 172.21.15.57:0/2575242374 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4701a7520 con 0x7ff47010be50 2024-06-11T13:26:02.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:26:02.927+0000 7ff4778a2640 1 -- 172.21.15.57:0/2575242374 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7ff4701a41b0 con 0x7ff47010be50 2024-06-11T13:26:02.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:26:02.927+0000 7ff4778a2640 1 -- 172.21.15.57:0/2575242374 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7ff4701a46f0 con 0x7ff47010be50 2024-06-11T13:26:02.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:26:02.927+0000 7ff4667fc640 1 -- 172.21.15.57:0/2575242374 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7ff468003e20 con 0x7ff47010be50 2024-06-11T13:26:02.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:26:02.927+0000 7ff4667fc640 1 -- 172.21.15.57:0/2575242374 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7ff468004d20 con 0x7ff47010be50 2024-06-11T13:26:02.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:26:02.928+0000 7ff4778a2640 1 -- 172.21.15.57:0/2575242374 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4540050b0 con 0x7ff47010be50 2024-06-11T13:26:02.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:26:02.928+0000 7ff4667fc640 1 -- 172.21.15.57:0/2575242374 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 15) ==== 92081+0+0 (unknown 2221930365 0 0) 0x7ff468003970 con 0x7ff47010be50 2024-06-11T13:26:02.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:26:02.929+0000 7ff4667fc640 1 -- 172.21.15.57:0/2575242374 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(21..21 src has 1..21) ==== 2511+0+0 (unknown 797114522 0 0) 0x7ff4680876e0 con 0x7ff47010be50 2024-06-11T13:26:02.945 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:26:02.933+0000 7ff4667fc640 1 -- 172.21.15.57:0/2575242374 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7ff468055c90 con 0x7ff47010be50 2024-06-11T13:26:03.032 INFO:journalctl@ceph.osd.2.smithi057.stdout:Jun 11 13:26:02 smithi057 sudo[59894]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-11T13:26:03.032 INFO:journalctl@ceph.osd.2.smithi057.stdout:Jun 11 13:26:02 smithi057 sudo[59894]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-11T13:26:03.032 INFO:journalctl@ceph.osd.2.smithi057.stdout:Jun 11 13:26:02 smithi057 sudo[59894]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-06-11T13:26:03.032 INFO:journalctl@ceph.osd.2.smithi057.stdout:Jun 11 13:26:02 smithi057 sudo[59894]: pam_unix(sudo:session): session closed for user root 2024-06-11T13:26:03.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:26:03.127+0000 7ff4778a2640 1 -- 172.21.15.57:0/2575242374 --> v1:172.21.15.57:6800/2989665238 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7ff454000bc0 con 0x7ff45c06ed80 2024-06-11T13:26:03.532 INFO:journalctl@ceph.osd.2.smithi057.stdout:Jun 11 13:26:03 smithi057 sudo[59985]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-11T13:26:03.532 INFO:journalctl@ceph.osd.2.smithi057.stdout:Jun 11 13:26:03 smithi057 sudo[59985]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-11T13:26:03.532 INFO:journalctl@ceph.osd.2.smithi057.stdout:Jun 11 13:26:03 smithi057 sudo[59985]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-06-11T13:26:03.532 INFO:journalctl@ceph.osd.2.smithi057.stdout:Jun 11 13:26:03 smithi057 sudo[59985]: pam_unix(sudo:session): session closed for user root 2024-06-11T13:26:03.820 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:03 smithi057 sudo[60042]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-11T13:26:03.820 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:03 smithi057 sudo[60042]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-11T13:26:03.820 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:03 smithi057 sudo[60042]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-06-11T13:26:03.820 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:03 smithi057 sudo[60042]: pam_unix(sudo:session): session closed for user root 2024-06-11T13:26:04.432 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:04 smithi107 ceph-mon[33178]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-06-11T13:26:04.433 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:04 smithi107 ceph-mon[33178]: from='client.14301 v1:172.21.15.57:0/2575242374' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:26:04.433 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:04 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-11T13:26:04.433 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:04 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-11T13:26:04.433 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:04 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:26:04.433 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:04 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-11T13:26:04.433 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:04 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-11T13:26:04.433 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:04 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-11T13:26:04.433 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:04 smithi107 sudo[35663]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-11T13:26:04.433 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:04 smithi107 sudo[35663]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-11T13:26:04.433 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:04 smithi107 sudo[35663]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-06-11T13:26:04.433 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:04 smithi107 sudo[35663]: pam_unix(sudo:session): session closed for user root 2024-06-11T13:26:04.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:04 smithi057 ceph-mon[40588]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-06-11T13:26:04.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:04 smithi057 ceph-mon[40588]: from='client.14301 v1:172.21.15.57:0/2575242374' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:26:04.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:04 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-11T13:26:04.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:04 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-11T13:26:04.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:04 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:26:04.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:04 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-11T13:26:04.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:04 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-11T13:26:04.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:04 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-11T13:26:04.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:04 smithi057 sudo[60065]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-11T13:26:04.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:04 smithi057 sudo[60065]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-11T13:26:04.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:04 smithi057 sudo[60065]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-06-11T13:26:04.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:04 smithi057 sudo[60065]: pam_unix(sudo:session): session closed for user root 2024-06-11T13:26:04.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:04 smithi057 ceph-mon[29805]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-06-11T13:26:04.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:04 smithi057 ceph-mon[29805]: from='client.14301 v1:172.21.15.57:0/2575242374' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:26:04.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:04 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-11T13:26:04.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:04 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-11T13:26:04.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:04 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:26:04.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:04 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-11T13:26:04.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:04 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-11T13:26:04.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:04 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-11T13:26:04.795 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:04 smithi107 sudo[35666]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-11T13:26:04.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:04 smithi107 sudo[35666]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-11T13:26:04.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:04 smithi107 sudo[35666]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-06-11T13:26:04.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:04 smithi107 sudo[35666]: pam_unix(sudo:session): session closed for user root 2024-06-11T13:26:05.532 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:05 smithi057 sudo[60097]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-11T13:26:05.532 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:05 smithi057 sudo[60097]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-11T13:26:05.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:05 smithi057 sudo[60097]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-06-11T13:26:05.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:05 smithi057 sudo[60097]: pam_unix(sudo:session): session closed for user root 2024-06-11T13:26:05.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:05 smithi057 sudo[60139]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-11T13:26:05.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:05 smithi057 sudo[60139]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-11T13:26:05.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:05 smithi057 sudo[60139]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-06-11T13:26:05.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:05 smithi057 sudo[60139]: pam_unix(sudo:session): session closed for user root 2024-06-11T13:26:05.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:05 smithi057 ceph-mon[40588]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-11T13:26:05.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:05 smithi057 ceph-mon[40588]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-11T13:26:05.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:05 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-11T13:26:05.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:05 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-11T13:26:05.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:05 smithi057 ceph-mon[40588]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-11T13:26:05.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:05 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-11T13:26:05.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:05 smithi057 ceph-mon[40588]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-11T13:26:05.931 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:05 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:05.932 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:05 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-11T13:26:05.932 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:05 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-11T13:26:05.932 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:05 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-11T13:26:05.932 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:05 smithi057 ceph-mon[29805]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-11T13:26:05.932 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:05 smithi057 ceph-mon[29805]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-11T13:26:05.932 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:05 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-11T13:26:05.932 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:05 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-11T13:26:05.932 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:05 smithi057 ceph-mon[29805]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-11T13:26:05.932 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:05 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-11T13:26:05.932 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:05 smithi057 ceph-mon[29805]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-11T13:26:05.932 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:05 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:05.933 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:05 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-11T13:26:05.933 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:05 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-11T13:26:05.933 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:05 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-11T13:26:06.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:05 smithi107 ceph-mon[33178]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-11T13:26:06.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:05 smithi107 ceph-mon[33178]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-11T13:26:06.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:05 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-11T13:26:06.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:05 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-11T13:26:06.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:05 smithi107 ceph-mon[33178]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-11T13:26:06.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:05 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-11T13:26:06.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:05 smithi107 ceph-mon[33178]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-11T13:26:06.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:05 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:06.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:05 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-11T13:26:06.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:05 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-11T13:26:06.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:05 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-11T13:26:07.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:06 smithi057 ceph-mon[29805]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-11T13:26:07.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:06 smithi057 ceph-mon[29805]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-11T13:26:07.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:06 smithi057 ceph-mon[29805]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-11T13:26:07.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:06 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2070148484' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "21aeff39-ee19-45ed-bb85-51827aa0a89e"}]: dispatch 2024-06-11T13:26:07.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:06 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2070148484' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "21aeff39-ee19-45ed-bb85-51827aa0a89e"}]': finished 2024-06-11T13:26:07.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:06 smithi057 ceph-mon[29805]: osdmap e22: 4 total, 3 up, 4 in 2024-06-11T13:26:07.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:06 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-11T13:26:07.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:06 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/701597520' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-11T13:26:07.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:06 smithi057 ceph-mon[40588]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-11T13:26:07.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:06 smithi057 ceph-mon[40588]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-11T13:26:07.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:06 smithi057 ceph-mon[40588]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-11T13:26:07.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:06 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2070148484' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "21aeff39-ee19-45ed-bb85-51827aa0a89e"}]: dispatch 2024-06-11T13:26:07.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:06 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2070148484' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "21aeff39-ee19-45ed-bb85-51827aa0a89e"}]': finished 2024-06-11T13:26:07.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:06 smithi057 ceph-mon[40588]: osdmap e22: 4 total, 3 up, 4 in 2024-06-11T13:26:07.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:06 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-11T13:26:07.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:06 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/701597520' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-11T13:26:07.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:06 smithi107 ceph-mon[33178]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-11T13:26:07.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:06 smithi107 ceph-mon[33178]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-11T13:26:07.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:06 smithi107 ceph-mon[33178]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-11T13:26:07.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:06 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2070148484' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "21aeff39-ee19-45ed-bb85-51827aa0a89e"}]: dispatch 2024-06-11T13:26:07.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:06 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2070148484' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "21aeff39-ee19-45ed-bb85-51827aa0a89e"}]': finished 2024-06-11T13:26:07.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:06 smithi107 ceph-mon[33178]: osdmap e22: 4 total, 3 up, 4 in 2024-06-11T13:26:07.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:06 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-11T13:26:07.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:06 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/701597520' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-11T13:26:08.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:07 smithi057 ceph-mon[29805]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-11T13:26:08.032 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:07 smithi057 ceph-mon[40588]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-11T13:26:08.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:07 smithi107 ceph-mon[33178]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-11T13:26:09.927 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:09 smithi057 ceph-mon[29805]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-11T13:26:09.927 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:09 smithi057 ceph-mon[40588]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-11T13:26:10.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:09 smithi107 ceph-mon[33178]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-11T13:26:11.862 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:11 smithi057 ceph-mon[29805]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-06-11T13:26:11.863 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:11 smithi057 ceph-mon[40588]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-06-11T13:26:12.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:11 smithi107 ceph-mon[33178]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-06-11T13:26:12.944 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:12 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-11T13:26:12.944 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:12 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:26:12.944 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:12 smithi057 ceph-mon[29805]: Deploying daemon osd.3 on smithi057 2024-06-11T13:26:12.944 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:12 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-11T13:26:12.944 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:12 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:26:12.944 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:12 smithi057 ceph-mon[40588]: Deploying daemon osd.3 on smithi057 2024-06-11T13:26:13.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:12 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-11T13:26:13.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:12 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:26:13.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:12 smithi107 ceph-mon[33178]: Deploying daemon osd.3 on smithi057 2024-06-11T13:26:13.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:13 smithi057 ceph-mon[29805]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-06-11T13:26:13.782 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:13 smithi057 ceph-mon[40588]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-06-11T13:26:14.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:13 smithi107 ceph-mon[33178]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-06-11T13:26:15.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:15 smithi057 ceph-mon[29805]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-06-11T13:26:15.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:15 smithi057 ceph-mon[40588]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-06-11T13:26:16.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:15 smithi107 ceph-mon[33178]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-06-11T13:26:17.917 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:17 smithi057 ceph-mon[40588]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 73 KiB/s wr, 5 op/s 2024-06-11T13:26:17.917 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:17 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:26:17.917 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:17 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:17.917 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:17 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:17.918 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:17 smithi057 ceph-mon[29805]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 73 KiB/s wr, 5 op/s 2024-06-11T13:26:17.918 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:17 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:26:17.918 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:17 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:17.918 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:17 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:18.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:17 smithi107 ceph-mon[33178]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 73 KiB/s wr, 5 op/s 2024-06-11T13:26:18.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:17 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:26:18.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:17 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:18.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:17 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:19.945 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:19 smithi057 ceph-mon[40588]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-06-11T13:26:19.945 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:19 smithi057 ceph-mon[29805]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-06-11T13:26:20.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:19 smithi107 ceph-mon[33178]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-06-11T13:26:20.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:26:20.629+0000 7ff4667fc640 1 -- 172.21.15.57:0/2575242374 <== mgr.14150 v1:172.21.15.57:6800/2989665238 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (unknown 0 0 708324239) 0x7ff454000bc0 con 0x7ff45c06ed80 2024-06-11T13:26:20.625 INFO:teuthology.orchestra.run.smithi057.stdout:Created osd(s) 3 on host 'smithi057' 2024-06-11T13:26:20.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:26:20.631+0000 7ff4778a2640 1 -- 172.21.15.57:0/2575242374 >> v1:172.21.15.57:6800/2989665238 conn(0x7ff45c06ed80 legacy=0x7ff45c071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:26:20.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:26:20.631+0000 7ff4778a2640 1 -- 172.21.15.57:0/2575242374 >> v1:172.21.15.57:6789/0 conn(0x7ff47010be50 legacy=0x7ff4701a2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:26:20.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:26:20.631+0000 7ff4778a2640 1 -- 172.21.15.57:0/2575242374 shutdown_connections 2024-06-11T13:26:20.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:26:20.631+0000 7ff4778a2640 1 -- 172.21.15.57:0/2575242374 >> 172.21.15.57:0/2575242374 conn(0x7ff4700fbd10 msgr2=0x7ff4701014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:26:20.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:26:20.631+0000 7ff4778a2640 1 -- 172.21.15.57:0/2575242374 shutdown_connections 2024-06-11T13:26:20.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:26:20.631+0000 7ff4778a2640 1 -- 172.21.15.57:0/2575242374 wait complete. 2024-06-11T13:26:21.200 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:20 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:21.200 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:20 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:21.200 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:20 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:26:21.200 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:20 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:26:21.201 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:20 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:21.201 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:20 smithi057 ceph-mon[40588]: from='osd.3 v1:172.21.15.57:6813/1307945371' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-11T13:26:21.201 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:20 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:26:21.201 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:20 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:21.201 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:20 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:21.201 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:20 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:21.201 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:20 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:21.201 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:20 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:26:21.201 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:20 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:26:21.201 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:20 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:21.202 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:20 smithi057 ceph-mon[29805]: from='osd.3 v1:172.21.15.57:6813/1307945371' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-11T13:26:21.202 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:20 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:26:21.202 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:20 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:21.202 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:20 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:21.227 DEBUG:teuthology.orchestra.run.smithi057:osd.3> sudo journalctl -f -n 0 -u ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@osd.3.service 2024-06-11T13:26:21.229 INFO:tasks.cephadm:Deploying osd.4 on smithi107 with /dev/vg_nvme/lv_4... 2024-06-11T13:26:21.230 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-11T13:26:21.258 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:20 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:21.258 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:20 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:21.258 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:20 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:26:21.258 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:20 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:26:21.258 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:20 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:21.259 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:20 smithi107 ceph-mon[33178]: from='osd.3 v1:172.21.15.57:6813/1307945371' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-11T13:26:21.259 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:20 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:26:21.259 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:20 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:21.259 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:20 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:21.473 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.b/config 2024-06-11T13:26:22.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:21 smithi057 ceph-mon[29805]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-06-11T13:26:22.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:21 smithi057 ceph-mon[29805]: from='osd.3 v1:172.21.15.57:6813/1307945371' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-11T13:26:22.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:21 smithi057 ceph-mon[29805]: osdmap e23: 4 total, 3 up, 4 in 2024-06-11T13:26:22.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:21 smithi057 ceph-mon[29805]: from='osd.3 v1:172.21.15.57:6813/1307945371' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-06-11T13:26:22.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:21 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-11T13:26:22.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:21 smithi057 ceph-mon[40588]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-06-11T13:26:22.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:21 smithi057 ceph-mon[40588]: from='osd.3 v1:172.21.15.57:6813/1307945371' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-11T13:26:22.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:21 smithi057 ceph-mon[40588]: osdmap e23: 4 total, 3 up, 4 in 2024-06-11T13:26:22.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:21 smithi057 ceph-mon[40588]: from='osd.3 v1:172.21.15.57:6813/1307945371' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-06-11T13:26:22.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:21 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-11T13:26:22.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:21 smithi107 ceph-mon[33178]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-06-11T13:26:22.295 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:21 smithi107 ceph-mon[33178]: from='osd.3 v1:172.21.15.57:6813/1307945371' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-11T13:26:22.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:21 smithi107 ceph-mon[33178]: osdmap e23: 4 total, 3 up, 4 in 2024-06-11T13:26:22.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:21 smithi107 ceph-mon[33178]: from='osd.3 v1:172.21.15.57:6813/1307945371' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-06-11T13:26:22.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:21 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-11T13:26:22.902 INFO:journalctl@ceph.osd.3.smithi057.stdout:Jun 11 13:26:22 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-3[61812]: 2024-06-11T13:26:22.792+0000 7f2b7b1e7640 -1 osd.3 0 waiting for initial osdmap 2024-06-11T13:26:22.902 INFO:journalctl@ceph.osd.3.smithi057.stdout:Jun 11 13:26:22 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-3[61812]: 2024-06-11T13:26:22.797+0000 7f2b727f3640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-11T13:26:23.146 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-11T13:26:23.237 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-11T13:26:23.251 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2024-06-11T13:26:23.251 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2024-06-11T13:26:23.251 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0128676 s, 815 MB/s 2024-06-11T13:26:23.251 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2024-06-11T13:26:23.581 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:23 smithi107 ceph-mon[33178]: from='osd.3 v1:172.21.15.57:6813/1307945371' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2024-06-11T13:26:23.581 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:23 smithi107 ceph-mon[33178]: osdmap e24: 4 total, 3 up, 4 in 2024-06-11T13:26:23.581 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:23 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-11T13:26:23.581 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:23 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-11T13:26:23.581 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:23 smithi107 ceph-mon[33178]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-11T13:26:23.581 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:23 smithi107 ceph-mon[33178]: from='osd.3 v1:172.21.15.57:6813/1307945371' entity='osd.3' 2024-06-11T13:26:23.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:23 smithi057 ceph-mon[29805]: from='osd.3 v1:172.21.15.57:6813/1307945371' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2024-06-11T13:26:23.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:23 smithi057 ceph-mon[29805]: osdmap e24: 4 total, 3 up, 4 in 2024-06-11T13:26:23.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:23 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-11T13:26:23.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:23 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-11T13:26:23.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:23 smithi057 ceph-mon[29805]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-11T13:26:23.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:23 smithi057 ceph-mon[29805]: from='osd.3 v1:172.21.15.57:6813/1307945371' entity='osd.3' 2024-06-11T13:26:23.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:23 smithi057 ceph-mon[40588]: from='osd.3 v1:172.21.15.57:6813/1307945371' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2024-06-11T13:26:23.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:23 smithi057 ceph-mon[40588]: osdmap e24: 4 total, 3 up, 4 in 2024-06-11T13:26:23.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:23 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-11T13:26:23.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:23 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-11T13:26:23.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:23 smithi057 ceph-mon[40588]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-06-11T13:26:23.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:23 smithi057 ceph-mon[40588]: from='osd.3 v1:172.21.15.57:6813/1307945371' entity='osd.3' 2024-06-11T13:26:23.858 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi107:vg_nvme/lv_4 2024-06-11T13:26:24.157 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.b/config 2024-06-11T13:26:24.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:24 smithi057 ceph-mon[29805]: purged_snaps scrub starts 2024-06-11T13:26:24.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:24 smithi057 ceph-mon[29805]: purged_snaps scrub ok 2024-06-11T13:26:24.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:24 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-11T13:26:24.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:24 smithi057 ceph-mon[29805]: osd.3 v1:172.21.15.57:6813/1307945371 boot 2024-06-11T13:26:24.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:24 smithi057 ceph-mon[29805]: osdmap e25: 4 total, 4 up, 4 in 2024-06-11T13:26:24.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:24 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-11T13:26:24.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:24 smithi057 ceph-mon[29805]: Detected new or changed devices on smithi057 2024-06-11T13:26:24.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:24 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:24.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:24 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:24.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:24 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:26:24.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:24 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:26:24.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:24 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:26:24.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:24 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:26:24.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:24 smithi057 ceph-mon[29805]: Adjusting osd_memory_target on smithi057 to 4017M 2024-06-11T13:26:24.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:24 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:24.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:24 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:26:24.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:24 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:26:24.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:24 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:24.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:24 smithi057 ceph-mon[40588]: purged_snaps scrub starts 2024-06-11T13:26:24.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:24 smithi057 ceph-mon[40588]: purged_snaps scrub ok 2024-06-11T13:26:24.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:24 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-11T13:26:24.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:24 smithi057 ceph-mon[40588]: osd.3 v1:172.21.15.57:6813/1307945371 boot 2024-06-11T13:26:24.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:24 smithi057 ceph-mon[40588]: osdmap e25: 4 total, 4 up, 4 in 2024-06-11T13:26:24.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:24 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-11T13:26:24.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:24 smithi057 ceph-mon[40588]: Detected new or changed devices on smithi057 2024-06-11T13:26:24.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:24 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:24.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:24 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:24.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:24 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:26:24.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:24 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:26:24.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:24 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:26:24.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:24 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:26:24.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:24 smithi057 ceph-mon[40588]: Adjusting osd_memory_target on smithi057 to 4017M 2024-06-11T13:26:24.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:24 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:24.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:24 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:26:24.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:24 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:26:24.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:24 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:24.545 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:24 smithi107 ceph-mon[33178]: purged_snaps scrub starts 2024-06-11T13:26:24.545 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:24 smithi107 ceph-mon[33178]: purged_snaps scrub ok 2024-06-11T13:26:24.545 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:24 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-11T13:26:24.545 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:24 smithi107 ceph-mon[33178]: osd.3 v1:172.21.15.57:6813/1307945371 boot 2024-06-11T13:26:24.545 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:24 smithi107 ceph-mon[33178]: osdmap e25: 4 total, 4 up, 4 in 2024-06-11T13:26:24.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:24 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-11T13:26:24.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:24 smithi107 ceph-mon[33178]: Detected new or changed devices on smithi057 2024-06-11T13:26:24.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:24 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:24.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:24 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:24.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:24 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:26:24.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:24 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:26:24.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:24 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:26:24.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:24 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:26:24.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:24 smithi107 ceph-mon[33178]: Adjusting osd_memory_target on smithi057 to 4017M 2024-06-11T13:26:24.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:24 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:24.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:24 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:26:24.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:24 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:26:24.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:24 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:25.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:25.509+0000 7f7915916640 1 -- 172.21.15.107:0/3162338733 >> v1:172.21.15.107:6789/0 conn(0x7f7910105e10 legacy=0x7f7910108200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:26:25.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:25.509+0000 7f7915916640 1 -- 172.21.15.107:0/3162338733 shutdown_connections 2024-06-11T13:26:25.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:25.509+0000 7f7915916640 1 -- 172.21.15.107:0/3162338733 >> 172.21.15.107:0/3162338733 conn(0x7f79100fbd10 msgr2=0x7f79100fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:26:25.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:25.509+0000 7f7915916640 1 -- 172.21.15.107:0/3162338733 shutdown_connections 2024-06-11T13:26:25.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:25.509+0000 7f7915916640 1 -- 172.21.15.107:0/3162338733 wait complete. 2024-06-11T13:26:25.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:25.510+0000 7f7915916640 1 Processor -- start 2024-06-11T13:26:25.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:25.510+0000 7f7915916640 1 -- start start 2024-06-11T13:26:25.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:25.510+0000 7f7915916640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f79101974f0 con 0x7f7910105e10 2024-06-11T13:26:25.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:25.510+0000 7f7915916640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f79101a2d40 con 0x7f7910102300 2024-06-11T13:26:25.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:25.510+0000 7f7915916640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f79101a3f20 con 0x7f7910109980 2024-06-11T13:26:25.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:25.510+0000 7f790effd640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f7910102300 0x7f7910196a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.107:35744/0 (socket says 172.21.15.107:35744) 2024-06-11T13:26:25.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:25.510+0000 7f790effd640 1 -- 172.21.15.107:0/3708892188 learned_addr learned my addr 172.21.15.107:0/3708892188 (peer_addr_for_me v1:172.21.15.107:0/0) 2024-06-11T13:26:25.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:25.510+0000 7f7914914640 1 -- 172.21.15.107:0/3708892188 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1171506067 0 0) 0x7f79101a2d40 con 0x7f7910102300 2024-06-11T13:26:25.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:25.510+0000 7f7914914640 1 -- 172.21.15.107:0/3708892188 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f78f8003520 con 0x7f7910102300 2024-06-11T13:26:25.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:25.511+0000 7f7914914640 1 -- 172.21.15.107:0/3708892188 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 27061525 0 0) 0x7f78f8003520 con 0x7f7910102300 2024-06-11T13:26:25.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:25.511+0000 7f7914914640 1 -- 172.21.15.107:0/3708892188 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f79101a2d40 con 0x7f7910102300 2024-06-11T13:26:25.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:25.511+0000 7f7914914640 1 -- 172.21.15.107:0/3708892188 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f7900004380 con 0x7f7910102300 2024-06-11T13:26:25.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:25.511+0000 7f7914914640 1 -- 172.21.15.107:0/3708892188 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 196372108 0 0) 0x7f79101974f0 con 0x7f7910105e10 2024-06-11T13:26:25.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:25.511+0000 7f7914914640 1 -- 172.21.15.107:0/3708892188 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f78f8003520 con 0x7f7910105e10 2024-06-11T13:26:25.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:25.511+0000 7f7914914640 1 -- 172.21.15.107:0/3708892188 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4043229643 0 0) 0x7f79101a3f20 con 0x7f7910109980 2024-06-11T13:26:25.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:25.511+0000 7f7914914640 1 -- 172.21.15.107:0/3708892188 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f79101974f0 con 0x7f7910109980 2024-06-11T13:26:25.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:25.511+0000 7f7914914640 1 -- 172.21.15.107:0/3708892188 <== mon.1 v1:172.21.15.107:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1985937263 0 0) 0x7f79101a2d40 con 0x7f7910102300 2024-06-11T13:26:25.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:25.511+0000 7f7914914640 1 -- 172.21.15.107:0/3708892188 >> v1:172.21.15.57:6790/0 conn(0x7f7910109980 legacy=0x7f79101a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:26:25.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:25.511+0000 7f7914914640 1 -- 172.21.15.107:0/3708892188 >> v1:172.21.15.57:6789/0 conn(0x7f7910105e10 legacy=0x7f791019cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:26:25.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:25.511+0000 7f7914914640 1 -- 172.21.15.107:0/3708892188 --> v1:172.21.15.107:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79101a5100 con 0x7f7910102300 2024-06-11T13:26:25.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:25.511+0000 7f7915916640 1 -- 172.21.15.107:0/3708892188 --> v1:172.21.15.107:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f79101a1d90 con 0x7f7910102300 2024-06-11T13:26:25.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:25.511+0000 7f7915916640 1 -- 172.21.15.107:0/3708892188 --> v1:172.21.15.107:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f79101a2250 con 0x7f7910102300 2024-06-11T13:26:25.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:25.511+0000 7f7914914640 1 -- 172.21.15.107:0/3708892188 <== mon.1 v1:172.21.15.107:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f7900001760 con 0x7f7910102300 2024-06-11T13:26:25.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:25.511+0000 7f7914914640 1 -- 172.21.15.107:0/3708892188 <== mon.1 v1:172.21.15.107:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f7900005fd0 con 0x7f7910102300 2024-06-11T13:26:25.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:25.512+0000 7f7915916640 1 -- 172.21.15.107:0/3708892188 --> v1:172.21.15.107:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78fc005080 con 0x7f7910102300 2024-06-11T13:26:25.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:25.513+0000 7f7914914640 1 -- 172.21.15.107:0/3708892188 <== mon.1 v1:172.21.15.107:6789/0 7 ==== mgrmap(e 15) ==== 92081+0+0 (unknown 2221930365 0 0) 0x7f79000012d0 con 0x7f7910102300 2024-06-11T13:26:25.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:25.513+0000 7f7914914640 1 -- 172.21.15.107:0/3708892188 <== mon.1 v1:172.21.15.107:6789/0 8 ==== osd_map(26..26 src has 1..26) ==== 2803+0+0 (unknown 3095225887 0 0) 0x7f7900088b70 con 0x7f7910102300 2024-06-11T13:26:25.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:25.518+0000 7f7914914640 1 -- 172.21.15.107:0/3708892188 <== mon.1 v1:172.21.15.107:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f7900057050 con 0x7f7910102300 2024-06-11T13:26:25.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:25.702+0000 7f7915916640 1 -- 172.21.15.107:0/3708892188 --> v1:172.21.15.57:6800/2989665238 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f78fc000b90 con 0x7f78f806ee80 2024-06-11T13:26:26.376 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:26 smithi107 ceph-mon[33178]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-11T13:26:26.376 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:26 smithi107 ceph-mon[33178]: osdmap e26: 4 total, 4 up, 4 in 2024-06-11T13:26:26.376 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:26 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-11T13:26:26.376 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:26 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-11T13:26:26.376 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:26 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:26:26.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:26 smithi057 ceph-mon[29805]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-11T13:26:26.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:26 smithi057 ceph-mon[29805]: osdmap e26: 4 total, 4 up, 4 in 2024-06-11T13:26:26.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:26 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-11T13:26:26.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:26 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-11T13:26:26.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:26 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:26:26.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:26 smithi057 ceph-mon[40588]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-11T13:26:26.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:26 smithi057 ceph-mon[40588]: osdmap e26: 4 total, 4 up, 4 in 2024-06-11T13:26:26.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:26 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-11T13:26:26.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:26 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-11T13:26:26.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:26 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:26:27.415 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:27 smithi107 ceph-mon[33178]: from='client.24226 v1:172.21.15.107:0/3708892188' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:26:27.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:27 smithi057 ceph-mon[29805]: from='client.24226 v1:172.21.15.107:0/3708892188' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:26:27.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:27 smithi057 ceph-mon[40588]: from='client.24226 v1:172.21.15.107:0/3708892188' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:26:28.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:28 smithi057 ceph-mon[29805]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-11T13:26:28.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:28 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.107:0/808157929' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "efdccdfc-63d9-42f1-968e-0c411294b675"}]: dispatch 2024-06-11T13:26:28.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:28 smithi057 ceph-mon[29805]: from='client.24232 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "efdccdfc-63d9-42f1-968e-0c411294b675"}]: dispatch 2024-06-11T13:26:28.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:28 smithi057 ceph-mon[29805]: from='client.24232 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "efdccdfc-63d9-42f1-968e-0c411294b675"}]': finished 2024-06-11T13:26:28.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:28 smithi057 ceph-mon[29805]: osdmap e27: 5 total, 4 up, 5 in 2024-06-11T13:26:28.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:28 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-11T13:26:28.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:28 smithi057 ceph-mon[40588]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-11T13:26:28.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:28 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.107:0/808157929' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "efdccdfc-63d9-42f1-968e-0c411294b675"}]: dispatch 2024-06-11T13:26:28.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:28 smithi057 ceph-mon[40588]: from='client.24232 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "efdccdfc-63d9-42f1-968e-0c411294b675"}]: dispatch 2024-06-11T13:26:28.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:28 smithi057 ceph-mon[40588]: from='client.24232 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "efdccdfc-63d9-42f1-968e-0c411294b675"}]': finished 2024-06-11T13:26:28.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:28 smithi057 ceph-mon[40588]: osdmap e27: 5 total, 4 up, 5 in 2024-06-11T13:26:28.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:28 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-11T13:26:28.545 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:28 smithi107 ceph-mon[33178]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-11T13:26:28.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:28 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.107:0/808157929' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "efdccdfc-63d9-42f1-968e-0c411294b675"}]: dispatch 2024-06-11T13:26:28.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:28 smithi107 ceph-mon[33178]: from='client.24232 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "efdccdfc-63d9-42f1-968e-0c411294b675"}]: dispatch 2024-06-11T13:26:28.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:28 smithi107 ceph-mon[33178]: from='client.24232 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "efdccdfc-63d9-42f1-968e-0c411294b675"}]': finished 2024-06-11T13:26:28.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:28 smithi107 ceph-mon[33178]: osdmap e27: 5 total, 4 up, 5 in 2024-06-11T13:26:28.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:28 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-11T13:26:29.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:29 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.107:0/3309423919' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-11T13:26:29.532 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:29 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.107:0/3309423919' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-11T13:26:29.545 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:29 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.107:0/3309423919' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-11T13:26:30.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:30 smithi057 ceph-mon[29805]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-11T13:26:30.532 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:30 smithi057 ceph-mon[40588]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-11T13:26:30.545 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:30 smithi107 ceph-mon[33178]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-11T13:26:31.411 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:31 smithi107 ceph-mon[33178]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-11T13:26:31.532 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:31 smithi057 ceph-mon[40588]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-11T13:26:31.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:31 smithi057 ceph-mon[29805]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-11T13:26:33.782 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:33 smithi057 ceph-mon[40588]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-11T13:26:33.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:33 smithi057 ceph-mon[29805]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-11T13:26:33.795 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:33 smithi107 ceph-mon[33178]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-11T13:26:34.617 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:34 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-11T13:26:34.617 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:34 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:26:34.617 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:34 smithi107 ceph-mon[33178]: Deploying daemon osd.4 on smithi107 2024-06-11T13:26:34.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:34 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-11T13:26:34.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:34 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:26:34.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:34 smithi057 ceph-mon[29805]: Deploying daemon osd.4 on smithi107 2024-06-11T13:26:34.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:34 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-11T13:26:34.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:34 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:26:34.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:34 smithi057 ceph-mon[40588]: Deploying daemon osd.4 on smithi107 2024-06-11T13:26:35.676 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:35 smithi107 ceph-mon[33178]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-11T13:26:35.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:35 smithi057 ceph-mon[29805]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-11T13:26:35.782 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:35 smithi057 ceph-mon[40588]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-11T13:26:37.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:37 smithi057 ceph-mon[29805]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-11T13:26:37.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:37 smithi057 ceph-mon[40588]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-11T13:26:37.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:37 smithi107 ceph-mon[33178]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-11T13:26:39.635 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:39 smithi107 ceph-mon[33178]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-11T13:26:39.635 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:39 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:26:39.635 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:39 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:39.635 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:39 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:40.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:39 smithi057 ceph-mon[29805]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-11T13:26:40.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:39 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:26:40.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:39 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:40.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:39 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:40.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:39 smithi057 ceph-mon[40588]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-11T13:26:40.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:39 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:26:40.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:39 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:40.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:39 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:41.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:41.461+0000 7f7914914640 1 -- 172.21.15.107:0/3708892188 <== mgr.14150 v1:172.21.15.57:6800/2989665238 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (unknown 0 0 634069141) 0x7f78fc000b90 con 0x7f78f806ee80 2024-06-11T13:26:41.457 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 4 on host 'smithi107' 2024-06-11T13:26:41.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:41.464+0000 7f7915916640 1 -- 172.21.15.107:0/3708892188 >> v1:172.21.15.57:6800/2989665238 conn(0x7f78f806ee80 legacy=0x7f78f8071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:26:41.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:41.464+0000 7f7915916640 1 -- 172.21.15.107:0/3708892188 >> v1:172.21.15.107:6789/0 conn(0x7f7910102300 legacy=0x7f7910196a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:26:41.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:41.464+0000 7f7915916640 1 -- 172.21.15.107:0/3708892188 shutdown_connections 2024-06-11T13:26:41.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:41.464+0000 7f7915916640 1 -- 172.21.15.107:0/3708892188 >> 172.21.15.107:0/3708892188 conn(0x7f79100fbd10 msgr2=0x7f79100fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:26:41.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:41.464+0000 7f7915916640 1 -- 172.21.15.107:0/3708892188 shutdown_connections 2024-06-11T13:26:41.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:41.464+0000 7f7915916640 1 -- 172.21.15.107:0/3708892188 wait complete. 2024-06-11T13:26:41.771 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:41 smithi107 ceph-mon[33178]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-11T13:26:41.771 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:41 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:41.771 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:41 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:42.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:41 smithi057 ceph-mon[29805]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-11T13:26:42.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:41 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:42.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:41 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:42.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:41 smithi057 ceph-mon[40588]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-11T13:26:42.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:41 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:42.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:41 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:42.064 DEBUG:teuthology.orchestra.run.smithi107:osd.4> sudo journalctl -f -n 0 -u ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@osd.4.service 2024-06-11T13:26:42.066 INFO:tasks.cephadm:Deploying osd.5 on smithi107 with /dev/vg_nvme/lv_3... 2024-06-11T13:26:42.066 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-11T13:26:42.355 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.b/config 2024-06-11T13:26:42.750 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:42 smithi107 ceph-mon[33178]: from='osd.4 v1:172.21.15.107:6800/2109046293' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-11T13:26:42.750 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:42 smithi107 ceph-mon[33178]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-11T13:26:42.750 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:42 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:42.750 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:42 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:42.934 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:42 smithi057 ceph-mon[40588]: from='osd.4 v1:172.21.15.107:6800/2109046293' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-11T13:26:42.934 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:42 smithi057 ceph-mon[40588]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-11T13:26:42.934 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:42 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:42.934 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:42 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:42.934 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:42 smithi057 ceph-mon[29805]: from='osd.4 v1:172.21.15.107:6800/2109046293' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-11T13:26:42.934 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:42 smithi057 ceph-mon[29805]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-11T13:26:42.935 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:42 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:42.935 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:42 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:43.648 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:43 smithi107 ceph-mon[33178]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-11T13:26:43.649 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:43 smithi107 ceph-mon[33178]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-11T13:26:43.649 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:43 smithi107 ceph-mon[33178]: from='osd.4 v1:172.21.15.107:6800/2109046293' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-11T13:26:43.649 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:43 smithi107 ceph-mon[33178]: osdmap e28: 5 total, 4 up, 5 in 2024-06-11T13:26:43.649 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:43 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-11T13:26:43.649 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:43 smithi107 ceph-mon[33178]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-11T13:26:44.032 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:43 smithi057 ceph-mon[40588]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-11T13:26:44.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:43 smithi057 ceph-mon[40588]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-11T13:26:44.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:43 smithi057 ceph-mon[40588]: from='osd.4 v1:172.21.15.107:6800/2109046293' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-11T13:26:44.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:43 smithi057 ceph-mon[40588]: osdmap e28: 5 total, 4 up, 5 in 2024-06-11T13:26:44.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:43 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-11T13:26:44.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:43 smithi057 ceph-mon[40588]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-11T13:26:44.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:43 smithi057 ceph-mon[29805]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-11T13:26:44.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:43 smithi057 ceph-mon[29805]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-11T13:26:44.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:43 smithi057 ceph-mon[29805]: from='osd.4 v1:172.21.15.107:6800/2109046293' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-11T13:26:44.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:43 smithi057 ceph-mon[29805]: osdmap e28: 5 total, 4 up, 5 in 2024-06-11T13:26:44.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:43 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-11T13:26:44.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:43 smithi057 ceph-mon[29805]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-11T13:26:44.113 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-11T13:26:44.200 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-11T13:26:44.214 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2024-06-11T13:26:44.214 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2024-06-11T13:26:44.214 INFO:teuthology.orchestra.run.smithi107.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0124428 s, 843 MB/s 2024-06-11T13:26:44.214 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2024-06-11T13:26:44.296 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 11 13:26:44 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-4[37950]: 2024-06-11T13:26:44.032+0000 7f300d6eb640 -1 osd.4 0 waiting for initial osdmap 2024-06-11T13:26:44.297 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 11 13:26:44 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-4[37950]: 2024-06-11T13:26:44.036+0000 7f300550b640 -1 osd.4 29 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-11T13:26:44.813 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi107:vg_nvme/lv_3 2024-06-11T13:26:44.836 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:44 smithi107 ceph-mon[33178]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-06-11T13:26:44.836 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:44 smithi107 ceph-mon[33178]: osdmap e29: 5 total, 4 up, 5 in 2024-06-11T13:26:44.836 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:44 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-11T13:26:44.836 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:44 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-11T13:26:44.836 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:44 smithi107 ceph-mon[33178]: from='osd.4 ' entity='osd.4' 2024-06-11T13:26:45.032 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:44 smithi057 ceph-mon[40588]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-06-11T13:26:45.032 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:44 smithi057 ceph-mon[40588]: osdmap e29: 5 total, 4 up, 5 in 2024-06-11T13:26:45.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:44 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-11T13:26:45.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:44 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-11T13:26:45.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:44 smithi057 ceph-mon[40588]: from='osd.4 ' entity='osd.4' 2024-06-11T13:26:45.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:44 smithi057 ceph-mon[29805]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-06-11T13:26:45.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:44 smithi057 ceph-mon[29805]: osdmap e29: 5 total, 4 up, 5 in 2024-06-11T13:26:45.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:44 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-11T13:26:45.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:44 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-11T13:26:45.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:44 smithi057 ceph-mon[29805]: from='osd.4 ' entity='osd.4' 2024-06-11T13:26:45.819 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:45 smithi107 ceph-mon[33178]: purged_snaps scrub starts 2024-06-11T13:26:45.819 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:45 smithi107 ceph-mon[33178]: purged_snaps scrub ok 2024-06-11T13:26:45.819 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:45 smithi107 ceph-mon[33178]: pgmap v108: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-11T13:26:45.820 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:45 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-11T13:26:45.820 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:45 smithi107 ceph-mon[33178]: osd.4 v1:172.21.15.107:6800/2109046293 boot 2024-06-11T13:26:45.820 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:45 smithi107 ceph-mon[33178]: osdmap e30: 5 total, 5 up, 5 in 2024-06-11T13:26:45.820 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:45 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-11T13:26:46.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:45 smithi057 ceph-mon[29805]: purged_snaps scrub starts 2024-06-11T13:26:46.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:45 smithi057 ceph-mon[29805]: purged_snaps scrub ok 2024-06-11T13:26:46.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:45 smithi057 ceph-mon[29805]: pgmap v108: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-11T13:26:46.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:45 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-11T13:26:46.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:45 smithi057 ceph-mon[29805]: osd.4 v1:172.21.15.107:6800/2109046293 boot 2024-06-11T13:26:46.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:45 smithi057 ceph-mon[29805]: osdmap e30: 5 total, 5 up, 5 in 2024-06-11T13:26:46.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:45 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-11T13:26:46.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:45 smithi057 ceph-mon[40588]: purged_snaps scrub starts 2024-06-11T13:26:46.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:45 smithi057 ceph-mon[40588]: purged_snaps scrub ok 2024-06-11T13:26:46.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:45 smithi057 ceph-mon[40588]: pgmap v108: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-06-11T13:26:46.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:45 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-11T13:26:46.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:45 smithi057 ceph-mon[40588]: osd.4 v1:172.21.15.107:6800/2109046293 boot 2024-06-11T13:26:46.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:45 smithi057 ceph-mon[40588]: osdmap e30: 5 total, 5 up, 5 in 2024-06-11T13:26:46.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:45 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-11T13:26:46.084 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.b/config 2024-06-11T13:26:46.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:46 smithi107 ceph-mon[33178]: osdmap e31: 5 total, 5 up, 5 in 2024-06-11T13:26:47.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:46 smithi057 ceph-mon[29805]: osdmap e31: 5 total, 5 up, 5 in 2024-06-11T13:26:47.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:46 smithi057 ceph-mon[40588]: osdmap e31: 5 total, 5 up, 5 in 2024-06-11T13:26:47.947 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:47 smithi107 ceph-mon[33178]: pgmap v111: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-11T13:26:47.947 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:47 smithi107 ceph-mon[33178]: osdmap e32: 5 total, 5 up, 5 in 2024-06-11T13:26:47.947 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:47 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:47.947 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:47 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:47.947 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:47 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:26:47.947 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:47 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:47.947 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:47 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:26:47.947 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:47 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:26:47.947 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:47 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:47.947 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:47 smithi107 ceph-mon[33178]: osdmap e33: 5 total, 5 up, 5 in 2024-06-11T13:26:47.948 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:47 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:26:47.948 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:47 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:26:47.948 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:47 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:26:47.948 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:47 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:48.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:47 smithi057 ceph-mon[29805]: pgmap v111: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-11T13:26:48.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:47 smithi057 ceph-mon[29805]: osdmap e32: 5 total, 5 up, 5 in 2024-06-11T13:26:48.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:47 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:48.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:47 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:48.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:47 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:26:48.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:47 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:48.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:47 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:26:48.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:47 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:26:48.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:47 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:48.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:47 smithi057 ceph-mon[29805]: osdmap e33: 5 total, 5 up, 5 in 2024-06-11T13:26:48.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:47 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:26:48.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:47 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:26:48.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:47 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:26:48.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:47 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:48.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:47 smithi057 ceph-mon[40588]: pgmap v111: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-11T13:26:48.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:47 smithi057 ceph-mon[40588]: osdmap e32: 5 total, 5 up, 5 in 2024-06-11T13:26:48.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:47 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:48.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:47 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:48.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:47 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:26:48.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:47 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:48.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:47 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:26:48.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:47 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:26:48.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:47 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:48.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:47 smithi057 ceph-mon[40588]: osdmap e33: 5 total, 5 up, 5 in 2024-06-11T13:26:48.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:47 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:26:48.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:47 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:26:48.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:47 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:26:48.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:47 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:26:48.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:48.170+0000 7f879c9bc640 1 -- 172.21.15.107:0/141076301 >> v1:172.21.15.107:6789/0 conn(0x7f8798104bd0 legacy=0x7f8798107020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:26:48.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:48.170+0000 7f879c9bc640 1 -- 172.21.15.107:0/141076301 shutdown_connections 2024-06-11T13:26:48.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:48.170+0000 7f879c9bc640 1 -- 172.21.15.107:0/141076301 >> 172.21.15.107:0/141076301 conn(0x7f87980fbd40 msgr2=0x7f87980fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:26:48.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:48.170+0000 7f879c9bc640 1 -- 172.21.15.107:0/141076301 shutdown_connections 2024-06-11T13:26:48.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:48.170+0000 7f879c9bc640 1 -- 172.21.15.107:0/141076301 wait complete. 2024-06-11T13:26:48.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:48.170+0000 7f879c9bc640 1 Processor -- start 2024-06-11T13:26:48.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:48.170+0000 7f879c9bc640 1 -- start start 2024-06-11T13:26:48.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:48.171+0000 7f879c9bc640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f879806ac60 con 0x7f8798108910 2024-06-11T13:26:48.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:48.171+0000 7f879c9bc640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f87981a3120 con 0x7f8798104bd0 2024-06-11T13:26:48.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:48.171+0000 7f879c9bc640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f87981a4300 con 0x7f87981007a0 2024-06-11T13:26:48.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:48.171+0000 7f878ffff640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f8798104bd0 0x7f879819d2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.107:33154/0 (socket says 172.21.15.107:33154) 2024-06-11T13:26:48.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:48.171+0000 7f878ffff640 1 -- 172.21.15.107:0/2395696620 learned_addr learned my addr 172.21.15.107:0/2395696620 (peer_addr_for_me v1:172.21.15.107:0/0) 2024-06-11T13:26:48.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:48.171+0000 7f878f7fe640 1 -- 172.21.15.107:0/2395696620 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 654882122 0 0) 0x7f87981a3120 con 0x7f8798104bd0 2024-06-11T13:26:48.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:48.171+0000 7f878f7fe640 1 -- 172.21.15.107:0/2395696620 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8784003520 con 0x7f8798104bd0 2024-06-11T13:26:48.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:48.171+0000 7f878f7fe640 1 -- 172.21.15.107:0/2395696620 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1778845117 0 0) 0x7f879806ac60 con 0x7f8798108910 2024-06-11T13:26:48.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:48.171+0000 7f878f7fe640 1 -- 172.21.15.107:0/2395696620 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f87981a3120 con 0x7f8798108910 2024-06-11T13:26:48.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:48.171+0000 7f878f7fe640 1 -- 172.21.15.107:0/2395696620 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 875074886 0 0) 0x7f87981a4300 con 0x7f87981007a0 2024-06-11T13:26:48.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:48.171+0000 7f878f7fe640 1 -- 172.21.15.107:0/2395696620 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f879806ac60 con 0x7f87981007a0 2024-06-11T13:26:48.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:48.171+0000 7f878f7fe640 1 -- 172.21.15.107:0/2395696620 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 697728857 0 0) 0x7f8784003520 con 0x7f8798104bd0 2024-06-11T13:26:48.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:48.171+0000 7f878f7fe640 1 -- 172.21.15.107:0/2395696620 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f87981a4300 con 0x7f8798104bd0 2024-06-11T13:26:48.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:48.171+0000 7f878f7fe640 1 -- 172.21.15.107:0/2395696620 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f8780001430 con 0x7f8798104bd0 2024-06-11T13:26:48.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:48.171+0000 7f878f7fe640 1 -- 172.21.15.107:0/2395696620 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3791610536 0 0) 0x7f87981a3120 con 0x7f8798108910 2024-06-11T13:26:48.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:48.171+0000 7f878f7fe640 1 -- 172.21.15.107:0/2395696620 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f8784003520 con 0x7f8798108910 2024-06-11T13:26:48.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:48.171+0000 7f878f7fe640 1 -- 172.21.15.107:0/2395696620 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f8790001b70 con 0x7f8798108910 2024-06-11T13:26:48.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:48.171+0000 7f878f7fe640 1 -- 172.21.15.107:0/2395696620 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2781677359 0 0) 0x7f879806ac60 con 0x7f87981007a0 2024-06-11T13:26:48.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:48.171+0000 7f878f7fe640 1 -- 172.21.15.107:0/2395696620 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f87981a3120 con 0x7f87981007a0 2024-06-11T13:26:48.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:48.171+0000 7f878f7fe640 1 -- 172.21.15.107:0/2395696620 <== mon.1 v1:172.21.15.107:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2395565646 0 0) 0x7f87981a4300 con 0x7f8798104bd0 2024-06-11T13:26:48.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:48.171+0000 7f878f7fe640 1 -- 172.21.15.107:0/2395696620 >> v1:172.21.15.57:6790/0 conn(0x7f87981007a0 legacy=0x7f8798199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:26:48.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:48.171+0000 7f878f7fe640 1 -- 172.21.15.107:0/2395696620 >> v1:172.21.15.57:6789/0 conn(0x7f8798108910 legacy=0x7f87981a09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:26:48.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:48.171+0000 7f878f7fe640 1 -- 172.21.15.107:0/2395696620 --> v1:172.21.15.107:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87981a5540 con 0x7f8798104bd0 2024-06-11T13:26:48.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:48.171+0000 7f879c9bc640 1 -- 172.21.15.107:0/2395696620 --> v1:172.21.15.107:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f87981a2170 con 0x7f8798104bd0 2024-06-11T13:26:48.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:48.172+0000 7f879c9bc640 1 -- 172.21.15.107:0/2395696620 --> v1:172.21.15.107:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f87981a26b0 con 0x7f8798104bd0 2024-06-11T13:26:48.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:48.172+0000 7f878f7fe640 1 -- 172.21.15.107:0/2395696620 <== mon.1 v1:172.21.15.107:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f8780001690 con 0x7f8798104bd0 2024-06-11T13:26:48.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:48.172+0000 7f878f7fe640 1 -- 172.21.15.107:0/2395696620 <== mon.1 v1:172.21.15.107:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f8780004de0 con 0x7f8798104bd0 2024-06-11T13:26:48.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:48.173+0000 7f879c9bc640 1 -- 172.21.15.107:0/2395696620 --> v1:172.21.15.107:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87780050b0 con 0x7f8798104bd0 2024-06-11T13:26:48.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:48.173+0000 7f878f7fe640 1 -- 172.21.15.107:0/2395696620 <== mon.1 v1:172.21.15.107:6789/0 7 ==== mgrmap(e 15) ==== 92081+0+0 (unknown 2221930365 0 0) 0x7f8780003a20 con 0x7f8798104bd0 2024-06-11T13:26:48.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:48.174+0000 7f878f7fe640 1 -- 172.21.15.107:0/2395696620 <== mon.1 v1:172.21.15.107:6789/0 8 ==== osd_map(33..33 src has 1..33) ==== 3205+0+0 (unknown 3418103670 0 0) 0x7f8780088ad0 con 0x7f8798104bd0 2024-06-11T13:26:48.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:48.178+0000 7f878f7fe640 1 -- 172.21.15.107:0/2395696620 <== mon.1 v1:172.21.15.107:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f8780056dc0 con 0x7f8798104bd0 2024-06-11T13:26:48.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:26:48.365+0000 7f879c9bc640 1 -- 172.21.15.107:0/2395696620 --> v1:172.21.15.57:6800/2989665238 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f8778000bc0 con 0x7f878406ede0 2024-06-11T13:26:48.912 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:48 smithi107 ceph-mon[33178]: Detected new or changed devices on smithi107 2024-06-11T13:26:48.913 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:48 smithi107 ceph-mon[33178]: Adjusting osd_memory_target on smithi107 to 17088M 2024-06-11T13:26:48.913 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:48 smithi107 ceph-mon[33178]: osdmap e34: 5 total, 5 up, 5 in 2024-06-11T13:26:48.913 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:48 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-11T13:26:48.913 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:48 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-11T13:26:48.913 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:48 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:26:49.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:48 smithi057 ceph-mon[29805]: Detected new or changed devices on smithi107 2024-06-11T13:26:49.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:48 smithi057 ceph-mon[29805]: Adjusting osd_memory_target on smithi107 to 17088M 2024-06-11T13:26:49.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:48 smithi057 ceph-mon[29805]: osdmap e34: 5 total, 5 up, 5 in 2024-06-11T13:26:49.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:48 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-11T13:26:49.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:48 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-11T13:26:49.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:48 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:26:49.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:48 smithi057 ceph-mon[40588]: Detected new or changed devices on smithi107 2024-06-11T13:26:49.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:48 smithi057 ceph-mon[40588]: Adjusting osd_memory_target on smithi107 to 17088M 2024-06-11T13:26:49.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:48 smithi057 ceph-mon[40588]: osdmap e34: 5 total, 5 up, 5 in 2024-06-11T13:26:49.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:48 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-11T13:26:49.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:48 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-11T13:26:49.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:48 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:26:49.879 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:49 smithi107 ceph-mon[33178]: from='client.24253 v1:172.21.15.107:0/2395696620' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:26:49.879 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:49 smithi107 ceph-mon[33178]: pgmap v115: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-11T13:26:50.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:49 smithi057 ceph-mon[29805]: from='client.24253 v1:172.21.15.107:0/2395696620' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:26:50.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:49 smithi057 ceph-mon[29805]: pgmap v115: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-11T13:26:50.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:49 smithi057 ceph-mon[40588]: from='client.24253 v1:172.21.15.107:0/2395696620' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:26:50.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:49 smithi057 ceph-mon[40588]: pgmap v115: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-11T13:26:52.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:51 smithi057 ceph-mon[29805]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-06-11T13:26:52.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:51 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.107:0/4192691357' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "beac608b-6839-4291-b866-69f354c0fc89"}]: dispatch 2024-06-11T13:26:52.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:51 smithi057 ceph-mon[29805]: from='client.24259 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "beac608b-6839-4291-b866-69f354c0fc89"}]: dispatch 2024-06-11T13:26:52.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:51 smithi057 ceph-mon[29805]: from='client.24259 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "beac608b-6839-4291-b866-69f354c0fc89"}]': finished 2024-06-11T13:26:52.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:51 smithi057 ceph-mon[29805]: osdmap e35: 6 total, 5 up, 6 in 2024-06-11T13:26:52.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:51 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-11T13:26:52.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:51 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.107:0/295267130' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-11T13:26:52.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:51 smithi057 ceph-mon[40588]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-06-11T13:26:52.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:51 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.107:0/4192691357' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "beac608b-6839-4291-b866-69f354c0fc89"}]: dispatch 2024-06-11T13:26:52.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:51 smithi057 ceph-mon[40588]: from='client.24259 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "beac608b-6839-4291-b866-69f354c0fc89"}]: dispatch 2024-06-11T13:26:52.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:51 smithi057 ceph-mon[40588]: from='client.24259 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "beac608b-6839-4291-b866-69f354c0fc89"}]': finished 2024-06-11T13:26:52.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:51 smithi057 ceph-mon[40588]: osdmap e35: 6 total, 5 up, 6 in 2024-06-11T13:26:52.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:51 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-11T13:26:52.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:51 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.107:0/295267130' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-11T13:26:52.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:51 smithi107 ceph-mon[33178]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-06-11T13:26:52.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:51 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.107:0/4192691357' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "beac608b-6839-4291-b866-69f354c0fc89"}]: dispatch 2024-06-11T13:26:52.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:51 smithi107 ceph-mon[33178]: from='client.24259 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "beac608b-6839-4291-b866-69f354c0fc89"}]: dispatch 2024-06-11T13:26:52.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:51 smithi107 ceph-mon[33178]: from='client.24259 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "beac608b-6839-4291-b866-69f354c0fc89"}]': finished 2024-06-11T13:26:52.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:51 smithi107 ceph-mon[33178]: osdmap e35: 6 total, 5 up, 6 in 2024-06-11T13:26:52.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:51 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-11T13:26:52.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:51 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.107:0/295267130' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-11T13:26:53.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:53 smithi107 ceph-mon[33178]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-06-11T13:26:54.032 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:53 smithi057 ceph-mon[40588]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-06-11T13:26:54.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:53 smithi057 ceph-mon[29805]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-06-11T13:26:55.787 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:55 smithi107 ceph-mon[33178]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-06-11T13:26:56.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:55 smithi057 ceph-mon[29805]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-06-11T13:26:56.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:55 smithi057 ceph-mon[40588]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-06-11T13:26:57.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:57 smithi107 ceph-mon[33178]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-06-11T13:26:57.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:57 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-11T13:26:57.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:57 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:26:57.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:57 smithi107 ceph-mon[33178]: Deploying daemon osd.5 on smithi107 2024-06-11T13:26:58.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:57 smithi057 ceph-mon[29805]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-06-11T13:26:58.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:57 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-11T13:26:58.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:57 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:26:58.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:57 smithi057 ceph-mon[29805]: Deploying daemon osd.5 on smithi107 2024-06-11T13:26:58.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:57 smithi057 ceph-mon[40588]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-06-11T13:26:58.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:57 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-11T13:26:58.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:57 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:26:58.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:57 smithi057 ceph-mon[40588]: Deploying daemon osd.5 on smithi107 2024-06-11T13:26:59.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:26:59 smithi107 ceph-mon[33178]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-11T13:27:00.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:26:59 smithi057 ceph-mon[29805]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-11T13:27:00.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:26:59 smithi057 ceph-mon[40588]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-11T13:27:02.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:01 smithi057 ceph-mon[29805]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-11T13:27:02.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:01 smithi057 ceph-mon[40588]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-11T13:27:02.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:01 smithi107 ceph-mon[33178]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-11T13:27:02.961 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:02 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:27:02.961 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:02 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:02.961 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:02 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:03.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:02 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:27:03.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:02 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:03.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:02 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:03.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:02 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:27:03.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:02 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:03.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:02 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:04.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:03 smithi057 ceph-mon[29805]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-11T13:27:04.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:03 smithi057 ceph-mon[40588]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-11T13:27:04.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:03 smithi107 ceph-mon[33178]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-11T13:27:04.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:04.626+0000 7f878f7fe640 1 -- 172.21.15.107:0/2395696620 <== mgr.14150 v1:172.21.15.57:6800/2989665238 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (unknown 0 0 2044644897) 0x7f8778000bc0 con 0x7f878406ede0 2024-06-11T13:27:04.622 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 5 on host 'smithi107' 2024-06-11T13:27:04.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:04.628+0000 7f879c9bc640 1 -- 172.21.15.107:0/2395696620 >> v1:172.21.15.57:6800/2989665238 conn(0x7f878406ede0 legacy=0x7f87840712a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:27:04.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:04.628+0000 7f879c9bc640 1 -- 172.21.15.107:0/2395696620 >> v1:172.21.15.107:6789/0 conn(0x7f8798104bd0 legacy=0x7f879819d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:27:04.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:04.628+0000 7f879c9bc640 1 -- 172.21.15.107:0/2395696620 shutdown_connections 2024-06-11T13:27:04.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:04.628+0000 7f879c9bc640 1 -- 172.21.15.107:0/2395696620 >> 172.21.15.107:0/2395696620 conn(0x7f87980fbd40 msgr2=0x7f8798107020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:27:04.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:04.628+0000 7f879c9bc640 1 -- 172.21.15.107:0/2395696620 shutdown_connections 2024-06-11T13:27:04.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:04.628+0000 7f879c9bc640 1 -- 172.21.15.107:0/2395696620 wait complete. 2024-06-11T13:27:05.590 DEBUG:teuthology.orchestra.run.smithi107:osd.5> sudo journalctl -f -n 0 -u ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@osd.5.service 2024-06-11T13:27:05.633 INFO:tasks.cephadm:Deploying osd.6 on smithi107 with /dev/vg_nvme/lv_2... 2024-06-11T13:27:05.633 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-11T13:27:05.917 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.b/config 2024-06-11T13:27:06.032 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:05 smithi057 ceph-mon[40588]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-11T13:27:06.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:05 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:06.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:05 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:06.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:05 smithi057 ceph-mon[40588]: from='osd.5 v1:172.21.15.107:6804/2372371519' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-11T13:27:06.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:05 smithi057 ceph-mon[40588]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-11T13:27:06.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:05 smithi057 ceph-mon[29805]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-11T13:27:06.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:05 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:06.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:05 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:06.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:05 smithi057 ceph-mon[29805]: from='osd.5 v1:172.21.15.107:6804/2372371519' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-11T13:27:06.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:05 smithi057 ceph-mon[29805]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-11T13:27:06.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:05 smithi107 ceph-mon[33178]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-11T13:27:06.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:05 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:06.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:05 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:06.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:05 smithi107 ceph-mon[33178]: from='osd.5 v1:172.21.15.107:6804/2372371519' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-11T13:27:06.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:05 smithi107 ceph-mon[33178]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-11T13:27:06.985 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:06 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:06.985 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:06 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:06.985 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:06 smithi107 ceph-mon[33178]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-11T13:27:06.985 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:06 smithi107 ceph-mon[33178]: osdmap e36: 6 total, 5 up, 6 in 2024-06-11T13:27:06.986 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:06 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-11T13:27:06.986 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:06 smithi107 ceph-mon[33178]: from='osd.5 v1:172.21.15.107:6804/2372371519' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-11T13:27:06.986 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:06 smithi107 ceph-mon[33178]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-11T13:27:07.032 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:06 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:07.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:06 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:07.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:06 smithi057 ceph-mon[40588]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-11T13:27:07.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:06 smithi057 ceph-mon[40588]: osdmap e36: 6 total, 5 up, 6 in 2024-06-11T13:27:07.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:06 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-11T13:27:07.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:06 smithi057 ceph-mon[40588]: from='osd.5 v1:172.21.15.107:6804/2372371519' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-11T13:27:07.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:06 smithi057 ceph-mon[40588]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-11T13:27:07.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:06 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:07.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:06 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:07.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:06 smithi057 ceph-mon[29805]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-11T13:27:07.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:06 smithi057 ceph-mon[29805]: osdmap e36: 6 total, 5 up, 6 in 2024-06-11T13:27:07.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:06 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-11T13:27:07.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:06 smithi057 ceph-mon[29805]: from='osd.5 v1:172.21.15.107:6804/2372371519' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-11T13:27:07.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:06 smithi057 ceph-mon[29805]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-11T13:27:07.546 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 13:27:07 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-5[43766]: 2024-06-11T13:27:07.345+0000 7f6994150640 -1 osd.5 0 waiting for initial osdmap 2024-06-11T13:27:07.546 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 13:27:07 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-5[43766]: 2024-06-11T13:27:07.348+0000 7f698b75c640 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-11T13:27:07.668 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-11T13:27:07.756 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-11T13:27:07.770 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2024-06-11T13:27:07.770 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2024-06-11T13:27:07.770 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0126431 s, 829 MB/s 2024-06-11T13:27:07.770 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2024-06-11T13:27:08.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:07 smithi057 ceph-mon[29805]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-11T13:27:08.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:07 smithi057 ceph-mon[29805]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-06-11T13:27:08.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:07 smithi057 ceph-mon[29805]: osdmap e37: 6 total, 5 up, 6 in 2024-06-11T13:27:08.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:07 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-11T13:27:08.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:07 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-11T13:27:08.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:07 smithi057 ceph-mon[29805]: osdmap e38: 6 total, 5 up, 6 in 2024-06-11T13:27:08.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:07 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-11T13:27:08.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:07 smithi057 ceph-mon[29805]: from='osd.5 ' entity='osd.5' 2024-06-11T13:27:08.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:07 smithi057 ceph-mon[40588]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-11T13:27:08.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:07 smithi057 ceph-mon[40588]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-06-11T13:27:08.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:07 smithi057 ceph-mon[40588]: osdmap e37: 6 total, 5 up, 6 in 2024-06-11T13:27:08.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:07 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-11T13:27:08.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:07 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-11T13:27:08.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:07 smithi057 ceph-mon[40588]: osdmap e38: 6 total, 5 up, 6 in 2024-06-11T13:27:08.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:07 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-11T13:27:08.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:07 smithi057 ceph-mon[40588]: from='osd.5 ' entity='osd.5' 2024-06-11T13:27:08.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:07 smithi107 ceph-mon[33178]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-11T13:27:08.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:07 smithi107 ceph-mon[33178]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-06-11T13:27:08.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:07 smithi107 ceph-mon[33178]: osdmap e37: 6 total, 5 up, 6 in 2024-06-11T13:27:08.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:07 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-11T13:27:08.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:07 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-11T13:27:08.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:07 smithi107 ceph-mon[33178]: osdmap e38: 6 total, 5 up, 6 in 2024-06-11T13:27:08.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:07 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-11T13:27:08.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:07 smithi107 ceph-mon[33178]: from='osd.5 ' entity='osd.5' 2024-06-11T13:27:09.026 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi107:vg_nvme/lv_2 2024-06-11T13:27:09.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:08 smithi057 ceph-mon[40588]: purged_snaps scrub starts 2024-06-11T13:27:09.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:08 smithi057 ceph-mon[40588]: purged_snaps scrub ok 2024-06-11T13:27:09.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:08 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-11T13:27:09.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:08 smithi057 ceph-mon[40588]: osd.5 v1:172.21.15.107:6804/2372371519 boot 2024-06-11T13:27:09.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:08 smithi057 ceph-mon[40588]: osdmap e39: 6 total, 6 up, 6 in 2024-06-11T13:27:09.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:08 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-11T13:27:09.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:08 smithi057 ceph-mon[29805]: purged_snaps scrub starts 2024-06-11T13:27:09.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:08 smithi057 ceph-mon[29805]: purged_snaps scrub ok 2024-06-11T13:27:09.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:08 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-11T13:27:09.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:08 smithi057 ceph-mon[29805]: osd.5 v1:172.21.15.107:6804/2372371519 boot 2024-06-11T13:27:09.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:08 smithi057 ceph-mon[29805]: osdmap e39: 6 total, 6 up, 6 in 2024-06-11T13:27:09.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:08 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-11T13:27:09.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:08 smithi107 ceph-mon[33178]: purged_snaps scrub starts 2024-06-11T13:27:09.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:08 smithi107 ceph-mon[33178]: purged_snaps scrub ok 2024-06-11T13:27:09.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:08 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-11T13:27:09.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:08 smithi107 ceph-mon[33178]: osd.5 v1:172.21.15.107:6804/2372371519 boot 2024-06-11T13:27:09.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:08 smithi107 ceph-mon[33178]: osdmap e39: 6 total, 6 up, 6 in 2024-06-11T13:27:09.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:08 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-11T13:27:09.720 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.b/config 2024-06-11T13:27:10.109 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:09 smithi107 ceph-mon[33178]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-11T13:27:10.109 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:09 smithi107 ceph-mon[33178]: osdmap e40: 6 total, 6 up, 6 in 2024-06-11T13:27:10.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:09 smithi057 ceph-mon[29805]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-11T13:27:10.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:09 smithi057 ceph-mon[29805]: osdmap e40: 6 total, 6 up, 6 in 2024-06-11T13:27:10.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:09 smithi057 ceph-mon[40588]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-06-11T13:27:10.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:09 smithi057 ceph-mon[40588]: osdmap e40: 6 total, 6 up, 6 in 2024-06-11T13:27:11.627 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:11 smithi107 ceph-mon[33178]: osdmap e41: 6 total, 6 up, 6 in 2024-06-11T13:27:11.627 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:11 smithi107 ceph-mon[33178]: pgmap v133: 1 pgs: 1 remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-11T13:27:11.627 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:11 smithi107 ceph-mon[33178]: Detected new or changed devices on smithi107 2024-06-11T13:27:11.627 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:11 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:11.627 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:11 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:11.627 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:11 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:27:11.627 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:11 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:27:11.627 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:11 smithi107 ceph-mon[33178]: Adjusting osd_memory_target on smithi107 to 8544M 2024-06-11T13:27:11.627 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:11 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:11.628 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:11 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:27:11.628 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:11 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:27:11.628 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:11 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:11.628 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:11 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:27:11.628 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:11 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:27:11.628 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:11 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:27:11.628 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:11 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:11.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:11 smithi057 ceph-mon[40588]: osdmap e41: 6 total, 6 up, 6 in 2024-06-11T13:27:11.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:11 smithi057 ceph-mon[40588]: pgmap v133: 1 pgs: 1 remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-11T13:27:11.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:11 smithi057 ceph-mon[40588]: Detected new or changed devices on smithi107 2024-06-11T13:27:11.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:11 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:11.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:11 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:11.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:11 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:27:11.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:11 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:27:11.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:11 smithi057 ceph-mon[40588]: Adjusting osd_memory_target on smithi107 to 8544M 2024-06-11T13:27:11.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:11 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:11.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:11 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:27:11.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:11 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:27:11.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:11 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:11.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:11 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:27:11.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:11 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:27:11.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:11 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:27:11.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:11 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:11.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:11 smithi057 ceph-mon[29805]: osdmap e41: 6 total, 6 up, 6 in 2024-06-11T13:27:11.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:11 smithi057 ceph-mon[29805]: pgmap v133: 1 pgs: 1 remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-11T13:27:11.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:11 smithi057 ceph-mon[29805]: Detected new or changed devices on smithi107 2024-06-11T13:27:11.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:11 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:11.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:11 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:11.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:11 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:27:11.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:11 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:27:11.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:11 smithi057 ceph-mon[29805]: Adjusting osd_memory_target on smithi107 to 8544M 2024-06-11T13:27:11.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:11 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:11.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:11 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:27:11.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:11 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:27:11.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:11 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:11.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:11 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:27:11.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:11 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:27:11.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:11 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:27:11.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:11 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:11.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:11.837+0000 7f7b0e20f640 1 -- 172.21.15.107:0/319248320 >> v1:172.21.15.107:6789/0 conn(0x7f7b08101880 legacy=0x7f7b08101ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:27:11.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:11.837+0000 7f7b0e20f640 1 -- 172.21.15.107:0/319248320 shutdown_connections 2024-06-11T13:27:11.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:11.837+0000 7f7b0e20f640 1 -- 172.21.15.107:0/319248320 >> 172.21.15.107:0/319248320 conn(0x7f7b080fce20 msgr2=0x7f7b080ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:27:11.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:11.838+0000 7f7b0e20f640 1 -- 172.21.15.107:0/319248320 shutdown_connections 2024-06-11T13:27:11.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:11.838+0000 7f7b0e20f640 1 -- 172.21.15.107:0/319248320 wait complete. 2024-06-11T13:27:11.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:11.838+0000 7f7b0e20f640 1 Processor -- start 2024-06-11T13:27:11.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:11.838+0000 7f7b0e20f640 1 -- start start 2024-06-11T13:27:11.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:11.838+0000 7f7b0e20f640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f7b08197910 con 0x7f7b08101880 2024-06-11T13:27:11.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:11.838+0000 7f7b0e20f640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f7b081a3160 con 0x7f7b08105cb0 2024-06-11T13:27:11.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:11.838+0000 7f7b0e20f640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f7b081a4340 con 0x7f7b081099f0 2024-06-11T13:27:11.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:11.839+0000 7f7b06ffd640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f7b08105cb0 0x7f7b0819d300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.107:51294/0 (socket says 172.21.15.107:51294) 2024-06-11T13:27:11.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:11.839+0000 7f7b06ffd640 1 -- 172.21.15.107:0/4114141578 learned_addr learned my addr 172.21.15.107:0/4114141578 (peer_addr_for_me v1:172.21.15.107:0/0) 2024-06-11T13:27:11.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:11.839+0000 7f7b04ff9640 1 -- 172.21.15.107:0/4114141578 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4269992506 0 0) 0x7f7b081a3160 con 0x7f7b08105cb0 2024-06-11T13:27:11.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:11.839+0000 7f7b04ff9640 1 -- 172.21.15.107:0/4114141578 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7af0003520 con 0x7f7b08105cb0 2024-06-11T13:27:11.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:11.839+0000 7f7b04ff9640 1 -- 172.21.15.107:0/4114141578 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2017445208 0 0) 0x7f7b08197910 con 0x7f7b08101880 2024-06-11T13:27:11.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:11.839+0000 7f7b04ff9640 1 -- 172.21.15.107:0/4114141578 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b081a3160 con 0x7f7b08101880 2024-06-11T13:27:11.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:11.839+0000 7f7b04ff9640 1 -- 172.21.15.107:0/4114141578 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1269770708 0 0) 0x7f7b081a4340 con 0x7f7b081099f0 2024-06-11T13:27:11.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:11.839+0000 7f7b04ff9640 1 -- 172.21.15.107:0/4114141578 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b08197910 con 0x7f7b081099f0 2024-06-11T13:27:11.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:11.839+0000 7f7b04ff9640 1 -- 172.21.15.107:0/4114141578 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 215574544 0 0) 0x7f7af0003520 con 0x7f7b08105cb0 2024-06-11T13:27:11.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:11.839+0000 7f7b04ff9640 1 -- 172.21.15.107:0/4114141578 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7b081a4340 con 0x7f7b08105cb0 2024-06-11T13:27:11.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:11.839+0000 7f7b04ff9640 1 -- 172.21.15.107:0/4114141578 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f7afc0018b0 con 0x7f7b08105cb0 2024-06-11T13:27:11.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:11.839+0000 7f7b04ff9640 1 -- 172.21.15.107:0/4114141578 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3629716519 0 0) 0x7f7b081a3160 con 0x7f7b08101880 2024-06-11T13:27:11.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:11.839+0000 7f7b04ff9640 1 -- 172.21.15.107:0/4114141578 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7af0003520 con 0x7f7b08101880 2024-06-11T13:27:11.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:11.839+0000 7f7b04ff9640 1 -- 172.21.15.107:0/4114141578 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f7af8001430 con 0x7f7b08101880 2024-06-11T13:27:11.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:11.839+0000 7f7b04ff9640 1 -- 172.21.15.107:0/4114141578 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3950903413 0 0) 0x7f7b08197910 con 0x7f7b081099f0 2024-06-11T13:27:11.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:11.839+0000 7f7b04ff9640 1 -- 172.21.15.107:0/4114141578 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7b081a3160 con 0x7f7b081099f0 2024-06-11T13:27:11.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:11.839+0000 7f7b04ff9640 1 -- 172.21.15.107:0/4114141578 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f7b00001af0 con 0x7f7b081099f0 2024-06-11T13:27:11.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:11.839+0000 7f7b04ff9640 1 -- 172.21.15.107:0/4114141578 <== mon.1 v1:172.21.15.107:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1170760558 0 0) 0x7f7b081a4340 con 0x7f7b08105cb0 2024-06-11T13:27:11.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:11.839+0000 7f7b04ff9640 1 -- 172.21.15.107:0/4114141578 >> v1:172.21.15.57:6790/0 conn(0x7f7b081099f0 legacy=0x7f7b081a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:27:11.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:11.839+0000 7f7b04ff9640 1 -- 172.21.15.107:0/4114141578 >> v1:172.21.15.57:6789/0 conn(0x7f7b08101880 legacy=0x7f7b08196e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:27:11.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:11.839+0000 7f7b04ff9640 1 -- 172.21.15.107:0/4114141578 --> v1:172.21.15.107:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b081a5520 con 0x7f7b08105cb0 2024-06-11T13:27:11.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:11.839+0000 7f7b0e20f640 1 -- 172.21.15.107:0/4114141578 --> v1:172.21.15.107:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f7b081a21b0 con 0x7f7b08105cb0 2024-06-11T13:27:11.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:11.839+0000 7f7b0e20f640 1 -- 172.21.15.107:0/4114141578 --> v1:172.21.15.107:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f7b081a26f0 con 0x7f7b08105cb0 2024-06-11T13:27:11.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:11.839+0000 7f7b04ff9640 1 -- 172.21.15.107:0/4114141578 <== mon.1 v1:172.21.15.107:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f7afc001c80 con 0x7f7b08105cb0 2024-06-11T13:27:11.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:11.839+0000 7f7b04ff9640 1 -- 172.21.15.107:0/4114141578 <== mon.1 v1:172.21.15.107:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f7afc005c20 con 0x7f7b08105cb0 2024-06-11T13:27:11.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:11.840+0000 7f7b0e20f640 1 -- 172.21.15.107:0/4114141578 --> v1:172.21.15.107:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7af40050b0 con 0x7f7b08105cb0 2024-06-11T13:27:11.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:11.841+0000 7f7b04ff9640 1 -- 172.21.15.107:0/4114141578 <== mon.1 v1:172.21.15.107:6789/0 7 ==== mgrmap(e 15) ==== 92081+0+0 (unknown 2221930365 0 0) 0x7f7afc003fe0 con 0x7f7b08105cb0 2024-06-11T13:27:11.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:11.841+0000 7f7b04ff9640 1 -- 172.21.15.107:0/4114141578 <== mon.1 v1:172.21.15.107:6789/0 8 ==== osd_map(42..42 src has 1..42) ==== 3497+0+0 (unknown 3520340471 0 0) 0x7f7afc0888d0 con 0x7f7b08105cb0 2024-06-11T13:27:11.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:11.845+0000 7f7b04ff9640 1 -- 172.21.15.107:0/4114141578 <== mon.1 v1:172.21.15.107:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f7afc056aa0 con 0x7f7b08105cb0 2024-06-11T13:27:12.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:12.028+0000 7f7b0e20f640 1 -- 172.21.15.107:0/4114141578 --> v1:172.21.15.57:6800/2989665238 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f7af4000bc0 con 0x7f7af006ee10 2024-06-11T13:27:12.686 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:12 smithi057 ceph-mon[40588]: osdmap e42: 6 total, 6 up, 6 in 2024-06-11T13:27:12.686 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:12 smithi057 ceph-mon[40588]: from='client.24280 v1:172.21.15.107:0/4114141578' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:27:12.686 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:12 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-11T13:27:12.686 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:12 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-11T13:27:12.687 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:12 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:27:12.687 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:12 smithi057 ceph-mon[29805]: osdmap e42: 6 total, 6 up, 6 in 2024-06-11T13:27:12.687 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:12 smithi057 ceph-mon[29805]: from='client.24280 v1:172.21.15.107:0/4114141578' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:27:12.687 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:12 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-11T13:27:12.687 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:12 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-11T13:27:12.687 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:12 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:27:12.688 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:12 smithi107 ceph-mon[33178]: osdmap e42: 6 total, 6 up, 6 in 2024-06-11T13:27:12.688 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:12 smithi107 ceph-mon[33178]: from='client.24280 v1:172.21.15.107:0/4114141578' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:27:12.688 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:12 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-11T13:27:12.688 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:12 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-11T13:27:12.688 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:12 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:27:13.641 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:13 smithi107 ceph-mon[33178]: osdmap e43: 6 total, 6 up, 6 in 2024-06-11T13:27:13.642 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:13 smithi107 ceph-mon[33178]: pgmap v136: 1 pgs: 1 remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-11T13:27:13.698 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:13 smithi057 ceph-mon[29805]: osdmap e43: 6 total, 6 up, 6 in 2024-06-11T13:27:13.698 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:13 smithi057 ceph-mon[29805]: pgmap v136: 1 pgs: 1 remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-11T13:27:13.698 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:13 smithi057 ceph-mon[40588]: osdmap e43: 6 total, 6 up, 6 in 2024-06-11T13:27:13.698 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:13 smithi057 ceph-mon[40588]: pgmap v136: 1 pgs: 1 remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-11T13:27:14.782 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:14 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.107:0/3680011950' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4f8d2d5-dabb-42ad-baea-cd5de8269914"}]: dispatch 2024-06-11T13:27:14.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:14 smithi057 ceph-mon[40588]: from='client.24286 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4f8d2d5-dabb-42ad-baea-cd5de8269914"}]: dispatch 2024-06-11T13:27:14.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:14 smithi057 ceph-mon[40588]: from='client.24286 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d4f8d2d5-dabb-42ad-baea-cd5de8269914"}]': finished 2024-06-11T13:27:14.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:14 smithi057 ceph-mon[40588]: osdmap e44: 7 total, 6 up, 7 in 2024-06-11T13:27:14.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:14 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-11T13:27:14.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:14 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.107:0/3680011950' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4f8d2d5-dabb-42ad-baea-cd5de8269914"}]: dispatch 2024-06-11T13:27:14.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:14 smithi057 ceph-mon[29805]: from='client.24286 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4f8d2d5-dabb-42ad-baea-cd5de8269914"}]: dispatch 2024-06-11T13:27:14.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:14 smithi057 ceph-mon[29805]: from='client.24286 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d4f8d2d5-dabb-42ad-baea-cd5de8269914"}]': finished 2024-06-11T13:27:14.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:14 smithi057 ceph-mon[29805]: osdmap e44: 7 total, 6 up, 7 in 2024-06-11T13:27:14.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:14 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-11T13:27:14.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:14 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.107:0/3680011950' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4f8d2d5-dabb-42ad-baea-cd5de8269914"}]: dispatch 2024-06-11T13:27:14.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:14 smithi107 ceph-mon[33178]: from='client.24286 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4f8d2d5-dabb-42ad-baea-cd5de8269914"}]: dispatch 2024-06-11T13:27:14.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:14 smithi107 ceph-mon[33178]: from='client.24286 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d4f8d2d5-dabb-42ad-baea-cd5de8269914"}]': finished 2024-06-11T13:27:14.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:14 smithi107 ceph-mon[33178]: osdmap e44: 7 total, 6 up, 7 in 2024-06-11T13:27:14.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:14 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-11T13:27:15.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:15 smithi107 ceph-mon[33178]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-06-11T13:27:15.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:15 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.107:0/1733939361' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-11T13:27:16.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:15 smithi057 ceph-mon[29805]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-06-11T13:27:16.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:15 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.107:0/1733939361' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-11T13:27:16.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:15 smithi057 ceph-mon[40588]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-06-11T13:27:16.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:15 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.107:0/1733939361' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-11T13:27:17.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:17 smithi107 ceph-mon[33178]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-11T13:27:18.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:17 smithi057 ceph-mon[29805]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-11T13:27:18.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:17 smithi057 ceph-mon[40588]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-11T13:27:19.776 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:19 smithi107 ceph-mon[33178]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-06-11T13:27:20.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:19 smithi057 ceph-mon[29805]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-06-11T13:27:20.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:19 smithi057 ceph-mon[40588]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-06-11T13:27:21.927 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:21 smithi107 ceph-mon[33178]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-06-11T13:27:21.927 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:21 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-11T13:27:21.928 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:21 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:27:21.928 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:21 smithi107 ceph-mon[33178]: Deploying daemon osd.6 on smithi107 2024-06-11T13:27:22.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:21 smithi057 ceph-mon[29805]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-06-11T13:27:22.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:21 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-11T13:27:22.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:21 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:27:22.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:21 smithi057 ceph-mon[29805]: Deploying daemon osd.6 on smithi107 2024-06-11T13:27:22.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:21 smithi057 ceph-mon[40588]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-06-11T13:27:22.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:21 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-11T13:27:22.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:21 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:27:22.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:21 smithi057 ceph-mon[40588]: Deploying daemon osd.6 on smithi107 2024-06-11T13:27:23.706 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:23 smithi107 ceph-mon[33178]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-11T13:27:24.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:23 smithi057 ceph-mon[29805]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-11T13:27:24.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:23 smithi057 ceph-mon[40588]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-11T13:27:26.016 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:25 smithi107 ceph-mon[33178]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-11T13:27:26.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:25 smithi057 ceph-mon[29805]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-11T13:27:26.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:25 smithi057 ceph-mon[40588]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-11T13:27:26.939 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:26 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:27:26.939 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:26 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:26.939 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:26 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:27.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:26 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:27:27.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:26 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:27.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:26 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:27.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:26 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:27:27.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:26 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:27.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:26 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:27.919 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:27 smithi107 ceph-mon[33178]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-11T13:27:28.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:27 smithi057 ceph-mon[29805]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-11T13:27:28.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:27 smithi057 ceph-mon[40588]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-11T13:27:29.225 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 6 on host 'smithi107' 2024-06-11T13:27:29.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:29.228+0000 7f7b04ff9640 1 -- 172.21.15.107:0/4114141578 <== mgr.14150 v1:172.21.15.57:6800/2989665238 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (unknown 0 0 2648767997) 0x7f7af4000bc0 con 0x7f7af006ee10 2024-06-11T13:27:29.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:29.231+0000 7f7b0e20f640 1 -- 172.21.15.107:0/4114141578 >> v1:172.21.15.57:6800/2989665238 conn(0x7f7af006ee10 legacy=0x7f7af00712d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:27:29.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:29.231+0000 7f7b0e20f640 1 -- 172.21.15.107:0/4114141578 >> v1:172.21.15.107:6789/0 conn(0x7f7b08105cb0 legacy=0x7f7b0819d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:27:29.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:29.231+0000 7f7b0e20f640 1 -- 172.21.15.107:0/4114141578 shutdown_connections 2024-06-11T13:27:29.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:29.231+0000 7f7b0e20f640 1 -- 172.21.15.107:0/4114141578 >> 172.21.15.107:0/4114141578 conn(0x7f7b080fce20 msgr2=0x7f7b08108100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:27:29.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:29.231+0000 7f7b0e20f640 1 -- 172.21.15.107:0/4114141578 shutdown_connections 2024-06-11T13:27:29.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:29.231+0000 7f7b0e20f640 1 -- 172.21.15.107:0/4114141578 wait complete. 2024-06-11T13:27:29.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:29 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:29.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:29 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:29.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:29 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:27:29.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:29 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:27:29.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:29 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:29.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:29 smithi057 ceph-mon[29805]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-11T13:27:29.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:29 smithi057 ceph-mon[29805]: from='osd.6 v1:172.21.15.107:6808/1305047914' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-11T13:27:29.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:29 smithi057 ceph-mon[29805]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-11T13:27:29.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:29 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:27:29.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:29 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:29.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:29 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:29.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:29 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:29.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:29 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:29.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:29 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:27:29.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:29 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:27:29.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:29 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:29.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:29 smithi057 ceph-mon[40588]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-11T13:27:29.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:29 smithi057 ceph-mon[40588]: from='osd.6 v1:172.21.15.107:6808/1305047914' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-11T13:27:29.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:29 smithi057 ceph-mon[40588]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-11T13:27:29.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:29 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:27:29.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:29 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:29.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:29 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:29.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:29 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:29.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:29 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:29.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:29 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:27:29.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:29 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:27:29.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:29 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:29.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:29 smithi107 ceph-mon[33178]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-11T13:27:29.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:29 smithi107 ceph-mon[33178]: from='osd.6 v1:172.21.15.107:6808/1305047914' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-11T13:27:29.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:29 smithi107 ceph-mon[33178]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-11T13:27:29.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:29 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:27:29.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:29 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:29.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:29 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:29.849 DEBUG:teuthology.orchestra.run.smithi107:osd.6> sudo journalctl -f -n 0 -u ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@osd.6.service 2024-06-11T13:27:29.851 INFO:tasks.cephadm:Deploying osd.7 on smithi107 with /dev/vg_nvme/lv_1... 2024-06-11T13:27:29.852 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-11T13:27:30.626 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.b/config 2024-06-11T13:27:30.748 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:30 smithi107 ceph-mon[33178]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-11T13:27:30.748 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:30 smithi107 ceph-mon[33178]: from='osd.6 v1:172.21.15.107:6808/1305047914' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-11T13:27:30.748 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:30 smithi107 ceph-mon[33178]: osdmap e45: 7 total, 6 up, 7 in 2024-06-11T13:27:30.748 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:30 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-11T13:27:30.748 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:30 smithi107 ceph-mon[33178]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-11T13:27:30.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:30 smithi057 ceph-mon[29805]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-11T13:27:30.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:30 smithi057 ceph-mon[29805]: from='osd.6 v1:172.21.15.107:6808/1305047914' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-11T13:27:30.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:30 smithi057 ceph-mon[29805]: osdmap e45: 7 total, 6 up, 7 in 2024-06-11T13:27:30.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:30 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-11T13:27:30.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:30 smithi057 ceph-mon[29805]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-11T13:27:30.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:30 smithi057 ceph-mon[40588]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-11T13:27:30.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:30 smithi057 ceph-mon[40588]: from='osd.6 v1:172.21.15.107:6808/1305047914' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-11T13:27:30.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:30 smithi057 ceph-mon[40588]: osdmap e45: 7 total, 6 up, 7 in 2024-06-11T13:27:30.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:30 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-11T13:27:30.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:30 smithi057 ceph-mon[40588]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-11T13:27:31.031 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 13:27:30 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T13:27:30.851+0000 7f9db1e39640 -1 osd.6 0 waiting for initial osdmap 2024-06-11T13:27:31.031 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 13:27:30 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T13:27:30.858+0000 7f9da9c59640 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-11T13:27:31.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:31 smithi107 ceph-mon[33178]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-06-11T13:27:31.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:31 smithi107 ceph-mon[33178]: osdmap e46: 7 total, 6 up, 7 in 2024-06-11T13:27:31.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:31 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-11T13:27:31.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:31 smithi107 ceph-mon[33178]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-11T13:27:31.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:31 smithi107 ceph-mon[33178]: from='osd.6 ' entity='osd.6' 2024-06-11T13:27:31.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:31 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-11T13:27:31.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:31 smithi057 ceph-mon[29805]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-06-11T13:27:31.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:31 smithi057 ceph-mon[29805]: osdmap e46: 7 total, 6 up, 7 in 2024-06-11T13:27:31.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:31 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-11T13:27:31.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:31 smithi057 ceph-mon[29805]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-11T13:27:31.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:31 smithi057 ceph-mon[29805]: from='osd.6 ' entity='osd.6' 2024-06-11T13:27:31.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:31 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-11T13:27:31.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:31 smithi057 ceph-mon[40588]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-06-11T13:27:31.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:31 smithi057 ceph-mon[40588]: osdmap e46: 7 total, 6 up, 7 in 2024-06-11T13:27:31.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:31 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-11T13:27:31.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:31 smithi057 ceph-mon[40588]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-06-11T13:27:31.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:31 smithi057 ceph-mon[40588]: from='osd.6 ' entity='osd.6' 2024-06-11T13:27:31.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:31 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-11T13:27:32.735 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:32 smithi107 ceph-mon[33178]: purged_snaps scrub starts 2024-06-11T13:27:32.736 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:32 smithi107 ceph-mon[33178]: purged_snaps scrub ok 2024-06-11T13:27:32.736 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:32 smithi107 ceph-mon[33178]: osd.6 v1:172.21.15.107:6808/1305047914 boot 2024-06-11T13:27:32.736 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:32 smithi107 ceph-mon[33178]: osdmap e47: 7 total, 7 up, 7 in 2024-06-11T13:27:32.736 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:32 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-11T13:27:32.736 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:32 smithi107 ceph-mon[33178]: osdmap e48: 7 total, 7 up, 7 in 2024-06-11T13:27:32.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:32 smithi057 ceph-mon[29805]: purged_snaps scrub starts 2024-06-11T13:27:32.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:32 smithi057 ceph-mon[29805]: purged_snaps scrub ok 2024-06-11T13:27:32.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:32 smithi057 ceph-mon[29805]: osd.6 v1:172.21.15.107:6808/1305047914 boot 2024-06-11T13:27:32.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:32 smithi057 ceph-mon[29805]: osdmap e47: 7 total, 7 up, 7 in 2024-06-11T13:27:32.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:32 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-11T13:27:32.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:32 smithi057 ceph-mon[29805]: osdmap e48: 7 total, 7 up, 7 in 2024-06-11T13:27:32.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:32 smithi057 ceph-mon[40588]: purged_snaps scrub starts 2024-06-11T13:27:32.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:32 smithi057 ceph-mon[40588]: purged_snaps scrub ok 2024-06-11T13:27:32.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:32 smithi057 ceph-mon[40588]: osd.6 v1:172.21.15.107:6808/1305047914 boot 2024-06-11T13:27:32.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:32 smithi057 ceph-mon[40588]: osdmap e47: 7 total, 7 up, 7 in 2024-06-11T13:27:32.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:32 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-11T13:27:32.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:32 smithi057 ceph-mon[40588]: osdmap e48: 7 total, 7 up, 7 in 2024-06-11T13:27:33.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:33 smithi057 ceph-mon[29805]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-06-11T13:27:33.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:33 smithi057 ceph-mon[29805]: osdmap e49: 7 total, 7 up, 7 in 2024-06-11T13:27:33.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:33 smithi057 ceph-mon[40588]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-06-11T13:27:33.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:33 smithi057 ceph-mon[40588]: osdmap e49: 7 total, 7 up, 7 in 2024-06-11T13:27:33.795 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:33 smithi107 ceph-mon[33178]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-06-11T13:27:33.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:33 smithi107 ceph-mon[33178]: osdmap e49: 7 total, 7 up, 7 in 2024-06-11T13:27:33.875 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-11T13:27:33.958 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-11T13:27:33.971 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2024-06-11T13:27:33.971 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2024-06-11T13:27:33.971 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113604 s, 923 MB/s 2024-06-11T13:27:33.972 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2024-06-11T13:27:34.491 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:34 smithi057 ceph-mon[29805]: Detected new or changed devices on smithi107 2024-06-11T13:27:34.491 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:34 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:34.492 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:34 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:34.492 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:34 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:27:34.492 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:34 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:27:34.492 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:34 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:27:34.492 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:34 smithi057 ceph-mon[29805]: Adjusting osd_memory_target on smithi107 to 5696M 2024-06-11T13:27:34.492 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:34 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:34.492 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:34 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:27:34.492 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:34 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:27:34.492 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:34 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:34.492 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:34 smithi057 ceph-mon[29805]: osdmap e50: 7 total, 7 up, 7 in 2024-06-11T13:27:34.492 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:34 smithi057 ceph-mon[40588]: Detected new or changed devices on smithi107 2024-06-11T13:27:34.492 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:34 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:34.493 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:34 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:34.493 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:34 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:27:34.493 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:34 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:27:34.493 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:34 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:27:34.493 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:34 smithi057 ceph-mon[40588]: Adjusting osd_memory_target on smithi107 to 5696M 2024-06-11T13:27:34.493 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:34 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:34.493 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:34 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:27:34.493 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:34 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:27:34.493 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:34 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:34.493 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:34 smithi057 ceph-mon[40588]: osdmap e50: 7 total, 7 up, 7 in 2024-06-11T13:27:34.535 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:34 smithi107 ceph-mon[33178]: Detected new or changed devices on smithi107 2024-06-11T13:27:34.535 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:34 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:34.535 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:34 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:34.535 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:34 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:27:34.536 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:34 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:27:34.536 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:34 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:27:34.536 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:34 smithi107 ceph-mon[33178]: Adjusting osd_memory_target on smithi107 to 5696M 2024-06-11T13:27:34.536 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:34 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:34.536 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:34 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:27:34.536 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:34 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:27:34.536 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:34 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:34.536 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:34 smithi107 ceph-mon[33178]: osdmap e50: 7 total, 7 up, 7 in 2024-06-11T13:27:34.562 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi107:vg_nvme/lv_1 2024-06-11T13:27:34.797 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.b/config 2024-06-11T13:27:35.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:35 smithi057 ceph-mon[29805]: pgmap v154: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-11T13:27:35.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:35 smithi057 ceph-mon[29805]: osdmap e51: 7 total, 7 up, 7 in 2024-06-11T13:27:35.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:35 smithi057 ceph-mon[40588]: pgmap v154: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-11T13:27:35.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:35 smithi057 ceph-mon[40588]: osdmap e51: 7 total, 7 up, 7 in 2024-06-11T13:27:35.795 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:35 smithi107 ceph-mon[33178]: pgmap v154: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-11T13:27:35.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:35 smithi107 ceph-mon[33178]: osdmap e51: 7 total, 7 up, 7 in 2024-06-11T13:27:36.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:36.426+0000 7f5bd3efb640 1 -- 172.21.15.107:0/996394017 >> v1:172.21.15.107:6789/0 conn(0x7f5bcc104ba0 legacy=0x7f5bcc106ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:27:36.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:36.426+0000 7f5bd3efb640 1 -- 172.21.15.107:0/996394017 shutdown_connections 2024-06-11T13:27:36.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:36.426+0000 7f5bd3efb640 1 -- 172.21.15.107:0/996394017 >> 172.21.15.107:0/996394017 conn(0x7f5bcc0fbd10 msgr2=0x7f5bcc0fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:27:36.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:36.426+0000 7f5bd3efb640 1 -- 172.21.15.107:0/996394017 shutdown_connections 2024-06-11T13:27:36.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:36.426+0000 7f5bd3efb640 1 -- 172.21.15.107:0/996394017 wait complete. 2024-06-11T13:27:36.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:36.427+0000 7f5bd3efb640 1 Processor -- start 2024-06-11T13:27:36.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:36.427+0000 7f5bd3efb640 1 -- start start 2024-06-11T13:27:36.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:36.427+0000 7f5bd3efb640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f5bcc196880 con 0x7f5bcc1088e0 2024-06-11T13:27:36.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:36.427+0000 7f5bd3efb640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f5bcc1a2090 con 0x7f5bcc100770 2024-06-11T13:27:36.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:36.427+0000 7f5bd3efb640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f5bcc1a3270 con 0x7f5bcc104ba0 2024-06-11T13:27:36.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:36.427+0000 7f5bd1c97640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f5bcc100770 0x7f5bcc195d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.107:58458/0 (socket says 172.21.15.107:58458) 2024-06-11T13:27:36.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:36.427+0000 7f5bd1c97640 1 -- 172.21.15.107:0/962159688 learned_addr learned my addr 172.21.15.107:0/962159688 (peer_addr_for_me v1:172.21.15.107:0/0) 2024-06-11T13:27:36.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:36.427+0000 7f5bc2ffd640 1 -- 172.21.15.107:0/962159688 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1108719366 0 0) 0x7f5bcc1a2090 con 0x7f5bcc100770 2024-06-11T13:27:36.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:36.427+0000 7f5bc2ffd640 1 -- 172.21.15.107:0/962159688 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5bb8003520 con 0x7f5bcc100770 2024-06-11T13:27:36.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:36.427+0000 7f5bc2ffd640 1 -- 172.21.15.107:0/962159688 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3287184397 0 0) 0x7f5bcc1a3270 con 0x7f5bcc104ba0 2024-06-11T13:27:36.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:36.427+0000 7f5bc2ffd640 1 -- 172.21.15.107:0/962159688 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5bcc1a2090 con 0x7f5bcc104ba0 2024-06-11T13:27:36.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:36.427+0000 7f5bc2ffd640 1 -- 172.21.15.107:0/962159688 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1270944369 0 0) 0x7f5bcc196880 con 0x7f5bcc1088e0 2024-06-11T13:27:36.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:36.427+0000 7f5bc2ffd640 1 -- 172.21.15.107:0/962159688 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5bcc1a3270 con 0x7f5bcc1088e0 2024-06-11T13:27:36.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:36.427+0000 7f5bc2ffd640 1 -- 172.21.15.107:0/962159688 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2609830235 0 0) 0x7f5bb8003520 con 0x7f5bcc100770 2024-06-11T13:27:36.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:36.427+0000 7f5bc2ffd640 1 -- 172.21.15.107:0/962159688 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5bcc196880 con 0x7f5bcc100770 2024-06-11T13:27:36.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:36.427+0000 7f5bc2ffd640 1 -- 172.21.15.107:0/962159688 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f5bc8001870 con 0x7f5bcc100770 2024-06-11T13:27:36.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:36.428+0000 7f5bc2ffd640 1 -- 172.21.15.107:0/962159688 <== mon.1 v1:172.21.15.107:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 760430974 0 0) 0x7f5bcc196880 con 0x7f5bcc100770 2024-06-11T13:27:36.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:36.428+0000 7f5bc2ffd640 1 -- 172.21.15.107:0/962159688 >> v1:172.21.15.57:6790/0 conn(0x7f5bcc104ba0 legacy=0x7f5bcc19c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:27:36.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:36.428+0000 7f5bc2ffd640 1 -- 172.21.15.107:0/962159688 >> v1:172.21.15.57:6789/0 conn(0x7f5bcc1088e0 legacy=0x7f5bcc19f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:27:36.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:36.428+0000 7f5bc2ffd640 1 -- 172.21.15.107:0/962159688 --> v1:172.21.15.107:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bcc1a4450 con 0x7f5bcc100770 2024-06-11T13:27:36.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:36.428+0000 7f5bd3efb640 1 -- 172.21.15.107:0/962159688 --> v1:172.21.15.107:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f5bcc1a34a0 con 0x7f5bcc100770 2024-06-11T13:27:36.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:36.428+0000 7f5bd3efb640 1 -- 172.21.15.107:0/962159688 --> v1:172.21.15.107:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f5bcc1a39e0 con 0x7f5bcc100770 2024-06-11T13:27:36.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:36.428+0000 7f5bc2ffd640 1 -- 172.21.15.107:0/962159688 <== mon.1 v1:172.21.15.107:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f5bc8003aa0 con 0x7f5bcc100770 2024-06-11T13:27:36.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:36.428+0000 7f5bc2ffd640 1 -- 172.21.15.107:0/962159688 <== mon.1 v1:172.21.15.107:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f5bc8005ba0 con 0x7f5bcc100770 2024-06-11T13:27:36.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:36.429+0000 7f5bd3efb640 1 -- 172.21.15.107:0/962159688 --> v1:172.21.15.107:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bb00050b0 con 0x7f5bcc100770 2024-06-11T13:27:36.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:36.429+0000 7f5bc2ffd640 1 -- 172.21.15.107:0/962159688 <== mon.1 v1:172.21.15.107:6789/0 7 ==== mgrmap(e 15) ==== 92081+0+0 (unknown 2221930365 0 0) 0x7f5bc8003610 con 0x7f5bcc100770 2024-06-11T13:27:36.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:36.430+0000 7f5bc2ffd640 1 -- 172.21.15.107:0/962159688 <== mon.1 v1:172.21.15.107:6789/0 8 ==== osd_map(51..51 src has 1..51) ==== 3789+0+0 (unknown 654308837 0 0) 0x7f5bc8088b10 con 0x7f5bcc100770 2024-06-11T13:27:36.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:36.434+0000 7f5bc2ffd640 1 -- 172.21.15.107:0/962159688 <== mon.1 v1:172.21.15.107:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f5bc8056c20 con 0x7f5bcc100770 2024-06-11T13:27:36.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:36.619+0000 7f5bd3efb640 1 -- 172.21.15.107:0/962159688 --> v1:172.21.15.57:6800/2989665238 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f5bb0000bc0 con 0x7f5bb8071020 2024-06-11T13:27:37.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:37 smithi107 ceph-mon[33178]: pgmap v156: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-06-11T13:27:37.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:37 smithi107 ceph-mon[33178]: from='client.24307 v1:172.21.15.107:0/962159688' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:27:37.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:37 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-11T13:27:37.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:37 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-11T13:27:37.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:37 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:27:37.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:37 smithi057 ceph-mon[29805]: pgmap v156: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-06-11T13:27:37.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:37 smithi057 ceph-mon[29805]: from='client.24307 v1:172.21.15.107:0/962159688' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:27:37.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:37 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-11T13:27:37.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:37 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-11T13:27:37.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:37 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:27:37.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:37 smithi057 ceph-mon[40588]: pgmap v156: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-06-11T13:27:37.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:37 smithi057 ceph-mon[40588]: from='client.24307 v1:172.21.15.107:0/962159688' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:27:37.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:37 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-11T13:27:37.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:37 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-11T13:27:37.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:37 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:27:39.719 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:39 smithi107 ceph-mon[33178]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-11T13:27:39.720 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:39 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.107:0/3527202826' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57952071-ea76-4ec1-9462-a78264b11913"}]: dispatch 2024-06-11T13:27:39.720 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:39 smithi107 ceph-mon[33178]: from='client.24313 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57952071-ea76-4ec1-9462-a78264b11913"}]: dispatch 2024-06-11T13:27:39.720 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:39 smithi107 ceph-mon[33178]: from='client.24313 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "57952071-ea76-4ec1-9462-a78264b11913"}]': finished 2024-06-11T13:27:39.720 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:39 smithi107 ceph-mon[33178]: osdmap e52: 8 total, 7 up, 8 in 2024-06-11T13:27:39.720 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:39 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-11T13:27:39.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:39 smithi057 ceph-mon[29805]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-11T13:27:39.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:39 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.107:0/3527202826' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57952071-ea76-4ec1-9462-a78264b11913"}]: dispatch 2024-06-11T13:27:39.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:39 smithi057 ceph-mon[29805]: from='client.24313 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57952071-ea76-4ec1-9462-a78264b11913"}]: dispatch 2024-06-11T13:27:39.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:39 smithi057 ceph-mon[29805]: from='client.24313 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "57952071-ea76-4ec1-9462-a78264b11913"}]': finished 2024-06-11T13:27:39.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:39 smithi057 ceph-mon[29805]: osdmap e52: 8 total, 7 up, 8 in 2024-06-11T13:27:39.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:39 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-11T13:27:39.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:39 smithi057 ceph-mon[40588]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-11T13:27:39.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:39 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.107:0/3527202826' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57952071-ea76-4ec1-9462-a78264b11913"}]: dispatch 2024-06-11T13:27:39.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:39 smithi057 ceph-mon[40588]: from='client.24313 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57952071-ea76-4ec1-9462-a78264b11913"}]: dispatch 2024-06-11T13:27:39.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:39 smithi057 ceph-mon[40588]: from='client.24313 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "57952071-ea76-4ec1-9462-a78264b11913"}]': finished 2024-06-11T13:27:39.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:39 smithi057 ceph-mon[40588]: osdmap e52: 8 total, 7 up, 8 in 2024-06-11T13:27:39.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:39 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-11T13:27:40.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:40 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.107:0/3770270627' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-11T13:27:41.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:40 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.107:0/3770270627' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-11T13:27:41.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:40 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.107:0/3770270627' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-11T13:27:41.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:41 smithi107 ceph-mon[33178]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-06-11T13:27:42.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:41 smithi057 ceph-mon[29805]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-06-11T13:27:42.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:41 smithi057 ceph-mon[40588]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-06-11T13:27:43.750 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:43 smithi107 ceph-mon[33178]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-06-11T13:27:43.834 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:43 smithi057 ceph-mon[40588]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-06-11T13:27:43.835 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:43 smithi057 ceph-mon[29805]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-06-11T13:27:45.879 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:45 smithi107 ceph-mon[33178]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-06-11T13:27:45.879 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:45 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-11T13:27:45.879 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:45 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:27:46.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:45 smithi057 ceph-mon[40588]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-06-11T13:27:46.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:45 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-11T13:27:46.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:45 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:27:46.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:45 smithi057 ceph-mon[29805]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-06-11T13:27:46.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:45 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-11T13:27:46.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:45 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:27:46.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:46 smithi107 ceph-mon[33178]: Deploying daemon osd.7 on smithi107 2024-06-11T13:27:47.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:46 smithi057 ceph-mon[29805]: Deploying daemon osd.7 on smithi107 2024-06-11T13:27:47.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:46 smithi057 ceph-mon[40588]: Deploying daemon osd.7 on smithi107 2024-06-11T13:27:47.823 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:47 smithi107 ceph-mon[33178]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-06-11T13:27:48.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:47 smithi057 ceph-mon[29805]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-06-11T13:27:48.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:47 smithi057 ceph-mon[40588]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-06-11T13:27:49.944 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:49 smithi107 ceph-mon[33178]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-06-11T13:27:50.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:49 smithi057 ceph-mon[29805]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-06-11T13:27:50.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:49 smithi057 ceph-mon[40588]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-06-11T13:27:50.938 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:50 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:27:50.938 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:50 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:50.939 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:50 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:51.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:50 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:27:51.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:50 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:51.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:50 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:51.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:50 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:27:51.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:50 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:51.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:50 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:51.905 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:51 smithi107 ceph-mon[33178]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-06-11T13:27:52.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:51 smithi057 ceph-mon[29805]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-06-11T13:27:52.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:51 smithi057 ceph-mon[40588]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-06-11T13:27:53.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:53.603+0000 7f5bc2ffd640 1 -- 172.21.15.107:0/962159688 <== mgr.14150 v1:172.21.15.57:6800/2989665238 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (unknown 0 0 3254063945) 0x7f5bb0000bc0 con 0x7f5bb8071020 2024-06-11T13:27:53.599 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 7 on host 'smithi107' 2024-06-11T13:27:53.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:53.606+0000 7f5bd3efb640 1 -- 172.21.15.107:0/962159688 >> v1:172.21.15.57:6800/2989665238 conn(0x7f5bb8071020 legacy=0x7f5bb80734e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:27:53.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:53.606+0000 7f5bd3efb640 1 -- 172.21.15.107:0/962159688 >> v1:172.21.15.107:6789/0 conn(0x7f5bcc100770 legacy=0x7f5bcc195d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:27:53.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:53.606+0000 7f5bd3efb640 1 -- 172.21.15.107:0/962159688 shutdown_connections 2024-06-11T13:27:53.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:53.606+0000 7f5bd3efb640 1 -- 172.21.15.107:0/962159688 >> 172.21.15.107:0/962159688 conn(0x7f5bcc0fbd10 msgr2=0x7f5bcc106ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:27:53.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:53.606+0000 7f5bd3efb640 1 -- 172.21.15.107:0/962159688 shutdown_connections 2024-06-11T13:27:53.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:27:53.606+0000 7f5bd3efb640 1 -- 172.21.15.107:0/962159688 wait complete. 2024-06-11T13:27:53.987 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:53 smithi107 ceph-mon[33178]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-06-11T13:27:53.987 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:53 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:54.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:53 smithi057 ceph-mon[29805]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-06-11T13:27:54.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:53 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:54.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:53 smithi057 ceph-mon[40588]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-06-11T13:27:54.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:53 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:54.332 DEBUG:teuthology.orchestra.run.smithi107:osd.7> sudo journalctl -f -n 0 -u ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@osd.7.service 2024-06-11T13:27:54.335 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-06-11T13:27:54.335 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph osd stat -f json 2024-06-11T13:27:54.598 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.a/config 2024-06-11T13:27:54.612 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:54 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:54.612 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:54 smithi107 ceph-mon[33178]: from='osd.7 v1:172.21.15.107:6812/3207390156' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-11T13:27:54.612 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:54 smithi107 ceph-mon[33178]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-11T13:27:54.612 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:54 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:54.613 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:54 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:54.613 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:54 smithi057 ceph-mon[29805]: from='osd.7 v1:172.21.15.107:6812/3207390156' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-11T13:27:54.614 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:54 smithi057 ceph-mon[29805]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-11T13:27:54.614 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:54 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:54.614 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:54 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:54.870 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:54 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:54.870 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:54 smithi057 ceph-mon[40588]: from='osd.7 v1:172.21.15.107:6812/3207390156' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-11T13:27:54.870 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:54 smithi057 ceph-mon[40588]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-11T13:27:54.870 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:54 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:54.871 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:54 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:55.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:54 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:55.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:55 smithi107 ceph-mon[33178]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-06-11T13:27:55.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:55 smithi107 ceph-mon[33178]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-11T13:27:55.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:55 smithi107 ceph-mon[33178]: from='osd.7 v1:172.21.15.107:6812/3207390156' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-11T13:27:55.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:55 smithi107 ceph-mon[33178]: osdmap e53: 8 total, 7 up, 8 in 2024-06-11T13:27:55.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:55 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-11T13:27:55.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:55 smithi107 ceph-mon[33178]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-11T13:27:56.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:55 smithi057 ceph-mon[40588]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-06-11T13:27:56.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:55 smithi057 ceph-mon[40588]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-11T13:27:56.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:55 smithi057 ceph-mon[40588]: from='osd.7 v1:172.21.15.107:6812/3207390156' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-11T13:27:56.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:55 smithi057 ceph-mon[40588]: osdmap e53: 8 total, 7 up, 8 in 2024-06-11T13:27:56.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:55 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-11T13:27:56.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:55 smithi057 ceph-mon[40588]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-11T13:27:56.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:55 smithi057 ceph-mon[29805]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-06-11T13:27:56.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:55 smithi057 ceph-mon[29805]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-11T13:27:56.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:55 smithi057 ceph-mon[29805]: from='osd.7 v1:172.21.15.107:6812/3207390156' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-11T13:27:56.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:55 smithi057 ceph-mon[29805]: osdmap e53: 8 total, 7 up, 8 in 2024-06-11T13:27:56.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:55 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-11T13:27:56.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:55 smithi057 ceph-mon[29805]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-11T13:27:56.327 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 13:27:56 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T13:27:56.076+0000 7f31badf5640 -1 osd.7 0 waiting for initial osdmap 2024-06-11T13:27:56.327 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 13:27:56 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T13:27:56.080+0000 7f31b2c15640 -1 osd.7 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-11T13:27:56.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.412+0000 7fe97a0ba640 1 -- 172.21.15.57:0/1011364232 >> v1:172.21.15.57:6789/0 conn(0x7fe974109a00 legacy=0x7fe97410bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:27:56.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.412+0000 7fe97a0ba640 1 -- 172.21.15.57:0/1011364232 shutdown_connections 2024-06-11T13:27:56.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.412+0000 7fe97a0ba640 1 -- 172.21.15.57:0/1011364232 >> 172.21.15.57:0/1011364232 conn(0x7fe9740fce50 msgr2=0x7fe9740ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:27:56.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.412+0000 7fe97a0ba640 1 -- 172.21.15.57:0/1011364232 shutdown_connections 2024-06-11T13:27:56.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.412+0000 7fe97a0ba640 1 -- 172.21.15.57:0/1011364232 wait complete. 2024-06-11T13:27:56.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.413+0000 7fe97a0ba640 1 Processor -- start 2024-06-11T13:27:56.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.413+0000 7fe97a0ba640 1 -- start start 2024-06-11T13:27:56.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.413+0000 7fe97a0ba640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fe974197750 con 0x7fe974101890 2024-06-11T13:27:56.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.413+0000 7fe97a0ba640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fe9741a2fa0 con 0x7fe974109a00 2024-06-11T13:27:56.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.413+0000 7fe97a0ba640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fe9741a4180 con 0x7fe974105cc0 2024-06-11T13:27:56.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.414+0000 7fe9737fe640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fe974101890 0x7fe974196c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:34520/0 (socket says 172.21.15.57:34520) 2024-06-11T13:27:56.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.414+0000 7fe9737fe640 1 -- 172.21.15.57:0/3904525132 learned_addr learned my addr 172.21.15.57:0/3904525132 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:27:56.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.414+0000 7fe972ffd640 1 --1- >> v1:172.21.15.57:6790/0 conn(0x7fe974105cc0 0x7fe97419d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6790/0 says I am v1:172.21.15.57:42816/0 (socket says 172.21.15.57:42816) 2024-06-11T13:27:56.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.414+0000 7fe970ff9640 1 -- 172.21.15.57:0/3904525132 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3045781116 0 0) 0x7fe9741a4180 con 0x7fe974105cc0 2024-06-11T13:27:56.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.414+0000 7fe970ff9640 1 -- 172.21.15.57:0/3904525132 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe95c0051b0 con 0x7fe974105cc0 2024-06-11T13:27:56.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.414+0000 7fe970ff9640 1 -- 172.21.15.57:0/3904525132 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3985479628 0 0) 0x7fe974197750 con 0x7fe974101890 2024-06-11T13:27:56.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.414+0000 7fe970ff9640 1 -- 172.21.15.57:0/3904525132 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe9741a4180 con 0x7fe974101890 2024-06-11T13:27:56.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.414+0000 7fe970ff9640 1 -- 172.21.15.57:0/3904525132 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3201158049 0 0) 0x7fe9741a2fa0 con 0x7fe974109a00 2024-06-11T13:27:56.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.414+0000 7fe970ff9640 1 -- 172.21.15.57:0/3904525132 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe974197750 con 0x7fe974109a00 2024-06-11T13:27:56.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.414+0000 7fe970ff9640 1 -- 172.21.15.57:0/3904525132 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2221746967 0 0) 0x7fe95c0051b0 con 0x7fe974105cc0 2024-06-11T13:27:56.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.414+0000 7fe970ff9640 1 -- 172.21.15.57:0/3904525132 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe9741a2fa0 con 0x7fe974105cc0 2024-06-11T13:27:56.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.414+0000 7fe970ff9640 1 -- 172.21.15.57:0/3904525132 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fe9680018b0 con 0x7fe974105cc0 2024-06-11T13:27:56.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.414+0000 7fe970ff9640 1 -- 172.21.15.57:0/3904525132 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 708316024 0 0) 0x7fe9741a4180 con 0x7fe974101890 2024-06-11T13:27:56.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.414+0000 7fe970ff9640 1 -- 172.21.15.57:0/3904525132 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe95c0051b0 con 0x7fe974101890 2024-06-11T13:27:56.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.414+0000 7fe970ff9640 1 -- 172.21.15.57:0/3904525132 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 553560202 0 0) 0x7fe974197750 con 0x7fe974109a00 2024-06-11T13:27:56.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.414+0000 7fe970ff9640 1 -- 172.21.15.57:0/3904525132 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe9741a4180 con 0x7fe974109a00 2024-06-11T13:27:56.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.414+0000 7fe970ff9640 1 -- 172.21.15.57:0/3904525132 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fe964001230 con 0x7fe974101890 2024-06-11T13:27:56.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.414+0000 7fe970ff9640 1 -- 172.21.15.57:0/3904525132 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fe96c001be0 con 0x7fe974109a00 2024-06-11T13:27:56.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.414+0000 7fe970ff9640 1 -- 172.21.15.57:0/3904525132 <== mon.2 v1:172.21.15.57:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4074514363 0 0) 0x7fe9741a2fa0 con 0x7fe974105cc0 2024-06-11T13:27:56.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.414+0000 7fe970ff9640 1 -- 172.21.15.57:0/3904525132 >> v1:172.21.15.107:6789/0 conn(0x7fe974109a00 legacy=0x7fe9741a0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:27:56.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.414+0000 7fe970ff9640 1 -- 172.21.15.57:0/3904525132 >> v1:172.21.15.57:6789/0 conn(0x7fe974101890 legacy=0x7fe974196c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:27:56.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.414+0000 7fe970ff9640 1 -- 172.21.15.57:0/3904525132 --> v1:172.21.15.57:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9741a5360 con 0x7fe974105cc0 2024-06-11T13:27:56.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.414+0000 7fe97a0ba640 1 -- 172.21.15.57:0/3904525132 --> v1:172.21.15.57:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fe9741a1ff0 con 0x7fe974105cc0 2024-06-11T13:27:56.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.414+0000 7fe97a0ba640 1 -- 172.21.15.57:0/3904525132 --> v1:172.21.15.57:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fe9741a2530 con 0x7fe974105cc0 2024-06-11T13:27:56.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.415+0000 7fe970ff9640 1 -- 172.21.15.57:0/3904525132 <== mon.2 v1:172.21.15.57:6790/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7fe968003aa0 con 0x7fe974105cc0 2024-06-11T13:27:56.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.415+0000 7fe970ff9640 1 -- 172.21.15.57:0/3904525132 <== mon.2 v1:172.21.15.57:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fe968005b50 con 0x7fe974105cc0 2024-06-11T13:27:56.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.416+0000 7fe97a0ba640 1 -- 172.21.15.57:0/3904525132 --> v1:172.21.15.57:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9600050b0 con 0x7fe974105cc0 2024-06-11T13:27:56.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.416+0000 7fe970ff9640 1 -- 172.21.15.57:0/3904525132 <== mon.2 v1:172.21.15.57:6790/0 7 ==== mgrmap(e 15) ==== 92081+0+0 (unknown 2221930365 0 0) 0x7fe96801c2f0 con 0x7fe974105cc0 2024-06-11T13:27:56.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.417+0000 7fe970ff9640 1 -- 172.21.15.57:0/3904525132 <== mon.2 v1:172.21.15.57:6790/0 8 ==== osd_map(54..54 src has 1..54) ==== 3941+0+0 (unknown 3236727861 0 0) 0x7fe968088920 con 0x7fe974105cc0 2024-06-11T13:27:56.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.422+0000 7fe970ff9640 1 -- 172.21.15.57:0/3904525132 <== mon.2 v1:172.21.15.57:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7fe968056930 con 0x7fe974105cc0 2024-06-11T13:27:56.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.611+0000 7fe97a0ba640 1 -- 172.21.15.57:0/3904525132 --> v1:172.21.15.57:6790/0 -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fe9600053a0 con 0x7fe974105cc0 2024-06-11T13:27:56.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.611+0000 7fe970ff9640 1 -- 172.21.15.57:0/3904525132 <== mon.2 v1:172.21.15.57:6790/0 10 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v54) ==== 74+0+130 (unknown 4107983022 0 4138298884) 0x7fe968056230 con 0x7fe974105cc0 2024-06-11T13:27:56.607 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:27:56.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.613+0000 7fe97a0ba640 1 -- 172.21.15.57:0/3904525132 >> v1:172.21.15.57:6800/2989665238 conn(0x7fe95c06eeb0 legacy=0x7fe95c071370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:27:56.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.613+0000 7fe97a0ba640 1 -- 172.21.15.57:0/3904525132 >> v1:172.21.15.57:6790/0 conn(0x7fe974105cc0 legacy=0x7fe97419d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:27:56.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.613+0000 7fe97a0ba640 1 -- 172.21.15.57:0/3904525132 shutdown_connections 2024-06-11T13:27:56.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.613+0000 7fe97a0ba640 1 -- 172.21.15.57:0/3904525132 >> 172.21.15.57:0/3904525132 conn(0x7fe9740fce50 msgr2=0x7fe9741089f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:27:56.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.613+0000 7fe97a0ba640 1 -- 172.21.15.57:0/3904525132 shutdown_connections 2024-06-11T13:27:56.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:27:56.613+0000 7fe97a0ba640 1 -- 172.21.15.57:0/3904525132 wait complete. 2024-06-11T13:27:56.906 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:56 smithi057 ceph-mon[40588]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-06-11T13:27:56.907 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:56 smithi057 ceph-mon[40588]: osdmap e54: 8 total, 7 up, 8 in 2024-06-11T13:27:56.907 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:56 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-11T13:27:56.907 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:56 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-11T13:27:56.907 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:56 smithi057 ceph-mon[40588]: from='osd.7 ' entity='osd.7' 2024-06-11T13:27:56.907 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:56 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3904525132' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-11T13:27:56.908 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:56 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-11T13:27:56.908 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:56 smithi057 ceph-mon[29805]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-06-11T13:27:56.908 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:56 smithi057 ceph-mon[29805]: osdmap e54: 8 total, 7 up, 8 in 2024-06-11T13:27:56.908 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:56 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-11T13:27:56.909 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:56 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-11T13:27:56.909 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:56 smithi057 ceph-mon[29805]: from='osd.7 ' entity='osd.7' 2024-06-11T13:27:56.909 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:56 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3904525132' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-11T13:27:56.909 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:56 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-11T13:27:57.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:56 smithi107 ceph-mon[33178]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-06-11T13:27:57.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:56 smithi107 ceph-mon[33178]: osdmap e54: 8 total, 7 up, 8 in 2024-06-11T13:27:57.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:56 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-11T13:27:57.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:56 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-11T13:27:57.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:56 smithi107 ceph-mon[33178]: from='osd.7 ' entity='osd.7' 2024-06-11T13:27:57.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:56 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3904525132' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-11T13:27:57.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:56 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-11T13:27:57.519 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":54,"num_osds":8,"num_up_osds":7,"osd_up_since":1718112451,"num_in_osds":8,"osd_in_since":1718112459,"num_remapped_pgs":0} 2024-06-11T13:27:57.733 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:57 smithi107 ceph-mon[33178]: purged_snaps scrub starts 2024-06-11T13:27:57.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:57 smithi107 ceph-mon[33178]: purged_snaps scrub ok 2024-06-11T13:27:57.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:57 smithi107 ceph-mon[33178]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-06-11T13:27:57.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:57 smithi107 ceph-mon[33178]: osd.7 v1:172.21.15.107:6812/3207390156 boot 2024-06-11T13:27:57.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:57 smithi107 ceph-mon[33178]: osdmap e55: 8 total, 8 up, 8 in 2024-06-11T13:27:57.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:57 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-11T13:27:57.849 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:57 smithi107 ceph-mon[33178]: osdmap e56: 8 total, 8 up, 8 in 2024-06-11T13:27:57.851 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:57 smithi057 ceph-mon[40588]: purged_snaps scrub starts 2024-06-11T13:27:57.851 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:57 smithi057 ceph-mon[40588]: purged_snaps scrub ok 2024-06-11T13:27:57.851 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:57 smithi057 ceph-mon[40588]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-06-11T13:27:57.851 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:57 smithi057 ceph-mon[40588]: osd.7 v1:172.21.15.107:6812/3207390156 boot 2024-06-11T13:27:57.851 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:57 smithi057 ceph-mon[40588]: osdmap e55: 8 total, 8 up, 8 in 2024-06-11T13:27:57.851 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:57 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-11T13:27:57.852 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:57 smithi057 ceph-mon[40588]: osdmap e56: 8 total, 8 up, 8 in 2024-06-11T13:27:57.852 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:57 smithi057 ceph-mon[29805]: purged_snaps scrub starts 2024-06-11T13:27:57.852 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:57 smithi057 ceph-mon[29805]: purged_snaps scrub ok 2024-06-11T13:27:57.852 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:57 smithi057 ceph-mon[29805]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-06-11T13:27:57.852 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:57 smithi057 ceph-mon[29805]: osd.7 v1:172.21.15.107:6812/3207390156 boot 2024-06-11T13:27:57.853 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:57 smithi057 ceph-mon[29805]: osdmap e55: 8 total, 8 up, 8 in 2024-06-11T13:27:57.853 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:57 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-11T13:27:57.853 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:57 smithi057 ceph-mon[29805]: osdmap e56: 8 total, 8 up, 8 in 2024-06-11T13:27:58.520 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph osd stat -f json 2024-06-11T13:27:58.786 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.a/config 2024-06-11T13:27:58.909 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:58 smithi057 ceph-mon[29805]: Detected new or changed devices on smithi107 2024-06-11T13:27:58.909 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:58 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:58.909 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:58 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:58.909 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:58 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:27:58.909 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:58 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:27:58.909 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:58 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:27:58.909 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:58 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:27:58.910 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:58 smithi057 ceph-mon[29805]: Adjusting osd_memory_target on smithi107 to 4272M 2024-06-11T13:27:58.910 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:58 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:58.910 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:58 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:27:58.910 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:58 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:27:58.910 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:58 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:58.910 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:58 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:27:58.910 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:58 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:27:58.910 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:58 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:27:58.911 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:58 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:58.911 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:58 smithi057 ceph-mon[29805]: osdmap e57: 8 total, 8 up, 8 in 2024-06-11T13:27:58.911 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:58 smithi057 ceph-mon[40588]: Detected new or changed devices on smithi107 2024-06-11T13:27:58.911 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:58 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:58.911 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:58 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:58.911 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:58 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:27:58.911 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:58 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:27:58.911 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:58 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:27:58.911 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:58 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:27:58.911 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:58 smithi057 ceph-mon[40588]: Adjusting osd_memory_target on smithi107 to 4272M 2024-06-11T13:27:58.911 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:58 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:58.912 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:58 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:27:58.912 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:58 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:27:58.912 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:58 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:58.912 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:58 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:27:58.912 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:58 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:27:58.912 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:58 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:27:58.912 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:58 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:58.912 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:58 smithi057 ceph-mon[40588]: osdmap e57: 8 total, 8 up, 8 in 2024-06-11T13:27:59.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:58 smithi107 ceph-mon[33178]: Detected new or changed devices on smithi107 2024-06-11T13:27:59.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:58 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:59.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:58 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:59.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:58 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:27:59.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:58 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:27:59.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:58 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:27:59.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:58 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:27:59.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:58 smithi107 ceph-mon[33178]: Adjusting osd_memory_target on smithi107 to 4272M 2024-06-11T13:27:59.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:58 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:59.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:58 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:27:59.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:58 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:27:59.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:58 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:59.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:58 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:27:59.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:58 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:27:59.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:58 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:27:59.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:58 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:27:59.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:58 smithi107 ceph-mon[33178]: osdmap e57: 8 total, 8 up, 8 in 2024-06-11T13:28:00.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:59 smithi057 ceph-mon[40588]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:28:00.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:27:59 smithi057 ceph-mon[40588]: osdmap e58: 8 total, 8 up, 8 in 2024-06-11T13:28:00.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:59 smithi057 ceph-mon[29805]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:28:00.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:27:59 smithi057 ceph-mon[29805]: osdmap e58: 8 total, 8 up, 8 in 2024-06-11T13:28:00.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:59 smithi107 ceph-mon[33178]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:28:00.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:27:59 smithi107 ceph-mon[33178]: osdmap e58: 8 total, 8 up, 8 in 2024-06-11T13:28:00.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:00.494+0000 7f2305f01640 1 -- 172.21.15.57:0/2790967030 >> v1:172.21.15.57:6790/0 conn(0x7f23001099b0 legacy=0x7f230010be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:00.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:00.494+0000 7f2305f01640 1 -- 172.21.15.57:0/2790967030 shutdown_connections 2024-06-11T13:28:00.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:00.494+0000 7f2305f01640 1 -- 172.21.15.57:0/2790967030 >> 172.21.15.57:0/2790967030 conn(0x7f23000fbd40 msgr2=0x7f23000fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:28:00.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:00.494+0000 7f2305f01640 1 -- 172.21.15.57:0/2790967030 shutdown_connections 2024-06-11T13:28:00.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:00.494+0000 7f2305f01640 1 -- 172.21.15.57:0/2790967030 wait complete. 2024-06-11T13:28:00.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:00.494+0000 7f2305f01640 1 Processor -- start 2024-06-11T13:28:00.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:00.495+0000 7f2305f01640 1 -- start start 2024-06-11T13:28:00.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:00.495+0000 7f2305f01640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f23001091d0 con 0x7f2300102330 2024-06-11T13:28:00.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:00.495+0000 7f2305f01640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f23001a5160 con 0x7f2300105e40 2024-06-11T13:28:00.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:00.495+0000 7f2305f01640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f23001a6340 con 0x7f23001099b0 2024-06-11T13:28:00.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:00.495+0000 7f22ff7fe640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f2300102330 0x7f23001086c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:33172/0 (socket says 172.21.15.57:33172) 2024-06-11T13:28:00.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:00.495+0000 7f22ff7fe640 1 -- 172.21.15.57:0/3144607290 learned_addr learned my addr 172.21.15.57:0/3144607290 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:28:00.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:00.495+0000 7f22fcff9640 1 -- 172.21.15.57:0/3144607290 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3013727341 0 0) 0x7f23001091d0 con 0x7f2300102330 2024-06-11T13:28:00.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:00.495+0000 7f22fcff9640 1 -- 172.21.15.57:0/3144607290 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f22e80051b0 con 0x7f2300102330 2024-06-11T13:28:00.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:00.495+0000 7f22fcff9640 1 -- 172.21.15.57:0/3144607290 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3146253552 0 0) 0x7f23001a5160 con 0x7f2300105e40 2024-06-11T13:28:00.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:00.495+0000 7f22fcff9640 1 -- 172.21.15.57:0/3144607290 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23001091d0 con 0x7f2300105e40 2024-06-11T13:28:00.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:00.495+0000 7f22fcff9640 1 -- 172.21.15.57:0/3144607290 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1420443394 0 0) 0x7f23001a6340 con 0x7f23001099b0 2024-06-11T13:28:00.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:00.495+0000 7f22fcff9640 1 -- 172.21.15.57:0/3144607290 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23001a5160 con 0x7f23001099b0 2024-06-11T13:28:00.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:00.495+0000 7f22fcff9640 1 -- 172.21.15.57:0/3144607290 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3367467195 0 0) 0x7f23001091d0 con 0x7f2300105e40 2024-06-11T13:28:00.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:00.495+0000 7f22fcff9640 1 -- 172.21.15.57:0/3144607290 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f23001a6340 con 0x7f2300105e40 2024-06-11T13:28:00.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:00.495+0000 7f22fcff9640 1 -- 172.21.15.57:0/3144607290 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f22f40013d0 con 0x7f2300105e40 2024-06-11T13:28:00.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:00.496+0000 7f22fcff9640 1 -- 172.21.15.57:0/3144607290 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2353418803 0 0) 0x7f22e80051b0 con 0x7f2300102330 2024-06-11T13:28:00.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:00.496+0000 7f22fcff9640 1 -- 172.21.15.57:0/3144607290 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f23001091d0 con 0x7f2300102330 2024-06-11T13:28:00.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:00.496+0000 7f22fcff9640 1 -- 172.21.15.57:0/3144607290 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 219697727 0 0) 0x7f23001a5160 con 0x7f23001099b0 2024-06-11T13:28:00.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:00.496+0000 7f22fcff9640 1 -- 172.21.15.57:0/3144607290 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f22e80051b0 con 0x7f23001099b0 2024-06-11T13:28:00.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:00.496+0000 7f22fcff9640 1 -- 172.21.15.57:0/3144607290 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f22f00018b0 con 0x7f2300102330 2024-06-11T13:28:00.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:00.496+0000 7f22fcff9640 1 -- 172.21.15.57:0/3144607290 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f22f8001be0 con 0x7f23001099b0 2024-06-11T13:28:00.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:00.496+0000 7f22fcff9640 1 -- 172.21.15.57:0/3144607290 <== mon.1 v1:172.21.15.107:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2475843200 0 0) 0x7f23001a6340 con 0x7f2300105e40 2024-06-11T13:28:00.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:00.496+0000 7f22fcff9640 1 -- 172.21.15.57:0/3144607290 >> v1:172.21.15.57:6790/0 conn(0x7f23001099b0 legacy=0x7f23001a2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:00.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:00.496+0000 7f22fcff9640 1 -- 172.21.15.57:0/3144607290 >> v1:172.21.15.57:6789/0 conn(0x7f2300102330 legacy=0x7f23001086c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:00.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:00.496+0000 7f22fcff9640 1 -- 172.21.15.57:0/3144607290 --> v1:172.21.15.107:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23001a7520 con 0x7f2300105e40 2024-06-11T13:28:00.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:00.496+0000 7f2305f01640 1 -- 172.21.15.57:0/3144607290 --> v1:172.21.15.107:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f23001a6570 con 0x7f2300105e40 2024-06-11T13:28:00.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:00.496+0000 7f2305f01640 1 -- 172.21.15.57:0/3144607290 --> v1:172.21.15.107:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f23001a6ab0 con 0x7f2300105e40 2024-06-11T13:28:00.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:00.496+0000 7f22fcff9640 1 -- 172.21.15.57:0/3144607290 <== mon.1 v1:172.21.15.107:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f22f4001d30 con 0x7f2300105e40 2024-06-11T13:28:00.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:00.496+0000 7f22fcff9640 1 -- 172.21.15.57:0/3144607290 <== mon.1 v1:172.21.15.107:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f22f4004870 con 0x7f2300105e40 2024-06-11T13:28:00.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:00.497+0000 7f2305f01640 1 -- 172.21.15.57:0/3144607290 --> v1:172.21.15.107:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22ec005080 con 0x7f2300105e40 2024-06-11T13:28:00.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:00.497+0000 7f22fcff9640 1 -- 172.21.15.57:0/3144607290 <== mon.1 v1:172.21.15.107:6789/0 7 ==== mgrmap(e 15) ==== 92081+0+0 (unknown 2221930365 0 0) 0x7f22f40038a0 con 0x7f2300105e40 2024-06-11T13:28:00.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:00.498+0000 7f22fcff9640 1 -- 172.21.15.57:0/3144607290 <== mon.1 v1:172.21.15.107:6789/0 8 ==== osd_map(59..59 src has 1..59) ==== 4081+0+0 (unknown 2363693774 0 0) 0x7f22f40877c0 con 0x7f2300105e40 2024-06-11T13:28:00.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:00.502+0000 7f22fcff9640 1 -- 172.21.15.57:0/3144607290 <== mon.1 v1:172.21.15.107:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f22f4055740 con 0x7f2300105e40 2024-06-11T13:28:00.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:00.691+0000 7f2305f01640 1 -- 172.21.15.57:0/3144607290 --> v1:172.21.15.107:6789/0 -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f22ec005370 con 0x7f2300105e40 2024-06-11T13:28:00.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:00.691+0000 7f22fcff9640 1 -- 172.21.15.57:0/3144607290 <== mon.1 v1:172.21.15.107:6789/0 10 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v59) ==== 74+0+130 (unknown 288024937 0 969881495) 0x7f22f4055040 con 0x7f2300105e40 2024-06-11T13:28:00.688 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:28:00.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:00.694+0000 7f2305f01640 1 -- 172.21.15.57:0/3144607290 >> v1:172.21.15.57:6800/2989665238 conn(0x7f22e806ee30 legacy=0x7f22e80712f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:00.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:00.694+0000 7f2305f01640 1 -- 172.21.15.57:0/3144607290 >> v1:172.21.15.107:6789/0 conn(0x7f2300105e40 legacy=0x7f230019f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:00.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:00.694+0000 7f2305f01640 1 -- 172.21.15.57:0/3144607290 shutdown_connections 2024-06-11T13:28:00.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:00.694+0000 7f2305f01640 1 -- 172.21.15.57:0/3144607290 >> 172.21.15.57:0/3144607290 conn(0x7f23000fbd40 msgr2=0x7f2300100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:28:00.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:00.694+0000 7f2305f01640 1 -- 172.21.15.57:0/3144607290 shutdown_connections 2024-06-11T13:28:00.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:00.694+0000 7f2305f01640 1 -- 172.21.15.57:0/3144607290 wait complete. 2024-06-11T13:28:01.295 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":59,"num_osds":8,"num_up_osds":8,"osd_up_since":1718112477,"num_in_osds":8,"osd_in_since":1718112459,"num_remapped_pgs":0} 2024-06-11T13:28:01.296 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-11T13:28:01.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:01 smithi057 ceph-mon[40588]: osdmap e59: 8 total, 8 up, 8 in 2024-06-11T13:28:01.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:01 smithi057 ceph-mon[40588]: pgmap v176: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail; 170 KiB/s, 0 objects/s recovering 2024-06-11T13:28:01.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:01 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3144607290' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-11T13:28:01.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:01 smithi057 ceph-mon[29805]: osdmap e59: 8 total, 8 up, 8 in 2024-06-11T13:28:01.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:01 smithi057 ceph-mon[29805]: pgmap v176: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail; 170 KiB/s, 0 objects/s recovering 2024-06-11T13:28:01.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:01 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3144607290' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-11T13:28:01.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:01 smithi107 ceph-mon[33178]: osdmap e59: 8 total, 8 up, 8 in 2024-06-11T13:28:01.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:01 smithi107 ceph-mon[33178]: pgmap v176: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail; 170 KiB/s, 0 objects/s recovering 2024-06-11T13:28:01.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:01 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3144607290' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-11T13:28:01.548 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.a/config 2024-06-11T13:28:03.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.237+0000 7f3ac5108640 1 -- 172.21.15.57:0/2861325238 >> v1:172.21.15.57:6790/0 conn(0x7f3ac01029e0 legacy=0x7f3ac0102dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:03.257 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.237+0000 7f3ac5108640 1 -- 172.21.15.57:0/2861325238 shutdown_connections 2024-06-11T13:28:03.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.237+0000 7f3ac5108640 1 -- 172.21.15.57:0/2861325238 >> 172.21.15.57:0/2861325238 conn(0x7f3ac00fc230 msgr2=0x7f3ac00fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:28:03.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.237+0000 7f3ac5108640 1 -- 172.21.15.57:0/2861325238 shutdown_connections 2024-06-11T13:28:03.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.237+0000 7f3ac5108640 1 -- 172.21.15.57:0/2861325238 wait complete. 2024-06-11T13:28:03.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.238+0000 7f3ac5108640 1 Processor -- start 2024-06-11T13:28:03.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.238+0000 7f3ac5108640 1 -- start start 2024-06-11T13:28:03.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.239+0000 7f3ac5108640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f3ac01a8ba0 con 0x7f3ac010a7f0 2024-06-11T13:28:03.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.239+0000 7f3ac5108640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f3ac01a9de0 con 0x7f3ac010e300 2024-06-11T13:28:03.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.239+0000 7f3ac5108640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f3ac01ab020 con 0x7f3ac01029e0 2024-06-11T13:28:03.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.239+0000 7f3abed9d640 1 --1- >> v1:172.21.15.57:6790/0 conn(0x7f3ac01029e0 0x7f3ac010a0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6790/0 says I am v1:172.21.15.57:34362/0 (socket says 172.21.15.57:34362) 2024-06-11T13:28:03.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.239+0000 7f3ab7fff640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f3ac010a7f0 0x7f3ac01a3af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:33186/0 (socket says 172.21.15.57:33186) 2024-06-11T13:28:03.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.239+0000 7f3abed9d640 1 -- 172.21.15.57:0/2355691136 learned_addr learned my addr 172.21.15.57:0/2355691136 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:28:03.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.239+0000 7f3abcd99640 1 -- 172.21.15.57:0/2355691136 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 753966327 0 0) 0x7f3ac01a8ba0 con 0x7f3ac010a7f0 2024-06-11T13:28:03.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.239+0000 7f3abcd99640 1 -- 172.21.15.57:0/2355691136 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3aac003520 con 0x7f3ac010a7f0 2024-06-11T13:28:03.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.239+0000 7f3abcd99640 1 -- 172.21.15.57:0/2355691136 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3664163522 0 0) 0x7f3ac01ab020 con 0x7f3ac01029e0 2024-06-11T13:28:03.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.239+0000 7f3abcd99640 1 -- 172.21.15.57:0/2355691136 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ac01a8ba0 con 0x7f3ac01029e0 2024-06-11T13:28:03.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.239+0000 7f3abcd99640 1 -- 172.21.15.57:0/2355691136 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2399252449 0 0) 0x7f3ac01a9de0 con 0x7f3ac010e300 2024-06-11T13:28:03.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.239+0000 7f3abcd99640 1 -- 172.21.15.57:0/2355691136 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ac01ab020 con 0x7f3ac010e300 2024-06-11T13:28:03.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.239+0000 7f3abcd99640 1 -- 172.21.15.57:0/2355691136 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 558939874 0 0) 0x7f3aac003520 con 0x7f3ac010a7f0 2024-06-11T13:28:03.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.239+0000 7f3abcd99640 1 -- 172.21.15.57:0/2355691136 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f3ac01a9de0 con 0x7f3ac010a7f0 2024-06-11T13:28:03.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.239+0000 7f3abcd99640 1 -- 172.21.15.57:0/2355691136 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f3aa8001820 con 0x7f3ac010a7f0 2024-06-11T13:28:03.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.239+0000 7f3abcd99640 1 -- 172.21.15.57:0/2355691136 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 380456874 0 0) 0x7f3ac01a8ba0 con 0x7f3ac01029e0 2024-06-11T13:28:03.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.239+0000 7f3abcd99640 1 -- 172.21.15.57:0/2355691136 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f3aac003520 con 0x7f3ac01029e0 2024-06-11T13:28:03.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.239+0000 7f3abcd99640 1 -- 172.21.15.57:0/2355691136 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f3ab0004490 con 0x7f3ac01029e0 2024-06-11T13:28:03.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.239+0000 7f3abcd99640 1 -- 172.21.15.57:0/2355691136 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 481730043 0 0) 0x7f3ac01ab020 con 0x7f3ac010e300 2024-06-11T13:28:03.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.239+0000 7f3abcd99640 1 -- 172.21.15.57:0/2355691136 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f3ac01a8ba0 con 0x7f3ac010e300 2024-06-11T13:28:03.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.239+0000 7f3abcd99640 1 -- 172.21.15.57:0/2355691136 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f3ab80013f0 con 0x7f3ac010e300 2024-06-11T13:28:03.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.239+0000 7f3abcd99640 1 -- 172.21.15.57:0/2355691136 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2305631930 0 0) 0x7f3ac01a9de0 con 0x7f3ac010a7f0 2024-06-11T13:28:03.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.239+0000 7f3abcd99640 1 -- 172.21.15.57:0/2355691136 >> v1:172.21.15.57:6790/0 conn(0x7f3ac01029e0 legacy=0x7f3ac010a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:03.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.239+0000 7f3abcd99640 1 -- 172.21.15.57:0/2355691136 >> v1:172.21.15.107:6789/0 conn(0x7f3ac010e300 legacy=0x7f3ac01a72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:03.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.239+0000 7f3abcd99640 1 -- 172.21.15.57:0/2355691136 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ac01ac260 con 0x7f3ac010a7f0 2024-06-11T13:28:03.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.240+0000 7f3ac5108640 1 -- 172.21.15.57:0/2355691136 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f3ac01ab250 con 0x7f3ac010a7f0 2024-06-11T13:28:03.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.240+0000 7f3ac5108640 1 -- 172.21.15.57:0/2355691136 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f3ac01ab710 con 0x7f3ac010a7f0 2024-06-11T13:28:03.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.240+0000 7f3abcd99640 1 -- 172.21.15.57:0/2355691136 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f3aa8003aa0 con 0x7f3ac010a7f0 2024-06-11T13:28:03.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.240+0000 7f3abcd99640 1 -- 172.21.15.57:0/2355691136 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f3aa8005b30 con 0x7f3ac010a7f0 2024-06-11T13:28:03.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.241+0000 7f3ac5108640 1 -- 172.21.15.57:0/2355691136 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3aa0005080 con 0x7f3ac010a7f0 2024-06-11T13:28:03.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.241+0000 7f3abcd99640 1 -- 172.21.15.57:0/2355691136 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 15) ==== 92081+0+0 (unknown 2221930365 0 0) 0x7f3aa8003610 con 0x7f3ac010a7f0 2024-06-11T13:28:03.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.242+0000 7f3abcd99640 1 -- 172.21.15.57:0/2355691136 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(59..59 src has 1..59) ==== 4081+0+0 (unknown 2363693774 0 0) 0x7f3aa8088a00 con 0x7f3ac010a7f0 2024-06-11T13:28:03.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.246+0000 7f3abcd99640 1 -- 172.21.15.57:0/2355691136 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f3aa80569e0 con 0x7f3ac010a7f0 2024-06-11T13:28:03.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.431+0000 7f3ac5108640 1 -- 172.21.15.57:0/2355691136 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3aa0005370 con 0x7f3ac010a7f0 2024-06-11T13:28:03.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.432+0000 7f3abcd99640 1 -- 172.21.15.57:0/2355691136 <== mon.0 v1:172.21.15.57:6789/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+11486 (unknown 1651271748 0 3634671469) 0x7f3aa80562e0 con 0x7f3ac010a7f0 2024-06-11T13:28:03.428 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:28:03.428 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":59,"fsid":"98a6635e-27f5-11ef-bc9b-c7b262605968","created":"2024-06-11T13:22:52.358653+0000","modified":"2024-06-11T13:28:00.347837+0000","last_up_change":"2024-06-11T13:27:57.083673+0000","last_in_change":"2024-06-11T13:27:39.099229+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-11T13:25:58.933362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b167b692-f22f-4806-ad75-e94e0ed1e57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6801","nonce":978735223}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6802","nonce":978735223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6804","nonce":978735223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6803","nonce":978735223}]},"public_addr":"172.21.15.57:6801/978735223","cluster_addr":"172.21.15.57:6802/978735223","heartbeat_back_addr":"172.21.15.57:6804/978735223","heartbeat_front_addr":"172.21.15.57:6803/978735223","state":["exists","up"]},{"osd":1,"uuid":"3e153beb-619d-47ba-ac9f-12d2fbc69b1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6805","nonce":137889538}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6806","nonce":137889538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6808","nonce":137889538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6807","nonce":137889538}]},"public_addr":"172.21.15.57:6805/137889538","cluster_addr":"172.21.15.57:6806/137889538","heartbeat_back_addr":"172.21.15.57:6808/137889538","heartbeat_front_addr":"172.21.15.57:6807/137889538","state":["exists","up"]},{"osd":2,"uuid":"0083f7a4-f96d-4efe-a2e9-4c9a4a66c0fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6809","nonce":385321295}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6810","nonce":385321295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6812","nonce":385321295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6811","nonce":385321295}]},"public_addr":"172.21.15.57:6809/385321295","cluster_addr":"172.21.15.57:6810/385321295","heartbeat_back_addr":"172.21.15.57:6812/385321295","heartbeat_front_addr":"172.21.15.57:6811/385321295","state":["exists","up"]},{"osd":3,"uuid":"21aeff39-ee19-45ed-bb85-51827aa0a89e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6813","nonce":1307945371}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6814","nonce":1307945371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6816","nonce":1307945371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6815","nonce":1307945371}]},"public_addr":"172.21.15.57:6813/1307945371","cluster_addr":"172.21.15.57:6814/1307945371","heartbeat_back_addr":"172.21.15.57:6816/1307945371","heartbeat_front_addr":"172.21.15.57:6815/1307945371","state":["exists","up"]},{"osd":4,"uuid":"efdccdfc-63d9-42f1-968e-0c411294b675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6800","nonce":2109046293}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6801","nonce":2109046293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6803","nonce":2109046293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6802","nonce":2109046293}]},"public_addr":"172.21.15.107:6800/2109046293","cluster_addr":"172.21.15.107:6801/2109046293","heartbeat_back_addr":"172.21.15.107:6803/2109046293","heartbeat_front_addr":"172.21.15.107:6802/2109046293","state":["exists","up"]},{"osd":5,"uuid":"beac608b-6839-4291-b866-69f354c0fc89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6804","nonce":2372371519}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6805","nonce":2372371519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6807","nonce":2372371519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6806","nonce":2372371519}]},"public_addr":"172.21.15.107:6804/2372371519","cluster_addr":"172.21.15.107:6805/2372371519","heartbeat_back_addr":"172.21.15.107:6807/2372371519","heartbeat_front_addr":"172.21.15.107:6806/2372371519","state":["exists","up"]},{"osd":6,"uuid":"d4f8d2d5-dabb-42ad-baea-cd5de8269914","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6808","nonce":1305047914}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6809","nonce":1305047914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6811","nonce":1305047914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6810","nonce":1305047914}]},"public_addr":"172.21.15.107:6808/1305047914","cluster_addr":"172.21.15.107:6809/1305047914","heartbeat_back_addr":"172.21.15.107:6811/1305047914","heartbeat_front_addr":"172.21.15.107:6810/1305047914","state":["exists","up"]},{"osd":7,"uuid":"57952071-ea76-4ec1-9462-a78264b11913","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6812","nonce":3207390156}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6813","nonce":3207390156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6815","nonce":3207390156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6814","nonce":3207390156}]},"public_addr":"172.21.15.107:6812/3207390156","cluster_addr":"172.21.15.107:6813/3207390156","heartbeat_back_addr":"172.21.15.107:6815/3207390156","heartbeat_front_addr":"172.21.15.107:6814/3207390156","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-11T13:25:06.508704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-11T13:25:31.399208+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-11T13:25:56.463834+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-11T13:26:21.380185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-11T13:26:42.749674+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-11T13:27:06.043909+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-11T13:27:29.706032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-11T13:27:54.608124+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/4123200049":"2024-06-12T13:23:46.423712+0000","172.21.15.57:0/602127561":"2024-06-12T13:23:20.613278+0000","172.21.15.57:0/1037722376":"2024-06-12T13:23:46.423712+0000","172.21.15.57:6800/262061228":"2024-06-12T13:23:20.613278+0000","172.21.15.57:0/482760167":"2024-06-12T13:23:46.423712+0000","172.21.15.57:6800/1064738157":"2024-06-12T13:23:46.423712+0000","172.21.15.57:0/3067099028":"2024-06-12T13:23:20.613278+0000","172.21.15.57:0/380379402":"2024-06-12T13:23:20.613278+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-11T13:28:03.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.434+0000 7f3ac5108640 1 -- 172.21.15.57:0/2355691136 >> v1:172.21.15.57:6800/2989665238 conn(0x7f3aac06ee00 legacy=0x7f3aac0712c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:03.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.434+0000 7f3ac5108640 1 -- 172.21.15.57:0/2355691136 >> v1:172.21.15.57:6789/0 conn(0x7f3ac010a7f0 legacy=0x7f3ac01a3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:03.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.434+0000 7f3ac5108640 1 -- 172.21.15.57:0/2355691136 shutdown_connections 2024-06-11T13:28:03.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.434+0000 7f3ac5108640 1 -- 172.21.15.57:0/2355691136 >> 172.21.15.57:0/2355691136 conn(0x7f3ac00fc230 msgr2=0x7f3ac00fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:28:03.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.434+0000 7f3ac5108640 1 -- 172.21.15.57:0/2355691136 shutdown_connections 2024-06-11T13:28:03.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:03.434+0000 7f3ac5108640 1 -- 172.21.15.57:0/2355691136 wait complete. 2024-06-11T13:28:03.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:03 smithi057 ceph-mon[29805]: pgmap v177: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-06-11T13:28:03.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:03 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2355691136' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-11T13:28:03.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:03 smithi057 ceph-mon[40588]: pgmap v177: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-06-11T13:28:03.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:03 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2355691136' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-11T13:28:03.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:03 smithi107 ceph-mon[33178]: pgmap v177: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-06-11T13:28:03.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:03 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2355691136' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-11T13:28:04.100 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-11T13:25:58.933362+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-06-11T13:28:04.101 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-06-11T13:28:04.348 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.a/config 2024-06-11T13:28:05.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:05 smithi057 ceph-mon[29805]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-06-11T13:28:05.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:05 smithi057 ceph-mon[40588]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-06-11T13:28:05.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:05 smithi107 ceph-mon[33178]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-06-11T13:28:06.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:06.100+0000 7fcf3159e640 1 -- 172.21.15.57:0/494718340 >> v1:172.21.15.57:6789/0 conn(0x7fcf2c107670 legacy=0x7fcf2c107a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:06.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:06.100+0000 7fcf3159e640 1 -- 172.21.15.57:0/494718340 shutdown_connections 2024-06-11T13:28:06.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:06.100+0000 7fcf3159e640 1 -- 172.21.15.57:0/494718340 >> 172.21.15.57:0/494718340 conn(0x7fcf2c0fd140 msgr2=0x7fcf2c0ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:28:06.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:06.100+0000 7fcf3159e640 1 -- 172.21.15.57:0/494718340 shutdown_connections 2024-06-11T13:28:06.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:06.100+0000 7fcf3159e640 1 -- 172.21.15.57:0/494718340 wait complete. 2024-06-11T13:28:06.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:06.101+0000 7fcf3159e640 1 Processor -- start 2024-06-11T13:28:06.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:06.101+0000 7fcf3159e640 1 -- start start 2024-06-11T13:28:06.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:06.101+0000 7fcf3159e640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fcf2c1a8770 con 0x7fcf2c107670 2024-06-11T13:28:06.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:06.101+0000 7fcf3159e640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fcf2c1a9950 con 0x7fcf2c10e0c0 2024-06-11T13:28:06.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:06.101+0000 7fcf3159e640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fcf2c1aab90 con 0x7fcf2c10a520 2024-06-11T13:28:06.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:06.101+0000 7fcf2bfff640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fcf2c107670 0x7fcf2c105910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:33206/0 (socket says 172.21.15.57:33206) 2024-06-11T13:28:06.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:06.101+0000 7fcf2bfff640 1 -- 172.21.15.57:0/242703200 learned_addr learned my addr 172.21.15.57:0/242703200 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:28:06.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:06.102+0000 7fcf297fa640 1 -- 172.21.15.57:0/242703200 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4014661539 0 0) 0x7fcf2c1aab90 con 0x7fcf2c10a520 2024-06-11T13:28:06.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:06.102+0000 7fcf297fa640 1 -- 172.21.15.57:0/242703200 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf14003520 con 0x7fcf2c10a520 2024-06-11T13:28:06.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:06.102+0000 7fcf297fa640 1 -- 172.21.15.57:0/242703200 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4173409242 0 0) 0x7fcf2c1a8770 con 0x7fcf2c107670 2024-06-11T13:28:06.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:06.102+0000 7fcf297fa640 1 -- 172.21.15.57:0/242703200 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf2c1aab90 con 0x7fcf2c107670 2024-06-11T13:28:06.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:06.102+0000 7fcf297fa640 1 -- 172.21.15.57:0/242703200 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 603826792 0 0) 0x7fcf2c1a9950 con 0x7fcf2c10e0c0 2024-06-11T13:28:06.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:06.102+0000 7fcf297fa640 1 -- 172.21.15.57:0/242703200 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf2c1a8770 con 0x7fcf2c10e0c0 2024-06-11T13:28:06.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:06.102+0000 7fcf297fa640 1 -- 172.21.15.57:0/242703200 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1230193818 0 0) 0x7fcf14003520 con 0x7fcf2c10a520 2024-06-11T13:28:06.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:06.102+0000 7fcf297fa640 1 -- 172.21.15.57:0/242703200 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fcf2c1a9950 con 0x7fcf2c10a520 2024-06-11T13:28:06.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:06.102+0000 7fcf297fa640 1 -- 172.21.15.57:0/242703200 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fcf200017b0 con 0x7fcf2c10a520 2024-06-11T13:28:06.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:06.102+0000 7fcf297fa640 1 -- 172.21.15.57:0/242703200 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1804789399 0 0) 0x7fcf2c1aab90 con 0x7fcf2c107670 2024-06-11T13:28:06.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:06.102+0000 7fcf297fa640 1 -- 172.21.15.57:0/242703200 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fcf14003520 con 0x7fcf2c107670 2024-06-11T13:28:06.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:06.102+0000 7fcf297fa640 1 -- 172.21.15.57:0/242703200 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fcf1c001510 con 0x7fcf2c107670 2024-06-11T13:28:06.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:06.102+0000 7fcf297fa640 1 -- 172.21.15.57:0/242703200 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1573592088 0 0) 0x7fcf2c1a8770 con 0x7fcf2c10e0c0 2024-06-11T13:28:06.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:06.102+0000 7fcf297fa640 1 -- 172.21.15.57:0/242703200 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fcf2c1aab90 con 0x7fcf2c10e0c0 2024-06-11T13:28:06.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:06.102+0000 7fcf297fa640 1 -- 172.21.15.57:0/242703200 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fcf24001bf0 con 0x7fcf2c10e0c0 2024-06-11T13:28:06.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:06.102+0000 7fcf297fa640 1 -- 172.21.15.57:0/242703200 <== mon.2 v1:172.21.15.57:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1717963690 0 0) 0x7fcf2c1a9950 con 0x7fcf2c10a520 2024-06-11T13:28:06.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:06.102+0000 7fcf297fa640 1 -- 172.21.15.57:0/242703200 >> v1:172.21.15.107:6789/0 conn(0x7fcf2c10e0c0 legacy=0x7fcf2c1a7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:06.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:06.102+0000 7fcf297fa640 1 -- 172.21.15.57:0/242703200 >> v1:172.21.15.57:6789/0 conn(0x7fcf2c107670 legacy=0x7fcf2c105910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:06.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:06.102+0000 7fcf297fa640 1 -- 172.21.15.57:0/242703200 --> v1:172.21.15.57:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf2c1abdd0 con 0x7fcf2c10a520 2024-06-11T13:28:06.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:06.102+0000 7fcf3159e640 1 -- 172.21.15.57:0/242703200 --> v1:172.21.15.57:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fcf2c1a89a0 con 0x7fcf2c10a520 2024-06-11T13:28:06.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:06.102+0000 7fcf3159e640 1 -- 172.21.15.57:0/242703200 --> v1:172.21.15.57:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fcf2c1a8e60 con 0x7fcf2c10a520 2024-06-11T13:28:06.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:06.102+0000 7fcf297fa640 1 -- 172.21.15.57:0/242703200 <== mon.2 v1:172.21.15.57:6790/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7fcf20003a80 con 0x7fcf2c10a520 2024-06-11T13:28:06.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:06.102+0000 7fcf297fa640 1 -- 172.21.15.57:0/242703200 <== mon.2 v1:172.21.15.57:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fcf20004980 con 0x7fcf2c10a520 2024-06-11T13:28:06.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:06.103+0000 7fcf3159e640 1 -- 172.21.15.57:0/242703200 --> v1:172.21.15.57:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf18005080 con 0x7fcf2c10a520 2024-06-11T13:28:06.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:06.104+0000 7fcf297fa640 1 -- 172.21.15.57:0/242703200 <== mon.2 v1:172.21.15.57:6790/0 7 ==== mgrmap(e 15) ==== 92081+0+0 (unknown 2221930365 0 0) 0x7fcf2001b120 con 0x7fcf2c10a520 2024-06-11T13:28:06.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:06.104+0000 7fcf297fa640 1 -- 172.21.15.57:0/242703200 <== mon.2 v1:172.21.15.57:6790/0 8 ==== osd_map(59..59 src has 1..59) ==== 4081+0+0 (unknown 2363693774 0 0) 0x7fcf200876d0 con 0x7fcf2c10a520 2024-06-11T13:28:06.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:06.109+0000 7fcf297fa640 1 -- 172.21.15.57:0/242703200 <== mon.2 v1:172.21.15.57:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7fcf20055790 con 0x7fcf2c10a520 2024-06-11T13:28:06.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:06.292+0000 7fcf3159e640 1 -- 172.21.15.57:0/242703200 --> v1:172.21.15.57:6790/0 -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcf18005b80 con 0x7fcf2c10a520 2024-06-11T13:28:06.288 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:06.292+0000 7fcf297fa640 1 -- 172.21.15.57:0/242703200 <== mon.2 v1:172.21.15.57:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) ==== 93+0+10 (unknown 4013917427 0 2170607528) 0x7fcf20055090 con 0x7fcf2c10a520 2024-06-11T13:28:06.288 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2024-06-11T13:28:06.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:06.294+0000 7fcf3159e640 1 -- 172.21.15.57:0/242703200 >> v1:172.21.15.57:6800/2989665238 conn(0x7fcf1406edf0 legacy=0x7fcf140712b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:06.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:06.294+0000 7fcf3159e640 1 -- 172.21.15.57:0/242703200 >> v1:172.21.15.57:6790/0 conn(0x7fcf2c10a520 legacy=0x7fcf2c1a3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:06.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:06.294+0000 7fcf3159e640 1 -- 172.21.15.57:0/242703200 shutdown_connections 2024-06-11T13:28:06.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:06.294+0000 7fcf3159e640 1 -- 172.21.15.57:0/242703200 >> 172.21.15.57:0/242703200 conn(0x7fcf2c0fd140 msgr2=0x7fcf2c1115e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:28:06.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:06.294+0000 7fcf3159e640 1 -- 172.21.15.57:0/242703200 shutdown_connections 2024-06-11T13:28:06.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:06.294+0000 7fcf3159e640 1 -- 172.21.15.57:0/242703200 wait complete. 2024-06-11T13:28:06.943 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:06 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/242703200' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-11T13:28:06.943 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:06 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/242703200' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-11T13:28:06.975 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi057 2024-06-11T13:28:06.976 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph orch apply rgw foo.a --placement '1;smithi057=foo.a' 2024-06-11T13:28:07.005 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:06 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/242703200' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-11T13:28:07.226 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.b/config 2024-06-11T13:28:07.627 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:07 smithi107 ceph-mon[33178]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-11T13:28:07.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:07 smithi057 ceph-mon[29805]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-11T13:28:07.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:07 smithi057 ceph-mon[40588]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-11T13:28:08.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:08.719+0000 7fde8cb37640 1 -- 172.21.15.107:0/2323365380 >> v1:172.21.15.107:6789/0 conn(0x7fde88108910 legacy=0x7fde8810add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:08.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:08.720+0000 7fde8cb37640 1 -- 172.21.15.107:0/2323365380 shutdown_connections 2024-06-11T13:28:08.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:08.720+0000 7fde8cb37640 1 -- 172.21.15.107:0/2323365380 >> 172.21.15.107:0/2323365380 conn(0x7fde880fbd40 msgr2=0x7fde880fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:28:08.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:08.720+0000 7fde8cb37640 1 -- 172.21.15.107:0/2323365380 shutdown_connections 2024-06-11T13:28:08.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:08.720+0000 7fde8cb37640 1 -- 172.21.15.107:0/2323365380 wait complete. 2024-06-11T13:28:08.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:08.720+0000 7fde8cb37640 1 Processor -- start 2024-06-11T13:28:08.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:08.720+0000 7fde8cb37640 1 -- start start 2024-06-11T13:28:08.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:08.721+0000 7fde8cb37640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fde8806ac40 con 0x7fde881007a0 2024-06-11T13:28:08.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:08.721+0000 7fde8cb37640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fde881a31c0 con 0x7fde88108910 2024-06-11T13:28:08.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:08.721+0000 7fde8cb37640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fde881a43a0 con 0x7fde88104bd0 2024-06-11T13:28:08.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:08.721+0000 7fde86d9d640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7fde88108910 0x7fde881a0a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.107:35236/0 (socket says 172.21.15.107:35236) 2024-06-11T13:28:08.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:08.721+0000 7fde86d9d640 1 -- 172.21.15.107:0/2571839249 learned_addr learned my addr 172.21.15.107:0/2571839249 (peer_addr_for_me v1:172.21.15.107:0/0) 2024-06-11T13:28:08.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:08.721+0000 7fde777fe640 1 -- 172.21.15.107:0/2571839249 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3839433825 0 0) 0x7fde881a31c0 con 0x7fde88108910 2024-06-11T13:28:08.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:08.721+0000 7fde777fe640 1 -- 172.21.15.107:0/2571839249 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde70003520 con 0x7fde88108910 2024-06-11T13:28:08.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:08.721+0000 7fde777fe640 1 -- 172.21.15.107:0/2571839249 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3447057023 0 0) 0x7fde881a43a0 con 0x7fde88104bd0 2024-06-11T13:28:08.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:08.721+0000 7fde777fe640 1 -- 172.21.15.107:0/2571839249 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde881a31c0 con 0x7fde88104bd0 2024-06-11T13:28:08.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:08.721+0000 7fde777fe640 1 -- 172.21.15.107:0/2571839249 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2072808539 0 0) 0x7fde8806ac40 con 0x7fde881007a0 2024-06-11T13:28:08.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:08.721+0000 7fde777fe640 1 -- 172.21.15.107:0/2571839249 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde881a43a0 con 0x7fde881007a0 2024-06-11T13:28:08.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:08.721+0000 7fde777fe640 1 -- 172.21.15.107:0/2571839249 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 528065455 0 0) 0x7fde70003520 con 0x7fde88108910 2024-06-11T13:28:08.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:08.721+0000 7fde777fe640 1 -- 172.21.15.107:0/2571839249 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fde8806ac40 con 0x7fde88108910 2024-06-11T13:28:08.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:08.721+0000 7fde777fe640 1 -- 172.21.15.107:0/2571839249 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fde800017b0 con 0x7fde88108910 2024-06-11T13:28:08.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:08.721+0000 7fde777fe640 1 -- 172.21.15.107:0/2571839249 <== mon.1 v1:172.21.15.107:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1760831792 0 0) 0x7fde8806ac40 con 0x7fde88108910 2024-06-11T13:28:08.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:08.721+0000 7fde777fe640 1 -- 172.21.15.107:0/2571839249 >> v1:172.21.15.57:6790/0 conn(0x7fde88104bd0 legacy=0x7fde8819d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:08.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:08.721+0000 7fde777fe640 1 -- 172.21.15.107:0/2571839249 >> v1:172.21.15.57:6789/0 conn(0x7fde881007a0 legacy=0x7fde88199c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:08.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:08.721+0000 7fde777fe640 1 -- 172.21.15.107:0/2571839249 --> v1:172.21.15.107:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde881a55c0 con 0x7fde88108910 2024-06-11T13:28:08.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:08.721+0000 7fde8cb37640 1 -- 172.21.15.107:0/2571839249 --> v1:172.21.15.107:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fde881a2210 con 0x7fde88108910 2024-06-11T13:28:08.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:08.722+0000 7fde8cb37640 1 -- 172.21.15.107:0/2571839249 --> v1:172.21.15.107:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fde881a2750 con 0x7fde88108910 2024-06-11T13:28:08.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:08.722+0000 7fde777fe640 1 -- 172.21.15.107:0/2571839249 <== mon.1 v1:172.21.15.107:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7fde80003a10 con 0x7fde88108910 2024-06-11T13:28:08.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:08.722+0000 7fde777fe640 1 -- 172.21.15.107:0/2571839249 <== mon.1 v1:172.21.15.107:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fde80005080 con 0x7fde88108910 2024-06-11T13:28:08.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:08.723+0000 7fde8cb37640 1 -- 172.21.15.107:0/2571839249 --> v1:172.21.15.107:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde680050b0 con 0x7fde88108910 2024-06-11T13:28:08.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:08.723+0000 7fde777fe640 1 -- 172.21.15.107:0/2571839249 <== mon.1 v1:172.21.15.107:6789/0 7 ==== mgrmap(e 15) ==== 92081+0+0 (unknown 2221930365 0 0) 0x7fde800052c0 con 0x7fde88108910 2024-06-11T13:28:08.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:08.723+0000 7fde777fe640 1 -- 172.21.15.107:0/2571839249 <== mon.1 v1:172.21.15.107:6789/0 8 ==== osd_map(59..59 src has 1..59) ==== 4081+0+0 (unknown 2363693774 0 0) 0x7fde80088f30 con 0x7fde88108910 2024-06-11T13:28:08.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:08.728+0000 7fde777fe640 1 -- 172.21.15.107:0/2571839249 <== mon.1 v1:172.21.15.107:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7fde80056ff0 con 0x7fde88108910 2024-06-11T13:28:08.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:08.916+0000 7fde8cb37640 1 -- 172.21.15.107:0/2571839249 --> v1:172.21.15.57:6800/2989665238 -- mgr_command(tid 0: {"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi057=foo.a", "target": ["mon-mgr", ""]}) -- 0x7fde68000bc0 con 0x7fde7006e860 2024-06-11T13:28:08.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:08.925+0000 7fde777fe640 1 -- 172.21.15.107:0/2571839249 <== mgr.14150 v1:172.21.15.57:6800/2989665238 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+30 (unknown 0 0 1123153589) 0x7fde68000bc0 con 0x7fde7006e860 2024-06-11T13:28:08.922 INFO:teuthology.orchestra.run.smithi107.stdout:Scheduled rgw.foo.a update... 2024-06-11T13:28:08.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:08.927+0000 7fde8cb37640 1 -- 172.21.15.107:0/2571839249 >> v1:172.21.15.57:6800/2989665238 conn(0x7fde7006e860 legacy=0x7fde70070d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:08.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:08.927+0000 7fde8cb37640 1 -- 172.21.15.107:0/2571839249 >> v1:172.21.15.107:6789/0 conn(0x7fde88108910 legacy=0x7fde881a0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:08.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:08.927+0000 7fde8cb37640 1 -- 172.21.15.107:0/2571839249 shutdown_connections 2024-06-11T13:28:08.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:08.927+0000 7fde8cb37640 1 -- 172.21.15.107:0/2571839249 >> 172.21.15.107:0/2571839249 conn(0x7fde880fbd40 msgr2=0x7fde8810b550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:28:08.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:08.927+0000 7fde8cb37640 1 -- 172.21.15.107:0/2571839249 shutdown_connections 2024-06-11T13:28:08.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:08.927+0000 7fde8cb37640 1 -- 172.21.15.107:0/2571839249 wait complete. 2024-06-11T13:28:09.497 DEBUG:teuthology.orchestra.run.smithi057:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@rgw.foo.a.service 2024-06-11T13:28:09.499 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi107 2024-06-11T13:28:09.500 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph osd pool create datapool 3 3 replicated 2024-06-11T13:28:09.775 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.b/config 2024-06-11T13:28:10.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:09 smithi057 ceph-mon[29805]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:28:10.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:09 smithi057 ceph-mon[29805]: from='client.24355 v1:172.21.15.107:0/2571839249' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi057=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:28:10.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:09 smithi057 ceph-mon[29805]: Saving service rgw.foo.a spec with placement smithi057=foo.a;count:1 2024-06-11T13:28:10.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:09 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:10.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:09 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:28:10.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:09 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:28:10.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:09 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:28:10.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:09 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:10.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:09 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-11T13:28:10.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:09 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-06-11T13:28:10.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:09 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:10.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:09 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:28:10.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:09 smithi057 ceph-mon[29805]: Deploying daemon rgw.foo.a on smithi057 2024-06-11T13:28:10.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:09 smithi057 ceph-mon[40588]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:28:10.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:09 smithi057 ceph-mon[40588]: from='client.24355 v1:172.21.15.107:0/2571839249' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi057=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:28:10.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:09 smithi057 ceph-mon[40588]: Saving service rgw.foo.a spec with placement smithi057=foo.a;count:1 2024-06-11T13:28:10.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:09 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:10.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:09 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:28:10.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:09 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:28:10.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:09 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:28:10.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:09 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:10.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:09 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-11T13:28:10.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:09 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-06-11T13:28:10.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:09 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:10.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:09 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:28:10.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:09 smithi057 ceph-mon[40588]: Deploying daemon rgw.foo.a on smithi057 2024-06-11T13:28:10.214 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:09 smithi107 ceph-mon[33178]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:28:10.214 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:09 smithi107 ceph-mon[33178]: from='client.24355 v1:172.21.15.107:0/2571839249' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi057=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:28:10.215 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:09 smithi107 ceph-mon[33178]: Saving service rgw.foo.a spec with placement smithi057=foo.a;count:1 2024-06-11T13:28:10.215 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:09 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:10.215 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:09 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:28:10.215 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:09 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:28:10.215 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:09 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:28:10.215 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:09 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:10.215 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:09 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-11T13:28:10.215 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:09 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-06-11T13:28:10.215 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:09 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:10.215 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:09 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:28:10.215 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:09 smithi107 ceph-mon[33178]: Deploying daemon rgw.foo.a on smithi057 2024-06-11T13:28:11.007 INFO:journalctl@ceph.rgw.foo.a.smithi057.stdout:Jun 11 13:28:10 smithi057 systemd[1]: Starting Ceph rgw.foo.a for 98a6635e-27f5-11ef-bc9b-c7b262605968... 2024-06-11T13:28:11.283 INFO:journalctl@ceph.rgw.foo.a.smithi057.stdout:Jun 11 13:28:11 smithi057 podman[67012]: 2024-06-11 13:28:10.913884815 +0000 UTC m=+0.017638616 image pull 2d7459ec65a098c35c774154617f2893b0ad051f56ce89b72ab795f65a97fc22 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310 2024-06-11T13:28:11.283 INFO:journalctl@ceph.rgw.foo.a.smithi057.stdout:Jun 11 13:28:11 smithi057 podman[67012]: 2024-06-11 13:28:11.03133055 +0000 UTC m=+0.135084350 container create 71407d0c32b0175c9467c2a8c84a30521c97013e6a9b04dd478a08445cb3a902 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-rgw-foo-a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.35.3, org.label-schema.build-date=20240528, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-3f51ea3, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-06-11T13:28:11.283 INFO:journalctl@ceph.rgw.foo.a.smithi057.stdout:Jun 11 13:28:11 smithi057 podman[67012]: 2024-06-11 13:28:11.242069467 +0000 UTC m=+0.345823270 container init 71407d0c32b0175c9467c2a8c84a30521c97013e6a9b04dd478a08445cb3a902 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-rgw-foo-a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-3f51ea3, io.buildah.version=1.35.3, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-06-11T13:28:11.283 INFO:journalctl@ceph.rgw.foo.a.smithi057.stdout:Jun 11 13:28:11 smithi057 podman[67012]: 2024-06-11 13:28:11.245496842 +0000 UTC m=+0.349250643 container start 71407d0c32b0175c9467c2a8c84a30521c97013e6a9b04dd478a08445cb3a902 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-rgw-foo-a, maintainer=Guillaume Abrioux , RELEASE=squid-3f51ea3, org.label-schema.build-date=20240528, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.35.3, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-06-11T13:28:11.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:11.366+0000 7fc3be9e6640 1 -- 172.21.15.107:0/1919531400 >> v1:172.21.15.107:6789/0 conn(0x7fc3b8104bb0 legacy=0x7fc3b8107000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:11.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:11.366+0000 7fc3be9e6640 1 -- 172.21.15.107:0/1919531400 shutdown_connections 2024-06-11T13:28:11.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:11.366+0000 7fc3be9e6640 1 -- 172.21.15.107:0/1919531400 >> 172.21.15.107:0/1919531400 conn(0x7fc3b80fbd40 msgr2=0x7fc3b80fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:28:11.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:11.366+0000 7fc3be9e6640 1 -- 172.21.15.107:0/1919531400 shutdown_connections 2024-06-11T13:28:11.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:11.366+0000 7fc3be9e6640 1 -- 172.21.15.107:0/1919531400 wait complete. 2024-06-11T13:28:11.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:11.366+0000 7fc3be9e6640 1 Processor -- start 2024-06-11T13:28:11.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:11.366+0000 7fc3be9e6640 1 -- start start 2024-06-11T13:28:11.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:11.367+0000 7fc3be9e6640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fc3b806ac60 con 0x7fc3b8104bb0 2024-06-11T13:28:11.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:11.367+0000 7fc3be9e6640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fc3b81a3120 con 0x7fc3b81088f0 2024-06-11T13:28:11.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:11.367+0000 7fc3be9e6640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fc3b81a4300 con 0x7fc3b8100780 2024-06-11T13:28:11.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:11.367+0000 7fc3bcf83640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7fc3b81088f0 0x7fc3b81a09f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.107:36914/0 (socket says 172.21.15.107:36914) 2024-06-11T13:28:11.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:11.367+0000 7fc3bcf83640 1 -- 172.21.15.107:0/1393340348 learned_addr learned my addr 172.21.15.107:0/1393340348 (peer_addr_for_me v1:172.21.15.107:0/0) 2024-06-11T13:28:11.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:11.367+0000 7fc3b5ffb640 1 -- 172.21.15.107:0/1393340348 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 978687754 0 0) 0x7fc3b81a3120 con 0x7fc3b81088f0 2024-06-11T13:28:11.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:11.367+0000 7fc3b5ffb640 1 -- 172.21.15.107:0/1393340348 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc3a4003520 con 0x7fc3b81088f0 2024-06-11T13:28:11.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:11.367+0000 7fc3b5ffb640 1 -- 172.21.15.107:0/1393340348 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1472262383 0 0) 0x7fc3b81a4300 con 0x7fc3b8100780 2024-06-11T13:28:11.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:11.367+0000 7fc3b5ffb640 1 -- 172.21.15.107:0/1393340348 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc3b81a3120 con 0x7fc3b8100780 2024-06-11T13:28:11.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:11.367+0000 7fc3b5ffb640 1 -- 172.21.15.107:0/1393340348 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 669768562 0 0) 0x7fc3b81a3120 con 0x7fc3b8100780 2024-06-11T13:28:11.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:11.367+0000 7fc3b5ffb640 1 -- 172.21.15.107:0/1393340348 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc3b81a4300 con 0x7fc3b8100780 2024-06-11T13:28:11.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:11.367+0000 7fc3b5ffb640 1 -- 172.21.15.107:0/1393340348 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fc3a80018b0 con 0x7fc3b8100780 2024-06-11T13:28:11.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:11.367+0000 7fc3b5ffb640 1 -- 172.21.15.107:0/1393340348 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4150247368 0 0) 0x7fc3a4003520 con 0x7fc3b81088f0 2024-06-11T13:28:11.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:11.367+0000 7fc3b5ffb640 1 -- 172.21.15.107:0/1393340348 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc3b81a3120 con 0x7fc3b81088f0 2024-06-11T13:28:11.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:11.367+0000 7fc3b5ffb640 1 -- 172.21.15.107:0/1393340348 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fc3b0001b90 con 0x7fc3b81088f0 2024-06-11T13:28:11.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:11.367+0000 7fc3b5ffb640 1 -- 172.21.15.107:0/1393340348 <== mon.2 v1:172.21.15.57:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2618891264 0 0) 0x7fc3b81a4300 con 0x7fc3b8100780 2024-06-11T13:28:11.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:11.367+0000 7fc3b5ffb640 1 -- 172.21.15.107:0/1393340348 >> v1:172.21.15.107:6789/0 conn(0x7fc3b81088f0 legacy=0x7fc3b81a09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:11.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:11.367+0000 7fc3b5ffb640 1 -- 172.21.15.107:0/1393340348 >> v1:172.21.15.57:6789/0 conn(0x7fc3b8104bb0 legacy=0x7fc3b819d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:11.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:11.367+0000 7fc3b5ffb640 1 -- 172.21.15.107:0/1393340348 --> v1:172.21.15.57:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3b81a5520 con 0x7fc3b8100780 2024-06-11T13:28:11.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:11.367+0000 7fc3be9e6640 1 -- 172.21.15.107:0/1393340348 --> v1:172.21.15.57:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fc3b81a4530 con 0x7fc3b8100780 2024-06-11T13:28:11.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:11.367+0000 7fc3be9e6640 1 -- 172.21.15.107:0/1393340348 --> v1:172.21.15.57:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fc3b81a4a70 con 0x7fc3b8100780 2024-06-11T13:28:11.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:11.368+0000 7fc3b5ffb640 1 -- 172.21.15.107:0/1393340348 <== mon.2 v1:172.21.15.57:6790/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7fc3a8003aa0 con 0x7fc3b8100780 2024-06-11T13:28:11.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:11.368+0000 7fc3b5ffb640 1 -- 172.21.15.107:0/1393340348 <== mon.2 v1:172.21.15.57:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fc3a8004b00 con 0x7fc3b8100780 2024-06-11T13:28:11.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:11.369+0000 7fc3be9e6640 1 -- 172.21.15.107:0/1393340348 --> v1:172.21.15.57:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3980050b0 con 0x7fc3b8100780 2024-06-11T13:28:11.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:11.369+0000 7fc3b5ffb640 1 -- 172.21.15.107:0/1393340348 <== mon.2 v1:172.21.15.57:6790/0 7 ==== mgrmap(e 15) ==== 92081+0+0 (unknown 2221930365 0 0) 0x7fc3a80042b0 con 0x7fc3b8100780 2024-06-11T13:28:11.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:11.369+0000 7fc3b5ffb640 1 -- 172.21.15.107:0/1393340348 <== mon.2 v1:172.21.15.57:6790/0 8 ==== osd_map(59..59 src has 1..59) ==== 4081+0+0 (unknown 2363693774 0 0) 0x7fc3a8087ae0 con 0x7fc3b8100780 2024-06-11T13:28:11.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:11.375+0000 7fc3b5ffb640 1 -- 172.21.15.107:0/1393340348 <== mon.2 v1:172.21.15.57:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7fc3a8055a60 con 0x7fc3b8100780 2024-06-11T13:28:11.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:11.554+0000 7fc3be9e6640 1 -- 172.21.15.107:0/1393340348 --> v1:172.21.15.57:6790/0 -- mon_command({"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} v 0) -- 0x7fc3980053a0 con 0x7fc3b8100780 2024-06-11T13:28:11.782 INFO:journalctl@ceph.rgw.foo.a.smithi057.stdout:Jun 11 13:28:11 smithi057 bash[67012]: 71407d0c32b0175c9467c2a8c84a30521c97013e6a9b04dd478a08445cb3a902 2024-06-11T13:28:11.783 INFO:journalctl@ceph.rgw.foo.a.smithi057.stdout:Jun 11 13:28:11 smithi057 systemd[1]: Started Ceph rgw.foo.a for 98a6635e-27f5-11ef-bc9b-c7b262605968. 2024-06-11T13:28:12.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:12.161+0000 7fc3b5ffb640 1 -- 172.21.15.107:0/1393340348 <== mon.2 v1:172.21.15.57:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]=0 pool 'datapool' created v60) ==== 160+0+0 (unknown 701681216 0 0) 0x7fc3a8055360 con 0x7fc3b8100780 2024-06-11T13:28:12.159 INFO:teuthology.orchestra.run.smithi107.stderr:pool 'datapool' created 2024-06-11T13:28:12.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:12.165+0000 7fc3be9e6640 1 -- 172.21.15.107:0/1393340348 >> v1:172.21.15.57:6800/2989665238 conn(0x7fc3a406ed00 legacy=0x7fc3a40711c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:12.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:12.165+0000 7fc3be9e6640 1 -- 172.21.15.107:0/1393340348 >> v1:172.21.15.57:6790/0 conn(0x7fc3b8100780 legacy=0x7fc3b8199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:12.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:12.165+0000 7fc3be9e6640 1 -- 172.21.15.107:0/1393340348 shutdown_connections 2024-06-11T13:28:12.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:12.165+0000 7fc3be9e6640 1 -- 172.21.15.107:0/1393340348 >> 172.21.15.107:0/1393340348 conn(0x7fc3b80fbd40 msgr2=0x7fc3b8107000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:28:12.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:12.165+0000 7fc3be9e6640 1 -- 172.21.15.107:0/1393340348 shutdown_connections 2024-06-11T13:28:12.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:12.165+0000 7fc3be9e6640 1 -- 172.21.15.107:0/1393340348 wait complete. 2024-06-11T13:28:12.456 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:12 smithi107 ceph-mon[33178]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:28:12.457 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:12 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.107:0/1393340348' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-06-11T13:28:12.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:12 smithi057 ceph-mon[40588]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:28:12.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:12 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.107:0/1393340348' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-06-11T13:28:12.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:12 smithi057 ceph-mon[29805]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:28:12.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:12 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.107:0/1393340348' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-06-11T13:28:12.789 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- rbd pool init datapool 2024-06-11T13:28:13.029 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.b/config 2024-06-11T13:28:13.444 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:13 smithi107 ceph-mon[33178]: from='client.34352 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-06-11T13:28:13.444 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:13 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:13.444 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:13 smithi107 ceph-mon[33178]: from='client.34352 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-06-11T13:28:13.444 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:13 smithi107 ceph-mon[33178]: osdmap e60: 8 total, 8 up, 8 in 2024-06-11T13:28:13.444 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:13 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1445700148' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-06-11T13:28:13.445 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:13 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:13.445 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:13 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:13.445 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:13 smithi107 ceph-mon[33178]: Saving service rgw.foo.a spec with placement smithi057=foo.a;count:1 2024-06-11T13:28:13.445 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:13 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:13.445 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:13 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:13.445 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:13 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:28:13.470 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:13 smithi057 ceph-mon[40588]: from='client.34352 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-06-11T13:28:13.471 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:13 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:13.471 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:13 smithi057 ceph-mon[40588]: from='client.34352 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-06-11T13:28:13.471 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:13 smithi057 ceph-mon[40588]: osdmap e60: 8 total, 8 up, 8 in 2024-06-11T13:28:13.471 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:13 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1445700148' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-06-11T13:28:13.471 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:13 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:13.471 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:13 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:13.471 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:13 smithi057 ceph-mon[40588]: Saving service rgw.foo.a spec with placement smithi057=foo.a;count:1 2024-06-11T13:28:13.471 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:13 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:13.471 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:13 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:13.471 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:13 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:28:13.472 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:13 smithi057 ceph-mon[29805]: from='client.34352 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-06-11T13:28:13.472 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:13 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:13.472 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:13 smithi057 ceph-mon[29805]: from='client.34352 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-06-11T13:28:13.472 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:13 smithi057 ceph-mon[29805]: osdmap e60: 8 total, 8 up, 8 in 2024-06-11T13:28:13.472 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:13 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1445700148' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-06-11T13:28:13.472 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:13 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:13.472 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:13 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:13.472 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:13 smithi057 ceph-mon[29805]: Saving service rgw.foo.a spec with placement smithi057=foo.a;count:1 2024-06-11T13:28:13.472 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:13 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:13.472 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:13 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:13.472 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:13 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:28:14.727 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:14 smithi107 ceph-mon[33178]: pgmap v183: 36 pgs: 5 creating+peering, 30 unknown, 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:28:14.728 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:14 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1445700148' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-06-11T13:28:14.728 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:14 smithi107 ceph-mon[33178]: osdmap e61: 8 total, 8 up, 8 in 2024-06-11T13:28:14.728 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:14 smithi107 ceph-mon[33178]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:28:14.728 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:14 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:14.728 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:14 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:14.728 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:14 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:28:14.728 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:14 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:28:14.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:14 smithi057 ceph-mon[29805]: pgmap v183: 36 pgs: 5 creating+peering, 30 unknown, 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:28:14.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:14 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1445700148' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-06-11T13:28:14.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:14 smithi057 ceph-mon[29805]: osdmap e61: 8 total, 8 up, 8 in 2024-06-11T13:28:14.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:14 smithi057 ceph-mon[29805]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:28:14.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:14 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:14.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:14 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:14.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:14 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:28:14.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:14 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:28:14.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:14 smithi057 ceph-mon[40588]: pgmap v183: 36 pgs: 5 creating+peering, 30 unknown, 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:28:14.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:14 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1445700148' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-06-11T13:28:14.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:14 smithi057 ceph-mon[40588]: osdmap e61: 8 total, 8 up, 8 in 2024-06-11T13:28:14.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:14 smithi057 ceph-mon[40588]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:28:14.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:14 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:14.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:14 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:14.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:14 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:28:14.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:14 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:28:15.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:15 smithi057 ceph-mon[29805]: Checking dashboard <-> RGW credentials 2024-06-11T13:28:15.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:15 smithi057 ceph-mon[29805]: osdmap e62: 8 total, 8 up, 8 in 2024-06-11T13:28:15.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:15 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2628526915' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-06-11T13:28:15.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:15 smithi057 ceph-mon[29805]: from='client.24379 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-06-11T13:28:15.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:15 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3167383653' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-06-11T13:28:15.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:15 smithi057 ceph-mon[29805]: from='client.34391 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-06-11T13:28:15.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:15 smithi057 ceph-mon[29805]: pgmap v186: 68 pgs: 24 active+clean, 11 creating+peering, 33 unknown; 578 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-06-11T13:28:15.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:15 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.107:0/2432399770' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-06-11T13:28:15.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:15 smithi057 ceph-mon[29805]: from='client.24412 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-06-11T13:28:15.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:15 smithi057 ceph-mon[40588]: Checking dashboard <-> RGW credentials 2024-06-11T13:28:15.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:15 smithi057 ceph-mon[40588]: osdmap e62: 8 total, 8 up, 8 in 2024-06-11T13:28:15.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:15 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2628526915' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-06-11T13:28:15.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:15 smithi057 ceph-mon[40588]: from='client.24379 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-06-11T13:28:15.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:15 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3167383653' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-06-11T13:28:15.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:15 smithi057 ceph-mon[40588]: from='client.34391 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-06-11T13:28:15.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:15 smithi057 ceph-mon[40588]: pgmap v186: 68 pgs: 24 active+clean, 11 creating+peering, 33 unknown; 578 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-06-11T13:28:15.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:15 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.107:0/2432399770' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-06-11T13:28:15.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:15 smithi057 ceph-mon[40588]: from='client.24412 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-06-11T13:28:15.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:15 smithi107 ceph-mon[33178]: Checking dashboard <-> RGW credentials 2024-06-11T13:28:15.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:15 smithi107 ceph-mon[33178]: osdmap e62: 8 total, 8 up, 8 in 2024-06-11T13:28:15.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:15 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2628526915' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-06-11T13:28:15.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:15 smithi107 ceph-mon[33178]: from='client.24379 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-06-11T13:28:15.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:15 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3167383653' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-06-11T13:28:15.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:15 smithi107 ceph-mon[33178]: from='client.34391 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-06-11T13:28:15.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:15 smithi107 ceph-mon[33178]: pgmap v186: 68 pgs: 24 active+clean, 11 creating+peering, 33 unknown; 578 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-06-11T13:28:15.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:15 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.107:0/2432399770' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-06-11T13:28:15.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:15 smithi107 ceph-mon[33178]: from='client.24412 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-06-11T13:28:16.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:16 smithi057 ceph-mon[29805]: from='client.24379 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-06-11T13:28:16.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:16 smithi057 ceph-mon[29805]: from='client.34391 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-06-11T13:28:16.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:16 smithi057 ceph-mon[29805]: from='client.24412 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-06-11T13:28:16.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:16 smithi057 ceph-mon[29805]: osdmap e63: 8 total, 8 up, 8 in 2024-06-11T13:28:16.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:16 smithi057 ceph-mon[40588]: from='client.24379 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-06-11T13:28:16.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:16 smithi057 ceph-mon[40588]: from='client.34391 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-06-11T13:28:16.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:16 smithi057 ceph-mon[40588]: from='client.24412 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-06-11T13:28:16.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:16 smithi057 ceph-mon[40588]: osdmap e63: 8 total, 8 up, 8 in 2024-06-11T13:28:16.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:16 smithi107 ceph-mon[33178]: from='client.24379 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-06-11T13:28:16.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:16 smithi107 ceph-mon[33178]: from='client.34391 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-06-11T13:28:16.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:16 smithi107 ceph-mon[33178]: from='client.24412 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-06-11T13:28:16.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:16 smithi107 ceph-mon[33178]: osdmap e63: 8 total, 8 up, 8 in 2024-06-11T13:28:17.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:17 smithi057 ceph-mon[29805]: osdmap e64: 8 total, 8 up, 8 in 2024-06-11T13:28:17.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:17 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3167383653' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-06-11T13:28:17.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:17 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2628526915' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-06-11T13:28:17.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:17 smithi057 ceph-mon[29805]: from='client.34391 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-06-11T13:28:17.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:17 smithi057 ceph-mon[29805]: from='client.24379 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-06-11T13:28:17.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:17 smithi057 ceph-mon[29805]: pgmap v189: 100 pgs: 34 active+clean, 11 creating+peering, 55 unknown; 578 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-06-11T13:28:17.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:17 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:17.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:17 smithi057 ceph-mon[40588]: osdmap e64: 8 total, 8 up, 8 in 2024-06-11T13:28:17.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:17 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3167383653' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-06-11T13:28:17.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:17 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2628526915' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-06-11T13:28:17.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:17 smithi057 ceph-mon[40588]: from='client.34391 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-06-11T13:28:17.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:17 smithi057 ceph-mon[40588]: from='client.24379 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-06-11T13:28:17.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:17 smithi057 ceph-mon[40588]: pgmap v189: 100 pgs: 34 active+clean, 11 creating+peering, 55 unknown; 578 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-06-11T13:28:17.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:17 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:17.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:17 smithi107 ceph-mon[33178]: osdmap e64: 8 total, 8 up, 8 in 2024-06-11T13:28:17.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:17 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3167383653' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-06-11T13:28:17.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:17 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2628526915' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-06-11T13:28:17.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:17 smithi107 ceph-mon[33178]: from='client.34391 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-06-11T13:28:17.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:17 smithi107 ceph-mon[33178]: from='client.24379 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-06-11T13:28:17.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:17 smithi107 ceph-mon[33178]: pgmap v189: 100 pgs: 34 active+clean, 11 creating+peering, 55 unknown; 578 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-06-11T13:28:17.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:17 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:18.094 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.107 --placement '1;smithi107=iscsi.a' 2024-06-11T13:28:18.332 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.b/config 2024-06-11T13:28:18.707 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:18 smithi107 ceph-mon[33178]: from='client.34391 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-06-11T13:28:18.708 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:18 smithi107 ceph-mon[33178]: from='client.24379 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-06-11T13:28:18.708 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:18 smithi107 ceph-mon[33178]: osdmap e65: 8 total, 8 up, 8 in 2024-06-11T13:28:18.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:18 smithi057 ceph-mon[29805]: from='client.34391 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-06-11T13:28:18.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:18 smithi057 ceph-mon[29805]: from='client.24379 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-06-11T13:28:18.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:18 smithi057 ceph-mon[29805]: osdmap e65: 8 total, 8 up, 8 in 2024-06-11T13:28:18.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:18 smithi057 ceph-mon[40588]: from='client.34391 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-06-11T13:28:18.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:18 smithi057 ceph-mon[40588]: from='client.24379 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-06-11T13:28:18.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:18 smithi057 ceph-mon[40588]: osdmap e65: 8 total, 8 up, 8 in 2024-06-11T13:28:19.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:19 smithi057 ceph-mon[29805]: osdmap e66: 8 total, 8 up, 8 in 2024-06-11T13:28:19.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:19 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2628526915' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-06-11T13:28:19.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:19 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3167383653' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-06-11T13:28:19.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:19 smithi057 ceph-mon[29805]: from='client.34391 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-06-11T13:28:19.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:19 smithi057 ceph-mon[29805]: from='client.24379 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-06-11T13:28:19.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:19 smithi057 ceph-mon[29805]: pgmap v192: 132 pgs: 52 active+clean, 3 creating+peering, 77 unknown; 578 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-06-11T13:28:19.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:19 smithi057 ceph-mon[29805]: from='client.34391 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-06-11T13:28:19.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:19 smithi057 ceph-mon[29805]: from='client.24379 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-06-11T13:28:19.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:19 smithi057 ceph-mon[29805]: osdmap e67: 8 total, 8 up, 8 in 2024-06-11T13:28:19.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:19 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3167383653' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-06-11T13:28:19.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:19 smithi057 ceph-mon[29805]: from='client.34391 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-06-11T13:28:19.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:19 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2628526915' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-06-11T13:28:19.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:19 smithi057 ceph-mon[29805]: from='client.24379 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-06-11T13:28:19.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:19 smithi057 ceph-mon[40588]: osdmap e66: 8 total, 8 up, 8 in 2024-06-11T13:28:19.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:19 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2628526915' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-06-11T13:28:19.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:19 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3167383653' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-06-11T13:28:19.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:19 smithi057 ceph-mon[40588]: from='client.34391 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-06-11T13:28:19.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:19 smithi057 ceph-mon[40588]: from='client.24379 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-06-11T13:28:19.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:19 smithi057 ceph-mon[40588]: pgmap v192: 132 pgs: 52 active+clean, 3 creating+peering, 77 unknown; 578 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-06-11T13:28:19.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:19 smithi057 ceph-mon[40588]: from='client.34391 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-06-11T13:28:19.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:19 smithi057 ceph-mon[40588]: from='client.24379 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-06-11T13:28:19.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:19 smithi057 ceph-mon[40588]: osdmap e67: 8 total, 8 up, 8 in 2024-06-11T13:28:19.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:19 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3167383653' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-06-11T13:28:19.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:19 smithi057 ceph-mon[40588]: from='client.34391 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-06-11T13:28:19.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:19 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2628526915' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-06-11T13:28:19.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:19 smithi057 ceph-mon[40588]: from='client.24379 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-06-11T13:28:19.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:19 smithi107 ceph-mon[33178]: osdmap e66: 8 total, 8 up, 8 in 2024-06-11T13:28:19.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:19 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2628526915' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-06-11T13:28:19.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:19 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3167383653' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-06-11T13:28:19.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:19 smithi107 ceph-mon[33178]: from='client.34391 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-06-11T13:28:19.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:19 smithi107 ceph-mon[33178]: from='client.24379 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-06-11T13:28:19.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:19 smithi107 ceph-mon[33178]: pgmap v192: 132 pgs: 52 active+clean, 3 creating+peering, 77 unknown; 578 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-06-11T13:28:19.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:19 smithi107 ceph-mon[33178]: from='client.34391 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-06-11T13:28:19.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:19 smithi107 ceph-mon[33178]: from='client.24379 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-06-11T13:28:19.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:19 smithi107 ceph-mon[33178]: osdmap e67: 8 total, 8 up, 8 in 2024-06-11T13:28:19.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:19 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3167383653' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-06-11T13:28:19.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:19 smithi107 ceph-mon[33178]: from='client.34391 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-06-11T13:28:19.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:19 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2628526915' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-06-11T13:28:19.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:19 smithi107 ceph-mon[33178]: from='client.24379 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-06-11T13:28:19.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:19.889+0000 7f94fd47b640 1 -- 172.21.15.107:0/4082124840 >> v1:172.21.15.107:6789/0 conn(0x7f94f8105e80 legacy=0x7f94f8108270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:19.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:19.889+0000 7f94fd47b640 1 -- 172.21.15.107:0/4082124840 shutdown_connections 2024-06-11T13:28:19.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:19.889+0000 7f94fd47b640 1 -- 172.21.15.107:0/4082124840 >> 172.21.15.107:0/4082124840 conn(0x7f94f80fbd40 msgr2=0x7f94f80fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:28:19.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:19.889+0000 7f94fd47b640 1 -- 172.21.15.107:0/4082124840 shutdown_connections 2024-06-11T13:28:19.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:19.889+0000 7f94fd47b640 1 -- 172.21.15.107:0/4082124840 wait complete. 2024-06-11T13:28:19.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:19.889+0000 7f94fd47b640 1 Processor -- start 2024-06-11T13:28:19.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:19.890+0000 7f94fd47b640 1 -- start start 2024-06-11T13:28:19.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:19.890+0000 7f94fd47b640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f94f81978c0 con 0x7f94f8102370 2024-06-11T13:28:19.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:19.890+0000 7f94fd47b640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f94f81a3110 con 0x7f94f8105e80 2024-06-11T13:28:19.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:19.890+0000 7f94fd47b640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f94f81a42f0 con 0x7f94f8109990 2024-06-11T13:28:19.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:19.890+0000 7f94f67fc640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f94f8105e80 0x7f94f819d2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.107:36944/0 (socket says 172.21.15.107:36944) 2024-06-11T13:28:19.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:19.890+0000 7f94f67fc640 1 -- 172.21.15.107:0/2985161081 learned_addr learned my addr 172.21.15.107:0/2985161081 (peer_addr_for_me v1:172.21.15.107:0/0) 2024-06-11T13:28:19.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:19.890+0000 7f94dffff640 1 -- 172.21.15.107:0/2985161081 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3478142027 0 0) 0x7f94f81a3110 con 0x7f94f8105e80 2024-06-11T13:28:19.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:19.890+0000 7f94dffff640 1 -- 172.21.15.107:0/2985161081 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94e0003520 con 0x7f94f8105e80 2024-06-11T13:28:19.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:19.890+0000 7f94dffff640 1 -- 172.21.15.107:0/2985161081 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 323452769 0 0) 0x7f94f81a42f0 con 0x7f94f8109990 2024-06-11T13:28:19.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:19.890+0000 7f94dffff640 1 -- 172.21.15.107:0/2985161081 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94f81a3110 con 0x7f94f8109990 2024-06-11T13:28:19.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:19.890+0000 7f94dffff640 1 -- 172.21.15.107:0/2985161081 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1401113035 0 0) 0x7f94f81978c0 con 0x7f94f8102370 2024-06-11T13:28:19.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:19.890+0000 7f94dffff640 1 -- 172.21.15.107:0/2985161081 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94f81a42f0 con 0x7f94f8102370 2024-06-11T13:28:19.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:19.890+0000 7f94dffff640 1 -- 172.21.15.107:0/2985161081 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 896264963 0 0) 0x7f94e0003520 con 0x7f94f8105e80 2024-06-11T13:28:19.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:19.890+0000 7f94dffff640 1 -- 172.21.15.107:0/2985161081 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f94f81978c0 con 0x7f94f8105e80 2024-06-11T13:28:19.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:19.891+0000 7f94dffff640 1 -- 172.21.15.107:0/2985161081 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f94ec001430 con 0x7f94f8105e80 2024-06-11T13:28:19.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:19.891+0000 7f94dffff640 1 -- 172.21.15.107:0/2985161081 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2258730173 0 0) 0x7f94f81a3110 con 0x7f94f8109990 2024-06-11T13:28:19.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:19.891+0000 7f94dffff640 1 -- 172.21.15.107:0/2985161081 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f94e0003520 con 0x7f94f8109990 2024-06-11T13:28:19.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:19.891+0000 7f94dffff640 1 -- 172.21.15.107:0/2985161081 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f94f0001c30 con 0x7f94f8109990 2024-06-11T13:28:19.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:19.891+0000 7f94dffff640 1 -- 172.21.15.107:0/2985161081 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 316794465 0 0) 0x7f94f81a42f0 con 0x7f94f8102370 2024-06-11T13:28:19.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:19.891+0000 7f94dffff640 1 -- 172.21.15.107:0/2985161081 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f94f81a3110 con 0x7f94f8102370 2024-06-11T13:28:19.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:19.891+0000 7f94dffff640 1 -- 172.21.15.107:0/2985161081 <== mon.1 v1:172.21.15.107:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1417966459 0 0) 0x7f94f81978c0 con 0x7f94f8105e80 2024-06-11T13:28:19.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:19.891+0000 7f94dffff640 1 -- 172.21.15.107:0/2985161081 >> v1:172.21.15.57:6790/0 conn(0x7f94f8109990 legacy=0x7f94f81a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:19.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:19.891+0000 7f94dffff640 1 -- 172.21.15.107:0/2985161081 >> v1:172.21.15.57:6789/0 conn(0x7f94f8102370 legacy=0x7f94f8196dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:19.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:19.891+0000 7f94dffff640 1 -- 172.21.15.107:0/2985161081 --> v1:172.21.15.107:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94f81a54d0 con 0x7f94f8105e80 2024-06-11T13:28:19.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:19.891+0000 7f94fd47b640 1 -- 172.21.15.107:0/2985161081 --> v1:172.21.15.107:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f94f81a2160 con 0x7f94f8105e80 2024-06-11T13:28:19.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:19.891+0000 7f94fd47b640 1 -- 172.21.15.107:0/2985161081 --> v1:172.21.15.107:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f94f81a26a0 con 0x7f94f8105e80 2024-06-11T13:28:19.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:19.891+0000 7f94dffff640 1 -- 172.21.15.107:0/2985161081 <== mon.1 v1:172.21.15.107:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f94ec003690 con 0x7f94f8105e80 2024-06-11T13:28:19.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:19.891+0000 7f94dffff640 1 -- 172.21.15.107:0/2985161081 <== mon.1 v1:172.21.15.107:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f94ec004f50 con 0x7f94f8105e80 2024-06-11T13:28:19.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:19.892+0000 7f94fd47b640 1 -- 172.21.15.107:0/2985161081 --> v1:172.21.15.107:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94e40050b0 con 0x7f94f8105e80 2024-06-11T13:28:19.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:19.892+0000 7f94dffff640 1 -- 172.21.15.107:0/2985161081 <== mon.1 v1:172.21.15.107:6789/0 7 ==== mgrmap(e 15) ==== 92081+0+0 (unknown 2221930365 0 0) 0x7f94ec003a20 con 0x7f94f8105e80 2024-06-11T13:28:19.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:19.893+0000 7f94dffff640 1 -- 172.21.15.107:0/2985161081 <== mon.1 v1:172.21.15.107:6789/0 8 ==== osd_map(67..67 src has 1..67) ==== 5926+0+0 (unknown 3229728417 0 0) 0x7f94ec0894d0 con 0x7f94f8105e80 2024-06-11T13:28:19.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:19.897+0000 7f94dffff640 1 -- 172.21.15.107:0/2985161081 <== mon.1 v1:172.21.15.107:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f94ec056d80 con 0x7f94f8105e80 2024-06-11T13:28:20.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:20.082+0000 7f94fd47b640 1 -- 172.21.15.107:0/2985161081 --> v1:172.21.15.57:6800/2989665238 -- mgr_command(tid 0: {"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.107", "placement": "1;smithi107=iscsi.a", "target": ["mon-mgr", ""]}) -- 0x7f94e4000c90 con 0x7f94e006ede0 2024-06-11T13:28:20.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:20.094+0000 7f94dffff640 1 -- 172.21.15.107:0/2985161081 <== mgr.14150 v1:172.21.15.57:6800/2989665238 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+35 (unknown 0 0 803663096) 0x7f94e4000c90 con 0x7f94e006ede0 2024-06-11T13:28:20.090 INFO:teuthology.orchestra.run.smithi107.stdout:Scheduled iscsi.datapool update... 2024-06-11T13:28:20.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:20.096+0000 7f94fd47b640 1 -- 172.21.15.107:0/2985161081 >> v1:172.21.15.57:6800/2989665238 conn(0x7f94e006ede0 legacy=0x7f94e00712a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:20.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:20.096+0000 7f94fd47b640 1 -- 172.21.15.107:0/2985161081 >> v1:172.21.15.107:6789/0 conn(0x7f94f8105e80 legacy=0x7f94f819d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:20.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:20.096+0000 7f94fd47b640 1 -- 172.21.15.107:0/2985161081 shutdown_connections 2024-06-11T13:28:20.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:20.096+0000 7f94fd47b640 1 -- 172.21.15.107:0/2985161081 >> 172.21.15.107:0/2985161081 conn(0x7f94f80fbd40 msgr2=0x7f94f80fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:28:20.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:20.096+0000 7f94fd47b640 1 -- 172.21.15.107:0/2985161081 shutdown_connections 2024-06-11T13:28:20.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:20.096+0000 7f94fd47b640 1 -- 172.21.15.107:0/2985161081 wait complete. 2024-06-11T13:28:20.697 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-06-11T13:28:20.697 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-06-11T13:28:20.698 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-06-11T13:28:20.731 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-06-11T13:28:20.731 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-06-11T13:28:20.759 DEBUG:teuthology.orchestra.run.smithi107:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@iscsi.iscsi.a.service 2024-06-11T13:28:20.783 INFO:journalctl@ceph.rgw.foo.a.smithi057.stdout:Jun 11 13:28:20 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-rgw-foo-a[67027]: 2024-06-11T13:28:20.516+0000 7fca2ea69880 -1 LDAP not started since no server URIs were provided in the configuration. 2024-06-11T13:28:20.803 INFO:tasks.cephadm:Adding prometheus.a on smithi107 2024-06-11T13:28:20.803 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph orch apply prometheus '1;smithi107=a' 2024-06-11T13:28:21.388 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:21 smithi107 ceph-mon[33178]: from='client.24418 v1:172.21.15.107:0/2985161081' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.107", "placement": "1;smithi107=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:28:21.388 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:21 smithi107 ceph-mon[33178]: Saving service iscsi.datapool spec with placement smithi107=iscsi.a;count:1 2024-06-11T13:28:21.388 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:21 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:21.388 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:21 smithi107 ceph-mon[33178]: from='client.34391 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-06-11T13:28:21.389 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:21 smithi107 ceph-mon[33178]: from='client.24379 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-06-11T13:28:21.389 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:21 smithi107 ceph-mon[33178]: osdmap e68: 8 total, 8 up, 8 in 2024-06-11T13:28:21.389 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:21 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:21.389 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:21 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:21.389 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:21 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:21.389 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:21 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-06-11T13:28:21.389 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:21 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-06-11T13:28:21.389 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:21 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:28:21.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:21 smithi057 ceph-mon[29805]: from='client.24418 v1:172.21.15.107:0/2985161081' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.107", "placement": "1;smithi107=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:28:21.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:21 smithi057 ceph-mon[29805]: Saving service iscsi.datapool spec with placement smithi107=iscsi.a;count:1 2024-06-11T13:28:21.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:21 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:21.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:21 smithi057 ceph-mon[29805]: from='client.34391 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-06-11T13:28:21.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:21 smithi057 ceph-mon[29805]: from='client.24379 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-06-11T13:28:21.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:21 smithi057 ceph-mon[29805]: osdmap e68: 8 total, 8 up, 8 in 2024-06-11T13:28:21.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:21 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:21.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:21 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:21.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:21 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:21.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:21 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-06-11T13:28:21.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:21 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-06-11T13:28:21.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:21 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:28:21.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:21 smithi057 ceph-mon[40588]: from='client.24418 v1:172.21.15.107:0/2985161081' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.107", "placement": "1;smithi107=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:28:21.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:21 smithi057 ceph-mon[40588]: Saving service iscsi.datapool spec with placement smithi107=iscsi.a;count:1 2024-06-11T13:28:21.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:21 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:21.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:21 smithi057 ceph-mon[40588]: from='client.34391 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-06-11T13:28:21.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:21 smithi057 ceph-mon[40588]: from='client.24379 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-06-11T13:28:21.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:21 smithi057 ceph-mon[40588]: osdmap e68: 8 total, 8 up, 8 in 2024-06-11T13:28:21.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:21 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:21.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:21 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:21.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:21 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:21.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:21 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-06-11T13:28:21.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:21 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-06-11T13:28:21.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:21 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:28:21.720 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.b/config 2024-06-11T13:28:22.434 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:22 smithi107 ceph-mon[33178]: pgmap v195: 132 pgs: 93 active+clean, 24 creating+peering, 15 unknown; 578 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 767 B/s wr, 1 op/s 2024-06-11T13:28:22.434 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:22 smithi107 ceph-mon[33178]: Deploying daemon iscsi.iscsi.a on smithi107 2024-06-11T13:28:22.434 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:22 smithi107 ceph-mon[33178]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-11T13:28:22.434 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:22 smithi107 ceph-mon[33178]: Cluster is now healthy 2024-06-11T13:28:22.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:22 smithi057 ceph-mon[29805]: pgmap v195: 132 pgs: 93 active+clean, 24 creating+peering, 15 unknown; 578 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 767 B/s wr, 1 op/s 2024-06-11T13:28:22.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:22 smithi057 ceph-mon[29805]: Deploying daemon iscsi.iscsi.a on smithi107 2024-06-11T13:28:22.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:22 smithi057 ceph-mon[29805]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-11T13:28:22.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:22 smithi057 ceph-mon[29805]: Cluster is now healthy 2024-06-11T13:28:22.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:22 smithi057 ceph-mon[40588]: pgmap v195: 132 pgs: 93 active+clean, 24 creating+peering, 15 unknown; 578 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 767 B/s wr, 1 op/s 2024-06-11T13:28:22.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:22 smithi057 ceph-mon[40588]: Deploying daemon iscsi.iscsi.a on smithi107 2024-06-11T13:28:22.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:22 smithi057 ceph-mon[40588]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-11T13:28:22.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:22 smithi057 ceph-mon[40588]: Cluster is now healthy 2024-06-11T13:28:23.164 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:28:22 smithi107 systemd[1]: Starting Ceph iscsi.iscsi.a for 98a6635e-27f5-11ef-bc9b-c7b262605968... 2024-06-11T13:28:23.478 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:23 smithi107 ceph-mon[33178]: osdmap e69: 8 total, 8 up, 8 in 2024-06-11T13:28:23.478 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:23 smithi107 ceph-mon[33178]: pgmap v197: 132 pgs: 105 active+clean, 21 creating+peering, 6 unknown; 579 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 3.5 KiB/s wr, 61 op/s 2024-06-11T13:28:23.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:23.681+0000 7fe38196d640 1 -- 172.21.15.107:0/3614929553 >> v1:172.21.15.107:6789/0 conn(0x7fe37c108910 legacy=0x7fe37c10add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:23.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:23.681+0000 7fe38196d640 1 -- 172.21.15.107:0/3614929553 shutdown_connections 2024-06-11T13:28:23.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:23.681+0000 7fe38196d640 1 -- 172.21.15.107:0/3614929553 >> 172.21.15.107:0/3614929553 conn(0x7fe37c0fbd40 msgr2=0x7fe37c0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:28:23.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:23.681+0000 7fe38196d640 1 -- 172.21.15.107:0/3614929553 shutdown_connections 2024-06-11T13:28:23.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:23.681+0000 7fe38196d640 1 -- 172.21.15.107:0/3614929553 wait complete. 2024-06-11T13:28:23.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:23.682+0000 7fe38196d640 1 Processor -- start 2024-06-11T13:28:23.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:23.682+0000 7fe38196d640 1 -- start start 2024-06-11T13:28:23.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:23.682+0000 7fe38196d640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fe37c1979e0 con 0x7fe37c1007a0 2024-06-11T13:28:23.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:23.682+0000 7fe38196d640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fe37c1a3230 con 0x7fe37c108910 2024-06-11T13:28:23.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:23.682+0000 7fe38196d640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fe37c1a4410 con 0x7fe37c104bd0 2024-06-11T13:28:23.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:23.682+0000 7fe37b7fe640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7fe37c108910 0x7fe37c1a0b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.107:48974/0 (socket says 172.21.15.107:48974) 2024-06-11T13:28:23.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:23.682+0000 7fe37b7fe640 1 -- 172.21.15.107:0/2781536574 learned_addr learned my addr 172.21.15.107:0/2781536574 (peer_addr_for_me v1:172.21.15.107:0/0) 2024-06-11T13:28:23.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:23.682+0000 7fe38096b640 1 -- 172.21.15.107:0/2781536574 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 136691129 0 0) 0x7fe37c1a3230 con 0x7fe37c108910 2024-06-11T13:28:23.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:23.682+0000 7fe38096b640 1 -- 172.21.15.107:0/2781536574 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe368003520 con 0x7fe37c108910 2024-06-11T13:28:23.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:23.683+0000 7fe38096b640 1 -- 172.21.15.107:0/2781536574 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3703131403 0 0) 0x7fe37c1a4410 con 0x7fe37c104bd0 2024-06-11T13:28:23.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:23.683+0000 7fe38096b640 1 -- 172.21.15.107:0/2781536574 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe37c1a3230 con 0x7fe37c104bd0 2024-06-11T13:28:23.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:23.683+0000 7fe38096b640 1 -- 172.21.15.107:0/2781536574 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 730210784 0 0) 0x7fe37c1979e0 con 0x7fe37c1007a0 2024-06-11T13:28:23.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:23.683+0000 7fe38096b640 1 -- 172.21.15.107:0/2781536574 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe37c1a4410 con 0x7fe37c1007a0 2024-06-11T13:28:23.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:23.683+0000 7fe38096b640 1 -- 172.21.15.107:0/2781536574 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 390786188 0 0) 0x7fe368003520 con 0x7fe37c108910 2024-06-11T13:28:23.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:23.683+0000 7fe38096b640 1 -- 172.21.15.107:0/2781536574 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe37c1979e0 con 0x7fe37c108910 2024-06-11T13:28:23.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:23.683+0000 7fe38096b640 1 -- 172.21.15.107:0/2781536574 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fe3740017b0 con 0x7fe37c108910 2024-06-11T13:28:23.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:23.683+0000 7fe38096b640 1 -- 172.21.15.107:0/2781536574 <== mon.1 v1:172.21.15.107:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1251865629 0 0) 0x7fe37c1979e0 con 0x7fe37c108910 2024-06-11T13:28:23.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:23.683+0000 7fe38096b640 1 -- 172.21.15.107:0/2781536574 >> v1:172.21.15.57:6790/0 conn(0x7fe37c104bd0 legacy=0x7fe37c19d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:23.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:23.683+0000 7fe38096b640 1 -- 172.21.15.107:0/2781536574 >> v1:172.21.15.57:6789/0 conn(0x7fe37c1007a0 legacy=0x7fe37c196ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:23.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:23.683+0000 7fe38096b640 1 -- 172.21.15.107:0/2781536574 --> v1:172.21.15.107:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe37c1a55f0 con 0x7fe37c108910 2024-06-11T13:28:23.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:23.683+0000 7fe38196d640 1 -- 172.21.15.107:0/2781536574 --> v1:172.21.15.107:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fe37c1a4640 con 0x7fe37c108910 2024-06-11T13:28:23.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:23.683+0000 7fe38196d640 1 -- 172.21.15.107:0/2781536574 --> v1:172.21.15.107:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fe37c1a4b80 con 0x7fe37c108910 2024-06-11T13:28:23.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:23.683+0000 7fe38096b640 1 -- 172.21.15.107:0/2781536574 <== mon.1 v1:172.21.15.107:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7fe374001f70 con 0x7fe37c108910 2024-06-11T13:28:23.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:23.683+0000 7fe38096b640 1 -- 172.21.15.107:0/2781536574 <== mon.1 v1:172.21.15.107:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fe3740050a0 con 0x7fe37c108910 2024-06-11T13:28:23.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:23.684+0000 7fe38196d640 1 -- 172.21.15.107:0/2781536574 --> v1:172.21.15.107:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe35c0050b0 con 0x7fe37c108910 2024-06-11T13:28:23.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:23.684+0000 7fe38096b640 1 -- 172.21.15.107:0/2781536574 <== mon.1 v1:172.21.15.107:6789/0 7 ==== mgrmap(e 15) ==== 92081+0+0 (unknown 2221930365 0 0) 0x7fe374003da0 con 0x7fe37c108910 2024-06-11T13:28:23.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:23.685+0000 7fe38096b640 1 -- 172.21.15.107:0/2781536574 <== mon.1 v1:172.21.15.107:6789/0 8 ==== osd_map(69..69 src has 1..69) ==== 5942+0+0 (unknown 3863100797 0 0) 0x7fe374089840 con 0x7fe37c108910 2024-06-11T13:28:23.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:23.689+0000 7fe38096b640 1 -- 172.21.15.107:0/2781536574 <== mon.1 v1:172.21.15.107:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7fe3740571c0 con 0x7fe37c108910 2024-06-11T13:28:23.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:23 smithi057 ceph-mon[29805]: osdmap e69: 8 total, 8 up, 8 in 2024-06-11T13:28:23.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:23 smithi057 ceph-mon[29805]: pgmap v197: 132 pgs: 105 active+clean, 21 creating+peering, 6 unknown; 579 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 3.5 KiB/s wr, 61 op/s 2024-06-11T13:28:23.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:23 smithi057 ceph-mon[40588]: osdmap e69: 8 total, 8 up, 8 in 2024-06-11T13:28:23.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:23 smithi057 ceph-mon[40588]: pgmap v197: 132 pgs: 105 active+clean, 21 creating+peering, 6 unknown; 579 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 3.5 KiB/s wr, 61 op/s 2024-06-11T13:28:23.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:23.884+0000 7fe38196d640 1 -- 172.21.15.107:0/2781536574 --> v1:172.21.15.57:6800/2989665238 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi107=a", "target": ["mon-mgr", ""]}) -- 0x7fe35c000bc0 con 0x7fe36806e9d0 2024-06-11T13:28:23.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:23.894+0000 7fe38096b640 1 -- 172.21.15.107:0/2781536574 <== mgr.14150 v1:172.21.15.57:6800/2989665238 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+31 (unknown 0 0 1342662408) 0x7fe35c000bc0 con 0x7fe36806e9d0 2024-06-11T13:28:23.890 INFO:teuthology.orchestra.run.smithi107.stdout:Scheduled prometheus update... 2024-06-11T13:28:23.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:23.896+0000 7fe38196d640 1 -- 172.21.15.107:0/2781536574 >> v1:172.21.15.57:6800/2989665238 conn(0x7fe36806e9d0 legacy=0x7fe368070e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:23.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:23.896+0000 7fe38196d640 1 -- 172.21.15.107:0/2781536574 >> v1:172.21.15.107:6789/0 conn(0x7fe37c108910 legacy=0x7fe37c1a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:23.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:23.896+0000 7fe38196d640 1 -- 172.21.15.107:0/2781536574 shutdown_connections 2024-06-11T13:28:23.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:23.896+0000 7fe38196d640 1 -- 172.21.15.107:0/2781536574 >> 172.21.15.107:0/2781536574 conn(0x7fe37c0fbd40 msgr2=0x7fe37c108140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:28:23.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:23.896+0000 7fe38196d640 1 -- 172.21.15.107:0/2781536574 shutdown_connections 2024-06-11T13:28:23.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:23.896+0000 7fe38196d640 1 -- 172.21.15.107:0/2781536574 wait complete. 2024-06-11T13:28:24.243 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:28:23 smithi107 podman[59806]: 2024-06-11 13:28:23.817438036 +0000 UTC m=+0.017557404 image pull 2d7459ec65a098c35c774154617f2893b0ad051f56ce89b72ab795f65a97fc22 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310 2024-06-11T13:28:24.243 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:28:23 smithi107 podman[59806]: 2024-06-11 13:28:23.924653177 +0000 UTC m=+0.124772556 container create 1be9b8255d66c7b9dd03b9b828b4fb59e3edc5afde3c8d935bbd35211d32ef5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-3f51ea3, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.35.3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-06-11T13:28:24.665 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:28:24 smithi107 podman[59806]: 2024-06-11 13:28:24.362286137 +0000 UTC m=+0.562405496 container init 1be9b8255d66c7b9dd03b9b828b4fb59e3edc5afde3c8d935bbd35211d32ef5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=squid-3f51ea3, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.35.3, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-11T13:28:24.665 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:28:24 smithi107 podman[59806]: 2024-06-11 13:28:24.365579194 +0000 UTC m=+0.565698561 container start 1be9b8255d66c7b9dd03b9b828b4fb59e3edc5afde3c8d935bbd35211d32ef5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, io.buildah.version=1.35.3, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-3f51ea3, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-06-11T13:28:24.665 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:28:24 smithi107 bash[59806]: 1be9b8255d66c7b9dd03b9b828b4fb59e3edc5afde3c8d935bbd35211d32ef5f 2024-06-11T13:28:24.665 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:28:24 smithi107 systemd[1]: Started Ceph iscsi.iscsi.a for 98a6635e-27f5-11ef-bc9b-c7b262605968. 2024-06-11T13:28:24.889 DEBUG:teuthology.orchestra.run.smithi107:prometheus.a> sudo journalctl -f -n 0 -u ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@prometheus.a.service 2024-06-11T13:28:24.891 INFO:tasks.cephadm:Adding node-exporter.a on smithi057 2024-06-11T13:28:24.891 INFO:tasks.cephadm:Adding node-exporter.b on smithi107 2024-06-11T13:28:24.891 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph orch apply node-exporter '2;smithi057=a;smithi107=b' 2024-06-11T13:28:24.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:24 smithi107 ceph-mon[33178]: from='client.24439 v1:172.21.15.107:0/2781536574' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi107=a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:28:24.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:24 smithi107 ceph-mon[33178]: Saving service prometheus spec with placement smithi107=a;count:1 2024-06-11T13:28:24.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:24 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:24.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:24 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:24.917 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:24 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:24.918 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:24 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:24.918 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:24 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:24.918 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:24 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:28:24.918 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:28:24 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug Started the configuration object watcher 2024-06-11T13:28:24.918 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:28:24 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug Checking for config object changes every 1s 2024-06-11T13:28:24.918 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:28:24 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug Processing osd blocklist entries for this node 2024-06-11T13:28:25.196 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:28:25 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug Reading the configuration object to update local LIO configuration 2024-06-11T13:28:25.197 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:28:25 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug Configuration does not have an entry for this host(smithi107.front.sepia.ceph.com) - nothing to define to LIO 2024-06-11T13:28:25.197 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:28:25 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: * Serving Flask app 'rbd-target-api' (lazy loading) 2024-06-11T13:28:25.197 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:28:25 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: * Environment: production 2024-06-11T13:28:25.197 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:28:25 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: WARNING: This is a development server. Do not use it in a production deployment. 2024-06-11T13:28:25.197 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:28:25 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: Use a production WSGI server instead. 2024-06-11T13:28:25.197 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:28:25 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: * Debug mode: off 2024-06-11T13:28:25.197 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:28:25 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug * Running on all addresses. 2024-06-11T13:28:25.197 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:28:25 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: WARNING: This is a development server. Do not use it in a production deployment. 2024-06-11T13:28:25.197 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:28:25 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: * Running on all addresses. 2024-06-11T13:28:25.197 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:28:25 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: WARNING: This is a development server. Do not use it in a production deployment. 2024-06-11T13:28:25.197 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:28:25 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug * Running on http://[fe80::ec4:7aff:fe8f:47eb]:5000/ (Press CTRL+C to quit) 2024-06-11T13:28:25.197 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:28:25 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: * Running on http://[fe80::ec4:7aff:fe8f:47eb]:5000/ (Press CTRL+C to quit) 2024-06-11T13:28:25.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:24 smithi057 ceph-mon[40588]: from='client.24439 v1:172.21.15.107:0/2781536574' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi107=a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:28:25.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:24 smithi057 ceph-mon[40588]: Saving service prometheus spec with placement smithi107=a;count:1 2024-06-11T13:28:25.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:24 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:25.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:24 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:25.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:24 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:25.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:24 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:25.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:24 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:25.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:24 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:28:25.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:24 smithi057 ceph-mon[29805]: from='client.24439 v1:172.21.15.107:0/2781536574' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi107=a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:28:25.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:24 smithi057 ceph-mon[29805]: Saving service prometheus spec with placement smithi107=a;count:1 2024-06-11T13:28:25.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:24 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:25.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:24 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:25.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:24 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:25.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:24 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:25.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:24 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:25.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:24 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:28:25.441 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.b/config 2024-06-11T13:28:26.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:25 smithi057 ceph-mon[29805]: pgmap v198: 132 pgs: 132 active+clean; 582 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 65 KiB/s rd, 6.3 KiB/s wr, 160 op/s 2024-06-11T13:28:26.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:25 smithi057 ceph-mon[29805]: Checking pool "datapool" exists for service iscsi.datapool 2024-06-11T13:28:26.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:25 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.107:0/1672139585' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-11T13:28:26.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:25 smithi057 ceph-mon[40588]: pgmap v198: 132 pgs: 132 active+clean; 582 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 65 KiB/s rd, 6.3 KiB/s wr, 160 op/s 2024-06-11T13:28:26.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:25 smithi057 ceph-mon[40588]: Checking pool "datapool" exists for service iscsi.datapool 2024-06-11T13:28:26.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:25 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.107:0/1672139585' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-11T13:28:26.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:25 smithi107 ceph-mon[33178]: pgmap v198: 132 pgs: 132 active+clean; 582 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 65 KiB/s rd, 6.3 KiB/s wr, 160 op/s 2024-06-11T13:28:26.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:25 smithi107 ceph-mon[33178]: Checking pool "datapool" exists for service iscsi.datapool 2024-06-11T13:28:26.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:25 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.107:0/1672139585' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-11T13:28:27.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:27.691+0000 7f19ecdad640 1 -- 172.21.15.107:0/829395442 >> v1:172.21.15.107:6789/0 conn(0x7f19e8104ba0 legacy=0x7f19e8106ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:27.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:27.691+0000 7f19ecdad640 1 -- 172.21.15.107:0/829395442 shutdown_connections 2024-06-11T13:28:27.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:27.691+0000 7f19ecdad640 1 -- 172.21.15.107:0/829395442 >> 172.21.15.107:0/829395442 conn(0x7f19e80fbd10 msgr2=0x7f19e80fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:28:27.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:27.691+0000 7f19ecdad640 1 -- 172.21.15.107:0/829395442 shutdown_connections 2024-06-11T13:28:27.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:27.691+0000 7f19ecdad640 1 -- 172.21.15.107:0/829395442 wait complete. 2024-06-11T13:28:27.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:27.692+0000 7f19ecdad640 1 Processor -- start 2024-06-11T13:28:27.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:27.692+0000 7f19ecdad640 1 -- start start 2024-06-11T13:28:27.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:27.692+0000 7f19ecdad640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f19e806a2b0 con 0x7f19e8104ba0 2024-06-11T13:28:27.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:27.692+0000 7f19ecdad640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f19e81a2d80 con 0x7f19e8100770 2024-06-11T13:28:27.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:27.692+0000 7f19ecdad640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f19e81a3fa0 con 0x7f19e81088e0 2024-06-11T13:28:27.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:27.692+0000 7f19e659c640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f19e8100770 0x7f19e81997f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.107:49058/0 (socket says 172.21.15.107:49058) 2024-06-11T13:28:27.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:27.692+0000 7f19e659c640 1 -- 172.21.15.107:0/3057794831 learned_addr learned my addr 172.21.15.107:0/3057794831 (peer_addr_for_me v1:172.21.15.107:0/0) 2024-06-11T13:28:27.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:27.692+0000 7f19d77fe640 1 -- 172.21.15.107:0/3057794831 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1444988751 0 0) 0x7f19e81a2d80 con 0x7f19e8100770 2024-06-11T13:28:27.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:27.692+0000 7f19d77fe640 1 -- 172.21.15.107:0/3057794831 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f19d0003520 con 0x7f19e8100770 2024-06-11T13:28:27.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:27.692+0000 7f19d77fe640 1 -- 172.21.15.107:0/3057794831 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1539821077 0 0) 0x7f19e81a3fa0 con 0x7f19e81088e0 2024-06-11T13:28:27.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:27.692+0000 7f19d77fe640 1 -- 172.21.15.107:0/3057794831 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f19e81a2d80 con 0x7f19e81088e0 2024-06-11T13:28:27.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:27.692+0000 7f19d77fe640 1 -- 172.21.15.107:0/3057794831 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1059737242 0 0) 0x7f19e806a2b0 con 0x7f19e8104ba0 2024-06-11T13:28:27.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:27.693+0000 7f19d77fe640 1 -- 172.21.15.107:0/3057794831 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f19e81a3fa0 con 0x7f19e8104ba0 2024-06-11T13:28:27.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:27.693+0000 7f19d77fe640 1 -- 172.21.15.107:0/3057794831 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4131801273 0 0) 0x7f19d0003520 con 0x7f19e8100770 2024-06-11T13:28:27.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:27.693+0000 7f19d77fe640 1 -- 172.21.15.107:0/3057794831 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f19e806a2b0 con 0x7f19e8100770 2024-06-11T13:28:27.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:27.693+0000 7f19d77fe640 1 -- 172.21.15.107:0/3057794831 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f19d8001980 con 0x7f19e8100770 2024-06-11T13:28:27.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:27.693+0000 7f19d77fe640 1 -- 172.21.15.107:0/3057794831 <== mon.1 v1:172.21.15.107:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1040848094 0 0) 0x7f19e806a2b0 con 0x7f19e8100770 2024-06-11T13:28:27.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:27.693+0000 7f19d77fe640 1 -- 172.21.15.107:0/3057794831 >> v1:172.21.15.57:6790/0 conn(0x7f19e81088e0 legacy=0x7f19e81a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:27.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:27.693+0000 7f19d77fe640 1 -- 172.21.15.107:0/3057794831 >> v1:172.21.15.57:6789/0 conn(0x7f19e8104ba0 legacy=0x7f19e819cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:27.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:27.693+0000 7f19d77fe640 1 -- 172.21.15.107:0/3057794831 --> v1:172.21.15.107:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e81a51e0 con 0x7f19e8100770 2024-06-11T13:28:27.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:27.693+0000 7f19ecdad640 1 -- 172.21.15.107:0/3057794831 --> v1:172.21.15.107:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f19e81a41d0 con 0x7f19e8100770 2024-06-11T13:28:27.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:27.693+0000 7f19ecdad640 1 -- 172.21.15.107:0/3057794831 --> v1:172.21.15.107:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f19e81a4710 con 0x7f19e8100770 2024-06-11T13:28:27.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:27.693+0000 7f19d77fe640 1 -- 172.21.15.107:0/3057794831 <== mon.1 v1:172.21.15.107:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f19d8001c80 con 0x7f19e8100770 2024-06-11T13:28:27.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:27.693+0000 7f19d77fe640 1 -- 172.21.15.107:0/3057794831 <== mon.1 v1:172.21.15.107:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f19d8005ca0 con 0x7f19e8100770 2024-06-11T13:28:27.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:27.694+0000 7f19ecdad640 1 -- 172.21.15.107:0/3057794831 --> v1:172.21.15.107:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19c80050b0 con 0x7f19e8100770 2024-06-11T13:28:27.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:27.694+0000 7f19d77fe640 1 -- 172.21.15.107:0/3057794831 <== mon.1 v1:172.21.15.107:6789/0 7 ==== mgrmap(e 16) ==== 92132+0+0 (unknown 2632097432 0 0) 0x7f19d801c630 con 0x7f19e8100770 2024-06-11T13:28:27.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:27.695+0000 7f19d77fe640 1 -- 172.21.15.107:0/3057794831 <== mon.1 v1:172.21.15.107:6789/0 8 ==== osd_map(69..69 src has 1..69) ==== 5942+0+0 (unknown 3863100797 0 0) 0x7f19d80895c0 con 0x7f19e8100770 2024-06-11T13:28:27.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:27.699+0000 7f19d77fe640 1 -- 172.21.15.107:0/3057794831 <== mon.1 v1:172.21.15.107:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f19d8056d80 con 0x7f19e8100770 2024-06-11T13:28:27.715 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:27 smithi107 ceph-mon[33178]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 66 KiB/s rd, 5.0 KiB/s wr, 159 op/s 2024-06-11T13:28:27.715 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:27 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:27.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:27 smithi057 ceph-mon[29805]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 66 KiB/s rd, 5.0 KiB/s wr, 159 op/s 2024-06-11T13:28:27.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:27 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:27.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:27 smithi057 ceph-mon[40588]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 66 KiB/s rd, 5.0 KiB/s wr, 159 op/s 2024-06-11T13:28:27.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:27 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:27.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:27.911+0000 7f19ecdad640 1 -- 172.21.15.107:0/3057794831 --> v1:172.21.15.57:6800/2989665238 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi057=a;smithi107=b", "target": ["mon-mgr", ""]}) -- 0x7f19c8000bc0 con 0x7f19d006ea70 2024-06-11T13:28:27.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:27.920+0000 7f19d77fe640 1 -- 172.21.15.107:0/3057794831 <== mgr.14150 v1:172.21.15.57:6800/2989665238 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+34 (unknown 0 0 240551134) 0x7f19c8000bc0 con 0x7f19d006ea70 2024-06-11T13:28:27.916 INFO:teuthology.orchestra.run.smithi107.stdout:Scheduled node-exporter update... 2024-06-11T13:28:27.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:27.922+0000 7f19ecdad640 1 -- 172.21.15.107:0/3057794831 >> v1:172.21.15.57:6800/2989665238 conn(0x7f19d006ea70 legacy=0x7f19d0070f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:27.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:27.922+0000 7f19ecdad640 1 -- 172.21.15.107:0/3057794831 >> v1:172.21.15.107:6789/0 conn(0x7f19e8100770 legacy=0x7f19e81997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:27.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:27.922+0000 7f19ecdad640 1 -- 172.21.15.107:0/3057794831 shutdown_connections 2024-06-11T13:28:27.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:27.922+0000 7f19ecdad640 1 -- 172.21.15.107:0/3057794831 >> 172.21.15.107:0/3057794831 conn(0x7f19e80fbd10 msgr2=0x7f19e810b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:28:27.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:27.922+0000 7f19ecdad640 1 -- 172.21.15.107:0/3057794831 shutdown_connections 2024-06-11T13:28:27.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:27.922+0000 7f19ecdad640 1 -- 172.21.15.107:0/3057794831 wait complete. 2024-06-11T13:28:28.727 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:28 smithi107 ceph-mon[33178]: mgrmap e16: y(active, since 4m), standbys: x 2024-06-11T13:28:28.742 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:28 smithi107 ceph-mon[33178]: from='client.24475 v1:172.21.15.107:0/3057794831' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi057=a;smithi107=b", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:28:28.742 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:28 smithi107 ceph-mon[33178]: Saving service node-exporter spec with placement smithi057=a;smithi107=b;count:2 2024-06-11T13:28:28.743 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:28 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:28.743 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:28 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:28.743 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:28 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:28.743 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:28 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:28:28.743 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:28 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:28:28.743 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:28 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:28.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:28 smithi057 ceph-mon[40588]: mgrmap e16: y(active, since 4m), standbys: x 2024-06-11T13:28:28.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:28 smithi057 ceph-mon[40588]: from='client.24475 v1:172.21.15.107:0/3057794831' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi057=a;smithi107=b", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:28:28.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:28 smithi057 ceph-mon[40588]: Saving service node-exporter spec with placement smithi057=a;smithi107=b;count:2 2024-06-11T13:28:28.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:28 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:28.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:28 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:28.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:28 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:28.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:28 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:28:28.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:28 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:28:28.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:28 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:28.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:28 smithi057 ceph-mon[29805]: mgrmap e16: y(active, since 4m), standbys: x 2024-06-11T13:28:28.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:28 smithi057 ceph-mon[29805]: from='client.24475 v1:172.21.15.107:0/3057794831' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi057=a;smithi107=b", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:28:28.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:28 smithi057 ceph-mon[29805]: Saving service node-exporter spec with placement smithi057=a;smithi107=b;count:2 2024-06-11T13:28:28.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:28 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:28.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:28 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:28.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:28 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:28.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:28 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:28:28.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:28 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:28:28.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:28 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:29.046 DEBUG:teuthology.orchestra.run.smithi057:node-exporter.a> sudo journalctl -f -n 0 -u ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@node-exporter.a.service 2024-06-11T13:28:29.049 DEBUG:teuthology.orchestra.run.smithi107:node-exporter.b> sudo journalctl -f -n 0 -u ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@node-exporter.b.service 2024-06-11T13:28:29.051 INFO:tasks.cephadm:Adding alertmanager.a on smithi057 2024-06-11T13:28:29.051 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph orch apply alertmanager '1;smithi057=a' 2024-06-11T13:28:29.344 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.b/config 2024-06-11T13:28:29.594 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:29 smithi107 ceph-mon[33178]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s rd, 4.4 KiB/s wr, 139 op/s 2024-06-11T13:28:29.594 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:29 smithi107 ceph-mon[33178]: Deploying daemon prometheus.a on smithi107 2024-06-11T13:28:29.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:29 smithi057 ceph-mon[29805]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s rd, 4.4 KiB/s wr, 139 op/s 2024-06-11T13:28:29.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:29 smithi057 ceph-mon[29805]: Deploying daemon prometheus.a on smithi107 2024-06-11T13:28:29.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:29 smithi057 ceph-mon[40588]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s rd, 4.4 KiB/s wr, 139 op/s 2024-06-11T13:28:29.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:29 smithi057 ceph-mon[40588]: Deploying daemon prometheus.a on smithi107 2024-06-11T13:28:31.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:31.093+0000 7fc9addf4640 1 -- 172.21.15.107:0/192534313 >> v1:172.21.15.107:6789/0 conn(0x7fc9a8100780 legacy=0x7fc9a8100ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:31.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:31.094+0000 7fc9addf4640 1 -- 172.21.15.107:0/192534313 shutdown_connections 2024-06-11T13:28:31.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:31.094+0000 7fc9addf4640 1 -- 172.21.15.107:0/192534313 >> 172.21.15.107:0/192534313 conn(0x7fc9a80fbd40 msgr2=0x7fc9a80fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:28:31.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:31.094+0000 7fc9addf4640 1 -- 172.21.15.107:0/192534313 shutdown_connections 2024-06-11T13:28:31.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:31.094+0000 7fc9addf4640 1 -- 172.21.15.107:0/192534313 wait complete. 2024-06-11T13:28:31.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:31.094+0000 7fc9addf4640 1 Processor -- start 2024-06-11T13:28:31.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:31.094+0000 7fc9addf4640 1 -- start start 2024-06-11T13:28:31.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:31.095+0000 7fc9addf4640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fc9a8195840 con 0x7fc9a8104bb0 2024-06-11T13:28:31.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:31.095+0000 7fc9addf4640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fc9a8195a50 con 0x7fc9a8100780 2024-06-11T13:28:31.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:31.095+0000 7fc9addf4640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fc9a81aa900 con 0x7fc9a81088f0 2024-06-11T13:28:31.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:31.095+0000 7fc9a77fe640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7fc9a8100780 0x7fc9a8194e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.107:59460/0 (socket says 172.21.15.107:59460) 2024-06-11T13:28:31.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:31.095+0000 7fc9a77fe640 1 -- 172.21.15.107:0/1782277782 learned_addr learned my addr 172.21.15.107:0/1782277782 (peer_addr_for_me v1:172.21.15.107:0/0) 2024-06-11T13:28:31.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:31.095+0000 7fc9a4ff9640 1 -- 172.21.15.107:0/1782277782 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1459329246 0 0) 0x7fc9a8195a50 con 0x7fc9a8100780 2024-06-11T13:28:31.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:31.095+0000 7fc9a4ff9640 1 -- 172.21.15.107:0/1782277782 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc994003520 con 0x7fc9a8100780 2024-06-11T13:28:31.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:31.095+0000 7fc9a4ff9640 1 -- 172.21.15.107:0/1782277782 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3453879201 0 0) 0x7fc9a8195840 con 0x7fc9a8104bb0 2024-06-11T13:28:31.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:31.095+0000 7fc9a4ff9640 1 -- 172.21.15.107:0/1782277782 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc9a8195a50 con 0x7fc9a8104bb0 2024-06-11T13:28:31.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:31.095+0000 7fc9a4ff9640 1 -- 172.21.15.107:0/1782277782 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3527004444 0 0) 0x7fc9a81aa900 con 0x7fc9a81088f0 2024-06-11T13:28:31.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:31.095+0000 7fc9a4ff9640 1 -- 172.21.15.107:0/1782277782 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc9a8195840 con 0x7fc9a81088f0 2024-06-11T13:28:31.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:31.095+0000 7fc9a4ff9640 1 -- 172.21.15.107:0/1782277782 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3410328431 0 0) 0x7fc994003520 con 0x7fc9a8100780 2024-06-11T13:28:31.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:31.095+0000 7fc9a4ff9640 1 -- 172.21.15.107:0/1782277782 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc9a81aa900 con 0x7fc9a8100780 2024-06-11T13:28:31.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:31.095+0000 7fc9a4ff9640 1 -- 172.21.15.107:0/1782277782 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fc998001430 con 0x7fc9a8100780 2024-06-11T13:28:31.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:31.095+0000 7fc9a4ff9640 1 -- 172.21.15.107:0/1782277782 <== mon.1 v1:172.21.15.107:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3881174633 0 0) 0x7fc9a81aa900 con 0x7fc9a8100780 2024-06-11T13:28:31.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:31.095+0000 7fc9a4ff9640 1 -- 172.21.15.107:0/1782277782 >> v1:172.21.15.57:6790/0 conn(0x7fc9a81088f0 legacy=0x7fc9a8198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:31.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:31.095+0000 7fc9a4ff9640 1 -- 172.21.15.107:0/1782277782 >> v1:172.21.15.57:6789/0 conn(0x7fc9a8104bb0 legacy=0x7fc9a8198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:31.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:31.095+0000 7fc9a4ff9640 1 -- 172.21.15.107:0/1782277782 --> v1:172.21.15.107:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9a81abb40 con 0x7fc9a8100780 2024-06-11T13:28:31.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:31.095+0000 7fc9addf4640 1 -- 172.21.15.107:0/1782277782 --> v1:172.21.15.107:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fc9a81aab30 con 0x7fc9a8100780 2024-06-11T13:28:31.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:31.095+0000 7fc9addf4640 1 -- 172.21.15.107:0/1782277782 --> v1:172.21.15.107:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fc9a81ab070 con 0x7fc9a8100780 2024-06-11T13:28:31.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:31.096+0000 7fc9a4ff9640 1 -- 172.21.15.107:0/1782277782 <== mon.1 v1:172.21.15.107:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7fc9980016e0 con 0x7fc9a8100780 2024-06-11T13:28:31.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:31.096+0000 7fc9a4ff9640 1 -- 172.21.15.107:0/1782277782 <== mon.1 v1:172.21.15.107:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fc998004ea0 con 0x7fc9a8100780 2024-06-11T13:28:31.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:31.096+0000 7fc9addf4640 1 -- 172.21.15.107:0/1782277782 --> v1:172.21.15.107:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9880050b0 con 0x7fc9a8100780 2024-06-11T13:28:31.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:31.097+0000 7fc9a4ff9640 1 -- 172.21.15.107:0/1782277782 <== mon.1 v1:172.21.15.107:6789/0 7 ==== mgrmap(e 16) ==== 92132+0+0 (unknown 2632097432 0 0) 0x7fc998003a20 con 0x7fc9a8100780 2024-06-11T13:28:31.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:31.097+0000 7fc9a4ff9640 1 -- 172.21.15.107:0/1782277782 <== mon.1 v1:172.21.15.107:6789/0 8 ==== osd_map(69..69 src has 1..69) ==== 5942+0+0 (unknown 3863100797 0 0) 0x7fc998089710 con 0x7fc9a8100780 2024-06-11T13:28:31.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:31.102+0000 7fc9a4ff9640 1 -- 172.21.15.107:0/1782277782 <== mon.1 v1:172.21.15.107:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7fc998056ed0 con 0x7fc9a8100780 2024-06-11T13:28:31.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:31.293+0000 7fc9addf4640 1 -- 172.21.15.107:0/1782277782 --> v1:172.21.15.57:6800/2989665238 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi057=a", "target": ["mon-mgr", ""]}) -- 0x7fc988000bc0 con 0x7fc99406e830 2024-06-11T13:28:31.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:31.310+0000 7fc9a4ff9640 1 -- 172.21.15.107:0/1782277782 <== mgr.14150 v1:172.21.15.57:6800/2989665238 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+33 (unknown 0 0 1850065467) 0x7fc988000bc0 con 0x7fc99406e830 2024-06-11T13:28:31.307 INFO:teuthology.orchestra.run.smithi107.stdout:Scheduled alertmanager update... 2024-06-11T13:28:31.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:31.313+0000 7fc9addf4640 1 -- 172.21.15.107:0/1782277782 >> v1:172.21.15.57:6800/2989665238 conn(0x7fc99406e830 legacy=0x7fc994070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:31.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:31.313+0000 7fc9addf4640 1 -- 172.21.15.107:0/1782277782 >> v1:172.21.15.107:6789/0 conn(0x7fc9a8100780 legacy=0x7fc9a8194e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:31.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:31.313+0000 7fc9addf4640 1 -- 172.21.15.107:0/1782277782 shutdown_connections 2024-06-11T13:28:31.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:31.313+0000 7fc9addf4640 1 -- 172.21.15.107:0/1782277782 >> 172.21.15.107:0/1782277782 conn(0x7fc9a80fbd40 msgr2=0x7fc9a8069e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:28:31.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:31.313+0000 7fc9addf4640 1 -- 172.21.15.107:0/1782277782 shutdown_connections 2024-06-11T13:28:31.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:31.313+0000 7fc9addf4640 1 -- 172.21.15.107:0/1782277782 wait complete. 2024-06-11T13:28:31.771 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:31 smithi107 ceph-mon[33178]: pgmap v201: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 47 KiB/s rd, 3.6 KiB/s wr, 113 op/s 2024-06-11T13:28:31.771 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:31 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:31.772 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:31 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:31.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:31 smithi057 ceph-mon[29805]: pgmap v201: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 47 KiB/s rd, 3.6 KiB/s wr, 113 op/s 2024-06-11T13:28:31.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:31 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:31.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:31 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:31.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:31 smithi057 ceph-mon[40588]: pgmap v201: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 47 KiB/s rd, 3.6 KiB/s wr, 113 op/s 2024-06-11T13:28:31.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:31 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:31.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:31 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:32.434 DEBUG:teuthology.orchestra.run.smithi057:alertmanager.a> sudo journalctl -f -n 0 -u ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@alertmanager.a.service 2024-06-11T13:28:32.437 INFO:tasks.cephadm:Adding grafana.a on smithi107 2024-06-11T13:28:32.437 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph orch apply grafana '1;smithi107=a' 2024-06-11T13:28:32.698 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.b/config 2024-06-11T13:28:32.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:32 smithi057 ceph-mon[40588]: from='client.24481 v1:172.21.15.107:0/1782277782' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi057=a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:28:32.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:32 smithi057 ceph-mon[40588]: Saving service alertmanager spec with placement smithi057=a;count:1 2024-06-11T13:28:32.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:32 smithi057 ceph-mon[29805]: from='client.24481 v1:172.21.15.107:0/1782277782' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi057=a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:28:32.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:32 smithi057 ceph-mon[29805]: Saving service alertmanager spec with placement smithi057=a;count:1 2024-06-11T13:28:32.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:32 smithi107 ceph-mon[33178]: from='client.24481 v1:172.21.15.107:0/1782277782' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi057=a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:28:32.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:32 smithi107 ceph-mon[33178]: Saving service alertmanager spec with placement smithi057=a;count:1 2024-06-11T13:28:33.674 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:33 smithi107 ceph-mon[33178]: pgmap v202: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 42 KiB/s rd, 2.8 KiB/s wr, 100 op/s 2024-06-11T13:28:34.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:33 smithi057 ceph-mon[29805]: pgmap v202: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 42 KiB/s rd, 2.8 KiB/s wr, 100 op/s 2024-06-11T13:28:34.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:33 smithi057 ceph-mon[40588]: pgmap v202: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 42 KiB/s rd, 2.8 KiB/s wr, 100 op/s 2024-06-11T13:28:34.994 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:28:34 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:28:35.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:35.476+0000 7fecf9f3d640 1 -- 172.21.15.107:0/282653658 >> v1:172.21.15.107:6789/0 conn(0x7fecf40fff30 legacy=0x7fecf4100350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:35.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:35.477+0000 7fecf9f3d640 1 -- 172.21.15.107:0/282653658 shutdown_connections 2024-06-11T13:28:35.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:35.477+0000 7fecf9f3d640 1 -- 172.21.15.107:0/282653658 >> 172.21.15.107:0/282653658 conn(0x7fecf40fb4f0 msgr2=0x7fecf40fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:28:35.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:35.477+0000 7fecf9f3d640 1 -- 172.21.15.107:0/282653658 shutdown_connections 2024-06-11T13:28:35.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:35.477+0000 7fecf9f3d640 1 -- 172.21.15.107:0/282653658 wait complete. 2024-06-11T13:28:35.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:35.477+0000 7fecf9f3d640 1 Processor -- start 2024-06-11T13:28:35.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:35.477+0000 7fecf9f3d640 1 -- start start 2024-06-11T13:28:35.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:35.478+0000 7fecf9f3d640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fecf41975e0 con 0x7fecf4104350 2024-06-11T13:28:35.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:35.478+0000 7fecf9f3d640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fecf41a2d90 con 0x7fecf4107fb0 2024-06-11T13:28:35.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:35.478+0000 7fecf9f3d640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fecf41a3f70 con 0x7fecf40fff30 2024-06-11T13:28:35.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:35.478+0000 7fecf3fff640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7fecf4107fb0 0x7fecf41a0660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.107:59466/0 (socket says 172.21.15.107:59466) 2024-06-11T13:28:35.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:35.478+0000 7fecf3fff640 1 -- 172.21.15.107:0/1694499457 learned_addr learned my addr 172.21.15.107:0/1694499457 (peer_addr_for_me v1:172.21.15.107:0/0) 2024-06-11T13:28:35.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:35.478+0000 7fecf17fa640 1 -- 172.21.15.107:0/1694499457 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 959776485 0 0) 0x7fecf41a2d90 con 0x7fecf4107fb0 2024-06-11T13:28:35.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:35.478+0000 7fecf17fa640 1 -- 172.21.15.107:0/1694499457 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fece0004630 con 0x7fecf4107fb0 2024-06-11T13:28:35.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:35.478+0000 7fecf17fa640 1 -- 172.21.15.107:0/1694499457 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2023042281 0 0) 0x7fecf41a3f70 con 0x7fecf40fff30 2024-06-11T13:28:35.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:35.478+0000 7fecf17fa640 1 -- 172.21.15.107:0/1694499457 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fecf41a2d90 con 0x7fecf40fff30 2024-06-11T13:28:35.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:35.478+0000 7fecf17fa640 1 -- 172.21.15.107:0/1694499457 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 262620302 0 0) 0x7fecf41975e0 con 0x7fecf4104350 2024-06-11T13:28:35.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:35.478+0000 7fecf17fa640 1 -- 172.21.15.107:0/1694499457 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fecf41a3f70 con 0x7fecf4104350 2024-06-11T13:28:35.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:35.478+0000 7fecf17fa640 1 -- 172.21.15.107:0/1694499457 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3718560610 0 0) 0x7fece0004630 con 0x7fecf4107fb0 2024-06-11T13:28:35.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:35.478+0000 7fecf17fa640 1 -- 172.21.15.107:0/1694499457 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fecf41975e0 con 0x7fecf4107fb0 2024-06-11T13:28:35.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:35.478+0000 7fecf17fa640 1 -- 172.21.15.107:0/1694499457 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fecec001cb0 con 0x7fecf4107fb0 2024-06-11T13:28:35.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:35.478+0000 7fecf17fa640 1 -- 172.21.15.107:0/1694499457 <== mon.1 v1:172.21.15.107:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2319054876 0 0) 0x7fecf41975e0 con 0x7fecf4107fb0 2024-06-11T13:28:35.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:35.478+0000 7fecf17fa640 1 -- 172.21.15.107:0/1694499457 >> v1:172.21.15.57:6790/0 conn(0x7fecf40fff30 legacy=0x7fecf4196af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:35.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:35.478+0000 7fecf17fa640 1 -- 172.21.15.107:0/1694499457 >> v1:172.21.15.57:6789/0 conn(0x7fecf4104350 legacy=0x7fecf419cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:35.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:35.478+0000 7fecf17fa640 1 -- 172.21.15.107:0/1694499457 --> v1:172.21.15.107:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecf41a5150 con 0x7fecf4107fb0 2024-06-11T13:28:35.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:35.478+0000 7fecf9f3d640 1 -- 172.21.15.107:0/1694499457 --> v1:172.21.15.107:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fecf41a41a0 con 0x7fecf4107fb0 2024-06-11T13:28:35.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:35.478+0000 7fecf9f3d640 1 -- 172.21.15.107:0/1694499457 --> v1:172.21.15.107:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fecf41a46e0 con 0x7fecf4107fb0 2024-06-11T13:28:35.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:35.478+0000 7fecf17fa640 1 -- 172.21.15.107:0/1694499457 <== mon.1 v1:172.21.15.107:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7fecec0040d0 con 0x7fecf4107fb0 2024-06-11T13:28:35.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:35.479+0000 7fecf17fa640 1 -- 172.21.15.107:0/1694499457 <== mon.1 v1:172.21.15.107:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fecec006060 con 0x7fecf4107fb0 2024-06-11T13:28:35.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:35.479+0000 7fecf9f3d640 1 -- 172.21.15.107:0/1694499457 --> v1:172.21.15.107:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecd40050b0 con 0x7fecf4107fb0 2024-06-11T13:28:35.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:35.480+0000 7fecf17fa640 1 -- 172.21.15.107:0/1694499457 <== mon.1 v1:172.21.15.107:6789/0 7 ==== mgrmap(e 16) ==== 92132+0+0 (unknown 2632097432 0 0) 0x7fecec01cb10 con 0x7fecf4107fb0 2024-06-11T13:28:35.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:35.480+0000 7fecf17fa640 1 -- 172.21.15.107:0/1694499457 <== mon.1 v1:172.21.15.107:6789/0 8 ==== osd_map(69..69 src has 1..69) ==== 5942+0+0 (unknown 3863100797 0 0) 0x7fecec089960 con 0x7fecf4107fb0 2024-06-11T13:28:35.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:35.485+0000 7fecf17fa640 1 -- 172.21.15.107:0/1694499457 <== mon.1 v1:172.21.15.107:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7fecec057120 con 0x7fecf4107fb0 2024-06-11T13:28:35.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:35.682+0000 7fecf9f3d640 1 -- 172.21.15.107:0/1694499457 --> v1:172.21.15.57:6800/2989665238 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi107=a", "target": ["mon-mgr", ""]}) -- 0x7fecd4000bc0 con 0x7fece006edb0 2024-06-11T13:28:35.678 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:35 smithi107 ceph-mon[33178]: pgmap v203: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 32 KiB/s rd, 2.1 KiB/s wr, 75 op/s 2024-06-11T13:28:35.679 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:35 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:28:35.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:35.694+0000 7fecf17fa640 1 -- 172.21.15.107:0/1694499457 <== mgr.14150 v1:172.21.15.57:6800/2989665238 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (unknown 0 0 664801700) 0x7fecd4000bc0 con 0x7fece006edb0 2024-06-11T13:28:35.690 INFO:teuthology.orchestra.run.smithi107.stdout:Scheduled grafana update... 2024-06-11T13:28:35.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:35.696+0000 7fecf9f3d640 1 -- 172.21.15.107:0/1694499457 >> v1:172.21.15.57:6800/2989665238 conn(0x7fece006edb0 legacy=0x7fece0071270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:35.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:35.696+0000 7fecf9f3d640 1 -- 172.21.15.107:0/1694499457 >> v1:172.21.15.107:6789/0 conn(0x7fecf4107fb0 legacy=0x7fecf41a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:35.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:35.696+0000 7fecf9f3d640 1 -- 172.21.15.107:0/1694499457 shutdown_connections 2024-06-11T13:28:35.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:35.696+0000 7fecf9f3d640 1 -- 172.21.15.107:0/1694499457 >> 172.21.15.107:0/1694499457 conn(0x7fecf40fb4f0 msgr2=0x7fecf40fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:28:35.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:35.696+0000 7fecf9f3d640 1 -- 172.21.15.107:0/1694499457 shutdown_connections 2024-06-11T13:28:35.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:35.696+0000 7fecf9f3d640 1 -- 172.21.15.107:0/1694499457 wait complete. 2024-06-11T13:28:36.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:35 smithi057 ceph-mon[29805]: pgmap v203: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 32 KiB/s rd, 2.1 KiB/s wr, 75 op/s 2024-06-11T13:28:36.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:35 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:28:36.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:35 smithi057 ceph-mon[40588]: pgmap v203: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 32 KiB/s rd, 2.1 KiB/s wr, 75 op/s 2024-06-11T13:28:36.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:35 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:28:37.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:36 smithi057 ceph-mon[40588]: from='client.24487 v1:172.21.15.107:0/1694499457' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi107=a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:28:37.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:36 smithi057 ceph-mon[40588]: Saving service grafana spec with placement smithi107=a;count:1 2024-06-11T13:28:37.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:36 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:37.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:36 smithi057 ceph-mon[29805]: from='client.24487 v1:172.21.15.107:0/1694499457' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi107=a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:28:37.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:36 smithi057 ceph-mon[29805]: Saving service grafana spec with placement smithi107=a;count:1 2024-06-11T13:28:37.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:36 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:37.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:36 smithi107 ceph-mon[33178]: from='client.24487 v1:172.21.15.107:0/1694499457' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi107=a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-11T13:28:37.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:36 smithi107 ceph-mon[33178]: Saving service grafana spec with placement smithi107=a;count:1 2024-06-11T13:28:37.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:36 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:37.768 DEBUG:teuthology.orchestra.run.smithi107:grafana.a> sudo journalctl -f -n 0 -u ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@grafana.a.service 2024-06-11T13:28:37.771 INFO:tasks.cephadm:Setting up client nodes... 2024-06-11T13:28:37.771 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-06-11T13:28:38.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:37 smithi057 ceph-mon[29805]: pgmap v204: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 85 B/s wr, 15 op/s 2024-06-11T13:28:38.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:37 smithi057 ceph-mon[40588]: pgmap v204: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 85 B/s wr, 15 op/s 2024-06-11T13:28:38.033 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.a/config 2024-06-11T13:28:38.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:37 smithi107 ceph-mon[33178]: pgmap v204: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 85 B/s wr, 15 op/s 2024-06-11T13:28:39.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:39.775+0000 7f1e463ef640 1 -- 172.21.15.57:0/468895175 >> v1:172.21.15.57:6789/0 conn(0x7f1e40100760 legacy=0x7f1e40100b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:39.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:39.775+0000 7f1e463ef640 1 -- 172.21.15.57:0/468895175 shutdown_connections 2024-06-11T13:28:39.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:39.775+0000 7f1e463ef640 1 -- 172.21.15.57:0/468895175 >> 172.21.15.57:0/468895175 conn(0x7f1e400fbd40 msgr2=0x7f1e400fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:28:39.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:39.775+0000 7f1e463ef640 1 -- 172.21.15.57:0/468895175 shutdown_connections 2024-06-11T13:28:39.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:39.775+0000 7f1e463ef640 1 -- 172.21.15.57:0/468895175 wait complete. 2024-06-11T13:28:39.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:39.775+0000 7f1e463ef640 1 Processor -- start 2024-06-11T13:28:39.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:39.775+0000 7f1e463ef640 1 -- start start 2024-06-11T13:28:39.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:39.776+0000 7f1e463ef640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f1e4006ac40 con 0x7f1e401088d0 2024-06-11T13:28:39.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:39.776+0000 7f1e463ef640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f1e401a2d40 con 0x7f1e40100760 2024-06-11T13:28:39.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:39.776+0000 7f1e463ef640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f1e401a3f60 con 0x7f1e40104b90 2024-06-11T13:28:39.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:39.776+0000 7f1e37fff640 1 --1- >> v1:172.21.15.57:6790/0 conn(0x7f1e40104b90 0x7f1e4019cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6790/0 says I am v1:172.21.15.57:54588/0 (socket says 172.21.15.57:54588) 2024-06-11T13:28:39.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:39.776+0000 7f1e37fff640 1 -- 172.21.15.57:0/2058797524 learned_addr learned my addr 172.21.15.57:0/2058797524 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:28:39.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:39.776+0000 7f1e3dffb640 1 -- 172.21.15.57:0/2058797524 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 167690629 0 0) 0x7f1e401a3f60 con 0x7f1e40104b90 2024-06-11T13:28:39.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:39.776+0000 7f1e3dffb640 1 -- 172.21.15.57:0/2058797524 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e2c003520 con 0x7f1e40104b90 2024-06-11T13:28:39.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:39.776+0000 7f1e3dffb640 1 -- 172.21.15.57:0/2058797524 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 563260011 0 0) 0x7f1e4006ac40 con 0x7f1e401088d0 2024-06-11T13:28:39.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:39.776+0000 7f1e3dffb640 1 -- 172.21.15.57:0/2058797524 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e401a3f60 con 0x7f1e401088d0 2024-06-11T13:28:39.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:39.776+0000 7f1e3dffb640 1 -- 172.21.15.57:0/2058797524 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1735356715 0 0) 0x7f1e401a3f60 con 0x7f1e401088d0 2024-06-11T13:28:39.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:39.776+0000 7f1e3dffb640 1 -- 172.21.15.57:0/2058797524 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1e4006ac40 con 0x7f1e401088d0 2024-06-11T13:28:39.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:39.776+0000 7f1e3dffb640 1 -- 172.21.15.57:0/2058797524 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f1e38001c30 con 0x7f1e401088d0 2024-06-11T13:28:39.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:39.776+0000 7f1e3dffb640 1 -- 172.21.15.57:0/2058797524 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 839995698 0 0) 0x7f1e2c003520 con 0x7f1e40104b90 2024-06-11T13:28:39.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:39.776+0000 7f1e3dffb640 1 -- 172.21.15.57:0/2058797524 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1e401a3f60 con 0x7f1e40104b90 2024-06-11T13:28:39.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:39.776+0000 7f1e3dffb640 1 -- 172.21.15.57:0/2058797524 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 61783530 0 0) 0x7f1e401a2d40 con 0x7f1e40100760 2024-06-11T13:28:39.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:39.776+0000 7f1e3dffb640 1 -- 172.21.15.57:0/2058797524 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e2c003520 con 0x7f1e40100760 2024-06-11T13:28:39.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:39.776+0000 7f1e3dffb640 1 -- 172.21.15.57:0/2058797524 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f1e28001810 con 0x7f1e40104b90 2024-06-11T13:28:39.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:39.776+0000 7f1e3dffb640 1 -- 172.21.15.57:0/2058797524 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 773814604 0 0) 0x7f1e4006ac40 con 0x7f1e401088d0 2024-06-11T13:28:39.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:39.776+0000 7f1e3dffb640 1 -- 172.21.15.57:0/2058797524 >> v1:172.21.15.57:6790/0 conn(0x7f1e40104b90 legacy=0x7f1e4019cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:39.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:39.776+0000 7f1e3dffb640 1 -- 172.21.15.57:0/2058797524 >> v1:172.21.15.107:6789/0 conn(0x7f1e40100760 legacy=0x7f1e401997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:39.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:39.776+0000 7f1e3dffb640 1 -- 172.21.15.57:0/2058797524 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e401a51a0 con 0x7f1e401088d0 2024-06-11T13:28:39.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:39.776+0000 7f1e463ef640 1 -- 172.21.15.57:0/2058797524 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f1e401a4190 con 0x7f1e401088d0 2024-06-11T13:28:39.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:39.777+0000 7f1e463ef640 1 -- 172.21.15.57:0/2058797524 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f1e401a46d0 con 0x7f1e401088d0 2024-06-11T13:28:39.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:39.777+0000 7f1e3dffb640 1 -- 172.21.15.57:0/2058797524 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f1e38003e20 con 0x7f1e401088d0 2024-06-11T13:28:39.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:39.777+0000 7f1e3dffb640 1 -- 172.21.15.57:0/2058797524 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f1e38004d80 con 0x7f1e401088d0 2024-06-11T13:28:39.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:39.778+0000 7f1e463ef640 1 -- 172.21.15.57:0/2058797524 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e200050b0 con 0x7f1e401088d0 2024-06-11T13:28:39.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:39.778+0000 7f1e3dffb640 1 -- 172.21.15.57:0/2058797524 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 16) ==== 92132+0+0 (unknown 2632097432 0 0) 0x7f1e38003970 con 0x7f1e401088d0 2024-06-11T13:28:39.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:39.779+0000 7f1e3dffb640 1 -- 172.21.15.57:0/2058797524 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(69..69 src has 1..69) ==== 5942+0+0 (unknown 3863100797 0 0) 0x7f1e38088600 con 0x7f1e401088d0 2024-06-11T13:28:39.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:39.783+0000 7f1e3dffb640 1 -- 172.21.15.57:0/2058797524 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f1e38055da0 con 0x7f1e401088d0 2024-06-11T13:28:39.981 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:39 smithi107 ceph-mon[33178]: pgmap v205: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-06-11T13:28:40.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:39 smithi057 ceph-mon[29805]: pgmap v205: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-06-11T13:28:40.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:39 smithi057 ceph-mon[40588]: pgmap v205: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-06-11T13:28:40.034 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:40.038+0000 7f1e463ef640 1 -- 172.21.15.57:0/2058797524 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f1e200053a0 con 0x7f1e401088d0 2024-06-11T13:28:40.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:40.045+0000 7f1e3dffb640 1 -- 172.21.15.57:0/2058797524 <== mon.0 v1:172.21.15.57:6789/0 10 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) ==== 170+0+59 (unknown 57899324 0 3172150891) 0x7f1e380556a0 con 0x7f1e401088d0 2024-06-11T13:28:40.041 INFO:teuthology.orchestra.run.smithi057.stdout:[client.0] 2024-06-11T13:28:40.041 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQAIUWhmxSlgAhAAwxO6iHJQj/AnIZerkT+yWQ== 2024-06-11T13:28:40.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:40.047+0000 7f1e463ef640 1 -- 172.21.15.57:0/2058797524 >> v1:172.21.15.57:6800/2989665238 conn(0x7f1e2c06ee90 legacy=0x7f1e2c071350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:40.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:40.047+0000 7f1e463ef640 1 -- 172.21.15.57:0/2058797524 >> v1:172.21.15.57:6789/0 conn(0x7f1e401088d0 legacy=0x7f1e401a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:40.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:40.047+0000 7f1e463ef640 1 -- 172.21.15.57:0/2058797524 shutdown_connections 2024-06-11T13:28:40.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:40.047+0000 7f1e463ef640 1 -- 172.21.15.57:0/2058797524 >> 172.21.15.57:0/2058797524 conn(0x7f1e400fbd40 msgr2=0x7f1e4010b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:28:40.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:40.047+0000 7f1e463ef640 1 -- 172.21.15.57:0/2058797524 shutdown_connections 2024-06-11T13:28:40.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:40.047+0000 7f1e463ef640 1 -- 172.21.15.57:0/2058797524 wait complete. 2024-06-11T13:28:40.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:40 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2058797524' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-11T13:28:40.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:40 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2058797524' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-11T13:28:40.890 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-06-11T13:28:40.890 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-06-11T13:28:40.890 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-06-11T13:28:40.933 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-06-11T13:28:41.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:40 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2058797524' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-11T13:28:41.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:40 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2058797524' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-11T13:28:41.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:40 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2058797524' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-11T13:28:41.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:40 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2058797524' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-11T13:28:41.190 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.b/config 2024-06-11T13:28:42.096 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:42 smithi107 ceph-mon[33178]: pgmap v206: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 1 op/s 2024-06-11T13:28:42.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:42 smithi057 ceph-mon[29805]: pgmap v206: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 1 op/s 2024-06-11T13:28:42.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:42 smithi057 ceph-mon[40588]: pgmap v206: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 1 op/s 2024-06-11T13:28:42.547 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:28:42 smithi107 systemd[1]: Starting Ceph prometheus.a for 98a6635e-27f5-11ef-bc9b-c7b262605968... 2024-06-11T13:28:42.884 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:28:42 smithi107 podman[61358]: 2024-06-11 13:28:42.471320926 +0000 UTC m=+0.018659491 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-06-11T13:28:43.136 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:28:43 smithi107 podman[61358]: 2024-06-11 13:28:43.030619257 +0000 UTC m=+0.577957798 container create 7dc6bb13301019745064a621f3c9d4a41d60fb7734d55e245bea28bbbcad91b7 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-11T13:28:43.750 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:28:43 smithi107 podman[61358]: 2024-06-11 13:28:43.511152429 +0000 UTC m=+1.058490979 container init 7dc6bb13301019745064a621f3c9d4a41d60fb7734d55e245bea28bbbcad91b7 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-11T13:28:43.750 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:28:43 smithi107 podman[61358]: 2024-06-11 13:28:43.514545673 +0000 UTC m=+1.061884224 container start 7dc6bb13301019745064a621f3c9d4a41d60fb7734d55e245bea28bbbcad91b7 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-11T13:28:43.750 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:28:43 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[61394]: ts=2024-06-11T13:28:43.538Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-06-11T13:28:43.750 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:28:43 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[61394]: ts=2024-06-11T13:28:43.538Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-06-11T13:28:43.750 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:28:43 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[61394]: ts=2024-06-11T13:28:43.538Z caller=main.go:570 level=info host_details="(Linux 5.14.0-452.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024 x86_64 smithi107 (none))" 2024-06-11T13:28:43.750 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:28:43 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[61394]: ts=2024-06-11T13:28:43.538Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-06-11T13:28:43.750 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:28:43 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[61394]: ts=2024-06-11T13:28:43.538Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-06-11T13:28:43.750 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:28:43 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[61394]: ts=2024-06-11T13:28:43.539Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-06-11T13:28:43.750 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:28:43 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[61394]: ts=2024-06-11T13:28:43.540Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-06-11T13:28:43.751 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:28:43 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[61394]: ts=2024-06-11T13:28:43.540Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-06-11T13:28:43.751 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:28:43 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[61394]: ts=2024-06-11T13:28:43.541Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-06-11T13:28:43.751 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:28:43 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[61394]: ts=2024-06-11T13:28:43.542Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-06-11T13:28:43.751 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:28:43 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[61394]: ts=2024-06-11T13:28:43.542Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=3.63µs 2024-06-11T13:28:43.751 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:28:43 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[61394]: ts=2024-06-11T13:28:43.542Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-06-11T13:28:43.751 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:28:43 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[61394]: ts=2024-06-11T13:28:43.543Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-06-11T13:28:43.751 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:28:43 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[61394]: ts=2024-06-11T13:28:43.543Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=40.855µs wal_replay_duration=537.464µs wbl_replay_duration=117ns total_replay_duration=816.322µs 2024-06-11T13:28:43.751 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:28:43 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[61394]: ts=2024-06-11T13:28:43.544Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-06-11T13:28:43.751 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:28:43 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[61394]: ts=2024-06-11T13:28:43.544Z caller=main.go:1029 level=info msg="TSDB started" 2024-06-11T13:28:43.751 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:28:43 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[61394]: ts=2024-06-11T13:28:43.544Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-06-11T13:28:43.751 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:28:43 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[61394]: ts=2024-06-11T13:28:43.565Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=21.236776ms db_storage=703ns remote_storage=1.054µs web_handler=369ns query_engine=527ns scrape=465.083µs scrape_sd=72.101µs notify=691ns notify_sd=1.217µs rules=20.426908ms tracing=5.57µs 2024-06-11T13:28:43.751 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:28:43 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[61394]: ts=2024-06-11T13:28:43.565Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-06-11T13:28:43.751 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:28:43 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[61394]: ts=2024-06-11T13:28:43.565Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-06-11T13:28:43.751 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:28:43 smithi107 bash[61358]: 7dc6bb13301019745064a621f3c9d4a41d60fb7734d55e245bea28bbbcad91b7 2024-06-11T13:28:43.752 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:28:43 smithi107 systemd[1]: Started Ceph prometheus.a for 98a6635e-27f5-11ef-bc9b-c7b262605968. 2024-06-11T13:28:43.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:43.844+0000 7f6dc227a640 1 -- 172.21.15.107:0/2773034457 >> v1:172.21.15.107:6789/0 conn(0x7f6dbc104b70 legacy=0x7f6dbc106fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:43.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:43.844+0000 7f6dc227a640 1 -- 172.21.15.107:0/2773034457 shutdown_connections 2024-06-11T13:28:43.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:43.844+0000 7f6dc227a640 1 -- 172.21.15.107:0/2773034457 >> 172.21.15.107:0/2773034457 conn(0x7f6dbc0fbd00 msgr2=0x7f6dbc0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:28:43.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:43.844+0000 7f6dc227a640 1 -- 172.21.15.107:0/2773034457 shutdown_connections 2024-06-11T13:28:43.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:43.844+0000 7f6dc227a640 1 -- 172.21.15.107:0/2773034457 wait complete. 2024-06-11T13:28:43.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:43.844+0000 7f6dc227a640 1 Processor -- start 2024-06-11T13:28:43.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:43.844+0000 7f6dc227a640 1 -- start start 2024-06-11T13:28:43.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:43.845+0000 7f6dc227a640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f6dbc1a40e0 con 0x7f6dbc104b70 2024-06-11T13:28:43.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:43.845+0000 7f6dc227a640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f6dbc1a5320 con 0x7f6dbc1088b0 2024-06-11T13:28:43.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:43.845+0000 7f6dc227a640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f6dbc1a6560 con 0x7f6dbc100740 2024-06-11T13:28:43.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:43.845+0000 7f6dc0817640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f6dbc1088b0 0x7f6dbc1a27e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.107:55882/0 (socket says 172.21.15.107:55882) 2024-06-11T13:28:43.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:43.845+0000 7f6dc0817640 1 -- 172.21.15.107:0/816836126 learned_addr learned my addr 172.21.15.107:0/816836126 (peer_addr_for_me v1:172.21.15.107:0/0) 2024-06-11T13:28:43.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:43.845+0000 7f6db97fa640 1 -- 172.21.15.107:0/816836126 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2937709537 0 0) 0x7f6dbc1a5320 con 0x7f6dbc1088b0 2024-06-11T13:28:43.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:43.845+0000 7f6db97fa640 1 -- 172.21.15.107:0/816836126 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6da4003520 con 0x7f6dbc1088b0 2024-06-11T13:28:43.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:43.845+0000 7f6db97fa640 1 -- 172.21.15.107:0/816836126 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3239365118 0 0) 0x7f6dbc1a6560 con 0x7f6dbc100740 2024-06-11T13:28:43.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:43.845+0000 7f6db97fa640 1 -- 172.21.15.107:0/816836126 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6dbc1a5320 con 0x7f6dbc100740 2024-06-11T13:28:43.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:43.845+0000 7f6db97fa640 1 -- 172.21.15.107:0/816836126 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1680497639 0 0) 0x7f6dbc1a40e0 con 0x7f6dbc104b70 2024-06-11T13:28:43.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:43.845+0000 7f6db97fa640 1 -- 172.21.15.107:0/816836126 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6dbc1a6560 con 0x7f6dbc104b70 2024-06-11T13:28:43.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:43.845+0000 7f6db97fa640 1 -- 172.21.15.107:0/816836126 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 154863462 0 0) 0x7f6da4003520 con 0x7f6dbc1088b0 2024-06-11T13:28:43.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:43.845+0000 7f6db97fa640 1 -- 172.21.15.107:0/816836126 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6dbc1a40e0 con 0x7f6dbc1088b0 2024-06-11T13:28:43.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:43.845+0000 7f6db97fa640 1 -- 172.21.15.107:0/816836126 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f6db4001ba0 con 0x7f6dbc1088b0 2024-06-11T13:28:43.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:43.845+0000 7f6db97fa640 1 -- 172.21.15.107:0/816836126 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3245802919 0 0) 0x7f6dbc1a6560 con 0x7f6dbc104b70 2024-06-11T13:28:43.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:43.845+0000 7f6db97fa640 1 -- 172.21.15.107:0/816836126 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6da4003520 con 0x7f6dbc104b70 2024-06-11T13:28:43.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:43.845+0000 7f6db97fa640 1 -- 172.21.15.107:0/816836126 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f6db0001430 con 0x7f6dbc104b70 2024-06-11T13:28:43.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:43.845+0000 7f6db97fa640 1 -- 172.21.15.107:0/816836126 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4172545072 0 0) 0x7f6dbc1a5320 con 0x7f6dbc100740 2024-06-11T13:28:43.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:43.845+0000 7f6db97fa640 1 -- 172.21.15.107:0/816836126 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6dbc1a6560 con 0x7f6dbc100740 2024-06-11T13:28:43.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:43.845+0000 7f6db97fa640 1 -- 172.21.15.107:0/816836126 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f6dac001830 con 0x7f6dbc100740 2024-06-11T13:28:43.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:43.845+0000 7f6db97fa640 1 -- 172.21.15.107:0/816836126 <== mon.1 v1:172.21.15.107:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3116517791 0 0) 0x7f6dbc1a40e0 con 0x7f6dbc1088b0 2024-06-11T13:28:43.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:43.845+0000 7f6db97fa640 1 -- 172.21.15.107:0/816836126 >> v1:172.21.15.57:6790/0 conn(0x7f6dbc100740 legacy=0x7f6dbc106a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:43.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:43.846+0000 7f6db97fa640 1 -- 172.21.15.107:0/816836126 >> v1:172.21.15.57:6789/0 conn(0x7f6dbc104b70 legacy=0x7f6dbc19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:43.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:43.846+0000 7f6db97fa640 1 -- 172.21.15.107:0/816836126 --> v1:172.21.15.107:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dbc1a77a0 con 0x7f6dbc1088b0 2024-06-11T13:28:43.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:43.846+0000 7f6dc227a640 1 -- 172.21.15.107:0/816836126 --> v1:172.21.15.107:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f6dbc1a6790 con 0x7f6dbc1088b0 2024-06-11T13:28:43.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:43.846+0000 7f6dc227a640 1 -- 172.21.15.107:0/816836126 --> v1:172.21.15.107:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f6dbc1a6cd0 con 0x7f6dbc1088b0 2024-06-11T13:28:43.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:43.846+0000 7f6db97fa640 1 -- 172.21.15.107:0/816836126 <== mon.1 v1:172.21.15.107:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f6db4003e20 con 0x7f6dbc1088b0 2024-06-11T13:28:43.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:43.846+0000 7f6db97fa640 1 -- 172.21.15.107:0/816836126 <== mon.1 v1:172.21.15.107:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f6db4005ed0 con 0x7f6dbc1088b0 2024-06-11T13:28:43.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:43.847+0000 7f6dc227a640 1 -- 172.21.15.107:0/816836126 --> v1:172.21.15.107:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6da80050b0 con 0x7f6dbc1088b0 2024-06-11T13:28:43.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:43.847+0000 7f6db97fa640 1 -- 172.21.15.107:0/816836126 <== mon.1 v1:172.21.15.107:6789/0 7 ==== mgrmap(e 16) ==== 92132+0+0 (unknown 2632097432 0 0) 0x7f6db4003990 con 0x7f6dbc1088b0 2024-06-11T13:28:43.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:43.848+0000 7f6db97fa640 1 -- 172.21.15.107:0/816836126 <== mon.1 v1:172.21.15.107:6789/0 8 ==== osd_map(69..69 src has 1..69) ==== 5942+0+0 (unknown 3863100797 0 0) 0x7f6db40896f0 con 0x7f6dbc1088b0 2024-06-11T13:28:43.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:43.852+0000 7f6db97fa640 1 -- 172.21.15.107:0/816836126 <== mon.1 v1:172.21.15.107:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f6db4056eb0 con 0x7f6dbc1088b0 2024-06-11T13:28:44.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:44.099+0000 7f6dc227a640 1 -- 172.21.15.107:0/816836126 --> v1:172.21.15.107:6789/0 -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f6da80053a0 con 0x7f6dbc1088b0 2024-06-11T13:28:44.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:44.104+0000 7f6db97fa640 1 -- 172.21.15.107:0/816836126 <== mon.1 v1:172.21.15.107:6789/0 10 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v18) ==== 170+0+59 (unknown 3508017507 0 27519858) 0x7f6db40567b0 con 0x7f6dbc1088b0 2024-06-11T13:28:44.100 INFO:teuthology.orchestra.run.smithi107.stdout:[client.1] 2024-06-11T13:28:44.100 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQAMUWhm+FwQBhAAdqG7jfyB46lOdlNQ921b3Q== 2024-06-11T13:28:44.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:44.106+0000 7f6dc227a640 1 -- 172.21.15.107:0/816836126 >> v1:172.21.15.57:6800/2989665238 conn(0x7f6da406ee50 legacy=0x7f6da4071310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:44.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:44.106+0000 7f6dc227a640 1 -- 172.21.15.107:0/816836126 >> v1:172.21.15.107:6789/0 conn(0x7f6dbc1088b0 legacy=0x7f6dbc1a27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:44.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:44.107+0000 7f6dc227a640 1 -- 172.21.15.107:0/816836126 shutdown_connections 2024-06-11T13:28:44.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:44.107+0000 7f6dc227a640 1 -- 172.21.15.107:0/816836126 >> 172.21.15.107:0/816836126 conn(0x7f6dbc0fbd00 msgr2=0x7f6dbc069c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:28:44.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:44.107+0000 7f6dc227a640 1 -- 172.21.15.107:0/816836126 shutdown_connections 2024-06-11T13:28:44.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-11T13:28:44.107+0000 7f6dc227a640 1 -- 172.21.15.107:0/816836126 wait complete. 2024-06-11T13:28:44.336 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:44 smithi057 ceph-mon[29805]: pgmap v207: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:28:44.336 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:44 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:44.336 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:44 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:44.336 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:44 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:44.336 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:44 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-06-11T13:28:44.336 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:44 smithi057 ceph-mon[40588]: pgmap v207: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:28:44.336 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:44 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:44.336 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:44 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:44.336 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:44 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:44.337 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:44 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-06-11T13:28:44.386 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:44 smithi107 ceph-mon[33178]: pgmap v207: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:28:44.386 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:44 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:44.386 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:44 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:44.386 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:44 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' 2024-06-11T13:28:44.386 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:44 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-06-11T13:28:44.975 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:28:44 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: ignoring --setuser ceph since I am not root 2024-06-11T13:28:44.976 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:28:44 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: ignoring --setgroup ceph since I am not root 2024-06-11T13:28:44.976 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:28:44 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:28:44.825+0000 7fe839717040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-11T13:28:44.976 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:28:44 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:28:44.979+0000 7fe839717040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-11T13:28:44.989 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:44 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ignoring --setuser ceph since I am not root 2024-06-11T13:28:44.989 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:44 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ignoring --setgroup ceph since I am not root 2024-06-11T13:28:44.989 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:44 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:28:44.835+0000 7febf508f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-11T13:28:44.989 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:44 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:28:44.992+0000 7febf508f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-11T13:28:45.254 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:28:45 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:28:45.050+0000 7fe839717040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-11T13:28:45.254 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:45 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.107:0/816836126' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-11T13:28:45.254 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:45 smithi107 ceph-mon[33178]: from='client.24499 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-11T13:28:45.254 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:45 smithi107 ceph-mon[33178]: from='client.24499 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-11T13:28:45.254 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:45 smithi107 ceph-mon[33178]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-06-11T13:28:45.254 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:45 smithi107 ceph-mon[33178]: mgrmap e17: y(active, since 4m), standbys: x 2024-06-11T13:28:45.283 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:45 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:28:45.065+0000 7febf508f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-11T13:28:45.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:45 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.107:0/816836126' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-11T13:28:45.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:45 smithi057 ceph-mon[29805]: from='client.24499 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-11T13:28:45.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:45 smithi057 ceph-mon[29805]: from='client.24499 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-11T13:28:45.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:45 smithi057 ceph-mon[29805]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-06-11T13:28:45.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:45 smithi057 ceph-mon[29805]: mgrmap e17: y(active, since 4m), standbys: x 2024-06-11T13:28:45.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:45 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.107:0/816836126' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-11T13:28:45.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:45 smithi057 ceph-mon[40588]: from='client.24499 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-11T13:28:45.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:45 smithi057 ceph-mon[40588]: from='client.24499 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-11T13:28:45.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:45 smithi057 ceph-mon[40588]: from='mgr.14150 v1:172.21.15.57:0/3133572991' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-06-11T13:28:45.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:45 smithi057 ceph-mon[40588]: mgrmap e17: y(active, since 4m), standbys: x 2024-06-11T13:28:45.546 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:28:45 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:28:45.310+0000 7fe839717040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-11T13:28:45.546 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:28:45 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:28:45.374+0000 7fe839717040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-11T13:28:45.783 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:45 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:28:45.327+0000 7febf508f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-11T13:28:45.783 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:45 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:28:45.390+0000 7febf508f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-11T13:28:46.226 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-06-11T13:28:46.226 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-06-11T13:28:46.227 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-06-11T13:28:46.265 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-06-11T13:28:46.265 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-06-11T13:28:46.266 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph mgr dump --format=json 2024-06-11T13:28:46.496 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:28:46 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:28:46.326+0000 7fe839717040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-11T13:28:46.496 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:28:46 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:28:46.442+0000 7fe839717040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-11T13:28:46.496 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:28:46 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:28:46.500+0000 7fe839717040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-11T13:28:46.521 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.a/config 2024-06-11T13:28:46.578 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:28:46.383+0000 7febf508f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-11T13:28:46.578 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:28:46.518+0000 7febf508f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-11T13:28:46.578 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:28:46.581+0000 7febf508f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-11T13:28:46.772 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:28:46 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:28:46.715+0000 7fe839717040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-11T13:28:46.877 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:28:46.815+0000 7febf508f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-11T13:28:46.877 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:28:46.881+0000 7febf508f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-11T13:28:47.046 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:28:46 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:28:46.776+0000 7fe839717040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-11T13:28:47.046 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:28:46 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:28:46.845+0000 7fe839717040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-11T13:28:47.046 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:28:46 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:28:46.993+0000 7fe839717040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-11T13:28:47.283 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:28:46.956+0000 7febf508f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-11T13:28:47.283 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:47 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:28:47.113+0000 7febf508f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-11T13:28:47.740 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:28:47 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:28:47.419+0000 7fe839717040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-11T13:28:47.741 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:28:47 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:28:47.744+0000 7fe839717040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-11T13:28:47.763 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:47 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:28:47.567+0000 7febf508f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-11T13:28:48.033 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:47 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:28:47.918+0000 7febf508f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-11T13:28:48.033 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:47 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:28:47.992+0000 7febf508f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-11T13:28:48.046 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:28:47 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:28:47.811+0000 7fe839717040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-11T13:28:48.046 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:28:47 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:28:47.964+0000 7fe839717040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-11T13:28:48.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.237+0000 7f010a1c0640 1 -- 172.21.15.57:0/4213175535 >> v1:172.21.15.57:6790/0 conn(0x7f01041006d0 legacy=0x7f0104100ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:48.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.237+0000 7f010a1c0640 1 -- 172.21.15.57:0/4213175535 shutdown_connections 2024-06-11T13:28:48.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.237+0000 7f010a1c0640 1 -- 172.21.15.57:0/4213175535 >> 172.21.15.57:0/4213175535 conn(0x7f0104075730 msgr2=0x7f0104075b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:28:48.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.237+0000 7f010a1c0640 1 -- 172.21.15.57:0/4213175535 shutdown_connections 2024-06-11T13:28:48.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.237+0000 7f010a1c0640 1 -- 172.21.15.57:0/4213175535 wait complete. 2024-06-11T13:28:48.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.238+0000 7f010a1c0640 1 Processor -- start 2024-06-11T13:28:48.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.238+0000 7f010a1c0640 1 -- start start 2024-06-11T13:28:48.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.238+0000 7f010a1c0640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f01041a6750 con 0x7f01041006d0 2024-06-11T13:28:48.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.238+0000 7f010a1c0640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f01041a7990 con 0x7f0104108df0 2024-06-11T13:28:48.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.239+0000 7f010a1c0640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f01041a8bd0 con 0x7f010410c960 2024-06-11T13:28:48.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.239+0000 7f01037fe640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f01041006d0 0x7f0104103d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:49858/0 (socket says 172.21.15.57:49858) 2024-06-11T13:28:48.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.239+0000 7f01037fe640 1 -- 172.21.15.57:0/2692572252 learned_addr learned my addr 172.21.15.57:0/2692572252 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:28:48.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.239+0000 7f0100ff9640 1 -- 172.21.15.57:0/2692572252 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 331449560 0 0) 0x7f01041a7990 con 0x7f0104108df0 2024-06-11T13:28:48.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.239+0000 7f0100ff9640 1 -- 172.21.15.57:0/2692572252 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00ec003160 con 0x7f0104108df0 2024-06-11T13:28:48.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.239+0000 7f0100ff9640 1 -- 172.21.15.57:0/2692572252 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 880940064 0 0) 0x7f01041a8bd0 con 0x7f010410c960 2024-06-11T13:28:48.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.239+0000 7f0100ff9640 1 -- 172.21.15.57:0/2692572252 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01041a7990 con 0x7f010410c960 2024-06-11T13:28:48.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.239+0000 7f0100ff9640 1 -- 172.21.15.57:0/2692572252 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1306464670 0 0) 0x7f01041a6750 con 0x7f01041006d0 2024-06-11T13:28:48.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.239+0000 7f0100ff9640 1 -- 172.21.15.57:0/2692572252 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01041a8bd0 con 0x7f01041006d0 2024-06-11T13:28:48.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.239+0000 7f0100ff9640 1 -- 172.21.15.57:0/2692572252 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3642842010 0 0) 0x7f00ec003160 con 0x7f0104108df0 2024-06-11T13:28:48.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.239+0000 7f0100ff9640 1 -- 172.21.15.57:0/2692572252 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f01041a6750 con 0x7f0104108df0 2024-06-11T13:28:48.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.239+0000 7f0100ff9640 1 -- 172.21.15.57:0/2692572252 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f00f80018b0 con 0x7f0104108df0 2024-06-11T13:28:48.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.239+0000 7f0100ff9640 1 -- 172.21.15.57:0/2692572252 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1767801206 0 0) 0x7f01041a8bd0 con 0x7f01041006d0 2024-06-11T13:28:48.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.239+0000 7f0100ff9640 1 -- 172.21.15.57:0/2692572252 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f00ec003160 con 0x7f01041006d0 2024-06-11T13:28:48.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.239+0000 7f0100ff9640 1 -- 172.21.15.57:0/2692572252 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f00f4001a00 con 0x7f01041006d0 2024-06-11T13:28:48.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.239+0000 7f0100ff9640 1 -- 172.21.15.57:0/2692572252 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4280064179 0 0) 0x7f01041a7990 con 0x7f010410c960 2024-06-11T13:28:48.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.239+0000 7f0100ff9640 1 -- 172.21.15.57:0/2692572252 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f01041a8bd0 con 0x7f010410c960 2024-06-11T13:28:48.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.239+0000 7f0100ff9640 1 -- 172.21.15.57:0/2692572252 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f00fc001a30 con 0x7f010410c960 2024-06-11T13:28:48.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.239+0000 7f0100ff9640 1 -- 172.21.15.57:0/2692572252 <== mon.1 v1:172.21.15.107:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3162001927 0 0) 0x7f01041a6750 con 0x7f0104108df0 2024-06-11T13:28:48.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.239+0000 7f0100ff9640 1 -- 172.21.15.57:0/2692572252 >> v1:172.21.15.57:6790/0 conn(0x7f010410c960 legacy=0x7f01041a4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:48.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.239+0000 7f0100ff9640 1 -- 172.21.15.57:0/2692572252 >> v1:172.21.15.57:6789/0 conn(0x7f01041006d0 legacy=0x7f0104103d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:48.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.239+0000 7f0100ff9640 1 -- 172.21.15.57:0/2692572252 --> v1:172.21.15.107:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01041a9e10 con 0x7f0104108df0 2024-06-11T13:28:48.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.240+0000 7f010a1c0640 1 -- 172.21.15.57:0/2692572252 --> v1:172.21.15.107:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f01041a6980 con 0x7f0104108df0 2024-06-11T13:28:48.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.240+0000 7f0100ff9640 1 -- 172.21.15.57:0/2692572252 <== mon.1 v1:172.21.15.107:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f00f8003aa0 con 0x7f0104108df0 2024-06-11T13:28:48.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.240+0000 7f0100ff9640 1 -- 172.21.15.57:0/2692572252 <== mon.1 v1:172.21.15.107:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f00f8004a30 con 0x7f0104108df0 2024-06-11T13:28:48.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.240+0000 7f010a1c0640 1 -- 172.21.15.57:0/2692572252 --> v1:172.21.15.107:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f01041a6e40 con 0x7f0104108df0 2024-06-11T13:28:48.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.241+0000 7f010a1c0640 1 -- 172.21.15.57:0/2692572252 --> v1:172.21.15.107:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0104068360 con 0x7f0104108df0 2024-06-11T13:28:48.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.241+0000 7f0100ff9640 1 -- 172.21.15.57:0/2692572252 <== mon.1 v1:172.21.15.107:6789/0 7 ==== mgrmap(e 17) ==== 92146+0+0 (unknown 4067323683 0 0) 0x7f00f8004d00 con 0x7f0104108df0 2024-06-11T13:28:48.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.242+0000 7f0100ff9640 1 -- 172.21.15.57:0/2692572252 <== mon.1 v1:172.21.15.107:6789/0 8 ==== osd_map(69..69 src has 1..69) ==== 5942+0+0 (unknown 3863100797 0 0) 0x7f00f8088380 con 0x7f0104108df0 2024-06-11T13:28:48.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.242+0000 7f01037fe640 1 -- 172.21.15.57:0/2692572252 >> v1:172.21.15.57:6800/2989665238 conn(0x7f00ec06efa0 legacy=0x7f00ec071460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.57:6800/2989665238 2024-06-11T13:28:48.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.246+0000 7f0100ff9640 1 -- 172.21.15.57:0/2692572252 <== mon.1 v1:172.21.15.107:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f00f8055a60 con 0x7f0104108df0 2024-06-11T13:28:48.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.443+0000 7f01037fe640 1 -- 172.21.15.57:0/2692572252 >> v1:172.21.15.57:6800/2989665238 conn(0x7f00ec06efa0 legacy=0x7f00ec071460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.57:6800/2989665238 2024-06-11T13:28:48.439 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:48 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:28:48.157+0000 7febf508f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-11T13:28:48.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.484+0000 7f010a1c0640 1 -- 172.21.15.57:0/2692572252 --> v1:172.21.15.107:6789/0 -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f0104111e80 con 0x7f0104108df0 2024-06-11T13:28:48.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.487+0000 7f0100ff9640 1 -- 172.21.15.57:0/2692572252 <== mon.1 v1:172.21.15.107:6789/0 10 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+175955 (unknown 831476310 0 906832010) 0x7f00f8055360 con 0x7f0104108df0 2024-06-11T13:28:48.486 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:28:48.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.489+0000 7f010a1c0640 1 -- 172.21.15.57:0/2692572252 >> v1:172.21.15.57:6800/2989665238 conn(0x7f00ec06efa0 legacy=0x7f00ec071460 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-11T13:28:48.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.489+0000 7f010a1c0640 1 -- 172.21.15.57:0/2692572252 >> v1:172.21.15.107:6789/0 conn(0x7f0104108df0 legacy=0x7f0104104410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:48.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.489+0000 7f010a1c0640 1 -- 172.21.15.57:0/2692572252 shutdown_connections 2024-06-11T13:28:48.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.489+0000 7f010a1c0640 1 -- 172.21.15.57:0/2692572252 >> 172.21.15.57:0/2692572252 conn(0x7f0104075730 msgr2=0x7f010410cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:28:48.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.490+0000 7f010a1c0640 1 -- 172.21.15.57:0/2692572252 shutdown_connections 2024-06-11T13:28:48.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:48.490+0000 7f010a1c0640 1 -- 172.21.15.57:0/2692572252 wait complete. 2024-06-11T13:28:48.525 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:28:48 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:28:48.278+0000 7fe839717040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-11T13:28:48.525 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:28:48 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:28:48.340+0000 7fe839717040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-11T13:28:48.767 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:48 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2692572252' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-11T13:28:48.767 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:48 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:28:48.501+0000 7febf508f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-11T13:28:48.767 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:48 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:28:48.569+0000 7febf508f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-11T13:28:48.767 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:48 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:28:48.770+0000 7febf508f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-11T13:28:48.768 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:48 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2692572252' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-11T13:28:48.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:48 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2692572252' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-11T13:28:48.796 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:28:48 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:28:48.529+0000 7fe839717040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-11T13:28:48.796 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:28:48 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:28:48.678+0000 7fe839717040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-11T13:28:48.797 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:28:48 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:28:48.744+0000 7fe839717040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-11T13:28:49.033 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:48 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:28:48.931+0000 7febf508f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-11T13:28:49.033 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:49 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:28:49.000+0000 7febf508f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-11T13:28:49.157 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"y","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6800","nonce":2989665238}]},"active_addr":"172.21.15.57:6800/2989665238","active_change":"2024-06-11T13:23:46.423818+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.57:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"cephadm","addrvec":[{"type":"v2","addr":"172.21.15.57:0","nonce":461546222}]},{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.57:0","nonce":2465286537}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.57:0","nonce":534678380}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.57:0","nonce":2279315136}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.57:0","nonce":1586679112}]}]} 2024-06-11T13:28:49.164 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-06-11T13:28:49.164 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-06-11T13:28:49.164 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-11T13:28:49.296 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:28:48 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:28:48.828+0000 7fe839717040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-11T13:28:49.296 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:28:48 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:28:48.889+0000 7fe839717040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-11T13:28:49.414 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.a/config 2024-06-11T13:28:49.533 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:49 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:28:49.089+0000 7febf508f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-11T13:28:49.533 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:49 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:28:49.155+0000 7febf508f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-11T13:28:49.702 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:49 smithi107 ceph-mon[33178]: Standby manager daemon x restarted 2024-06-11T13:28:49.702 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:49 smithi107 ceph-mon[33178]: Standby manager daemon x started 2024-06-11T13:28:49.702 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:49 smithi107 ceph-mon[33178]: from='mgr.? v1:172.21.15.107:0/2175033607' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-06-11T13:28:49.702 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:49 smithi107 ceph-mon[33178]: from='mgr.? v1:172.21.15.107:0/2175033607' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-11T13:28:49.703 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:49 smithi107 ceph-mon[33178]: from='mgr.? v1:172.21.15.107:0/2175033607' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-06-11T13:28:49.703 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:49 smithi107 ceph-mon[33178]: from='mgr.? v1:172.21.15.107:0/2175033607' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-11T13:28:49.703 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:28:49 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:28:49.433+0000 7fe839717040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-11T13:28:49.703 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:28:49 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:28:49.599+0000 7fe839717040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-11T13:28:49.703 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:28:49 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: [11/Jun/2024:13:28:49] ENGINE Bus STARTING 2024-06-11T13:28:49.703 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:28:49 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: CherryPy Checker: 2024-06-11T13:28:49.703 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:28:49 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: The Application mounted at '' has an empty config. 2024-06-11T13:28:49.703 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:28:49 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: 2024-06-11T13:28:49.703 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:28:49 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: [11/Jun/2024:13:28:49] ENGINE Serving on http://:::9283 2024-06-11T13:28:49.703 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 13:28:49 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x[35008]: [11/Jun/2024:13:28:49] ENGINE Bus STARTED 2024-06-11T13:28:49.807 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:49 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:28:49.758+0000 7febf508f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-11T13:28:49.807 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:49 smithi057 ceph-mon[40588]: Standby manager daemon x restarted 2024-06-11T13:28:49.807 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:49 smithi057 ceph-mon[40588]: Standby manager daemon x started 2024-06-11T13:28:49.807 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:49 smithi057 ceph-mon[40588]: from='mgr.? v1:172.21.15.107:0/2175033607' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-06-11T13:28:49.807 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:49 smithi057 ceph-mon[40588]: from='mgr.? v1:172.21.15.107:0/2175033607' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-11T13:28:49.807 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:49 smithi057 ceph-mon[40588]: from='mgr.? v1:172.21.15.107:0/2175033607' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-06-11T13:28:49.808 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:49 smithi057 ceph-mon[40588]: from='mgr.? v1:172.21.15.107:0/2175033607' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-11T13:28:49.808 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:49 smithi057 ceph-mon[29805]: Standby manager daemon x restarted 2024-06-11T13:28:49.808 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:49 smithi057 ceph-mon[29805]: Standby manager daemon x started 2024-06-11T13:28:49.808 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:49 smithi057 ceph-mon[29805]: from='mgr.? v1:172.21.15.107:0/2175033607' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-06-11T13:28:49.808 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:49 smithi057 ceph-mon[29805]: from='mgr.? v1:172.21.15.107:0/2175033607' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-11T13:28:49.808 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:49 smithi057 ceph-mon[29805]: from='mgr.? v1:172.21.15.107:0/2175033607' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-06-11T13:28:49.808 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:49 smithi057 ceph-mon[29805]: from='mgr.? v1:172.21.15.107:0/2175033607' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-11T13:28:50.137 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:49 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:28:49.934+0000 7febf508f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-11T13:28:50.137 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: [11/Jun/2024:13:28:50] ENGINE Bus STARTING 2024-06-11T13:28:50.138 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: CherryPy Checker: 2024-06-11T13:28:50.138 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: The Application mounted at '' has an empty config. 2024-06-11T13:28:50.138 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:28:50.533 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: [11/Jun/2024:13:28:50] ENGINE Serving on http://:::9283 2024-06-11T13:28:50.533 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: [11/Jun/2024:13:28:50] ENGINE Bus STARTED 2024-06-11T13:28:50.913 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[29805]: mgrmap e18: y(active, since 5m), standbys: x 2024-06-11T13:28:50.914 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[29805]: Active manager daemon y restarted 2024-06-11T13:28:50.914 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[29805]: Activating manager daemon y 2024-06-11T13:28:50.914 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[29805]: osdmap e70: 8 total, 8 up, 8 in 2024-06-11T13:28:50.914 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[29805]: mgrmap e19: y(active, starting, since 0.0117318s), standbys: x 2024-06-11T13:28:50.914 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-11T13:28:50.914 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-11T13:28:50.914 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-11T13:28:50.914 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-11T13:28:50.914 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-11T13:28:50.914 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-11T13:28:50.914 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-11T13:28:50.915 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-11T13:28:50.915 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-11T13:28:50.915 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-11T13:28:50.915 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-11T13:28:50.915 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-11T13:28:50.915 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-11T13:28:50.915 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-11T13:28:50.915 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-11T13:28:50.915 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-11T13:28:50.915 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[29805]: Manager daemon y is now available 2024-06-11T13:28:50.915 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:28:50.915 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:50.915 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:28:50.915 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-11T13:28:50.916 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-11T13:28:50.916 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-11T13:28:50.916 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-11T13:28:50.916 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[40588]: mgrmap e18: y(active, since 5m), standbys: x 2024-06-11T13:28:50.916 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[40588]: Active manager daemon y restarted 2024-06-11T13:28:50.916 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[40588]: Activating manager daemon y 2024-06-11T13:28:50.916 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[40588]: osdmap e70: 8 total, 8 up, 8 in 2024-06-11T13:28:50.916 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[40588]: mgrmap e19: y(active, starting, since 0.0117318s), standbys: x 2024-06-11T13:28:50.916 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-11T13:28:50.916 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-11T13:28:50.916 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-11T13:28:50.916 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-11T13:28:50.917 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-11T13:28:50.917 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-11T13:28:50.917 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-11T13:28:50.917 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-11T13:28:50.917 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-11T13:28:50.917 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-11T13:28:50.917 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-11T13:28:50.917 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-11T13:28:50.917 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-11T13:28:50.917 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-11T13:28:50.917 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-11T13:28:50.917 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-11T13:28:50.917 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[40588]: Manager daemon y is now available 2024-06-11T13:28:50.918 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:28:50.918 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:50.918 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:28:50.918 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-11T13:28:50.918 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-11T13:28:50.918 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-11T13:28:50.918 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:50 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-11T13:28:50.953 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:50 smithi107 ceph-mon[33178]: mgrmap e18: y(active, since 5m), standbys: x 2024-06-11T13:28:50.953 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:50 smithi107 ceph-mon[33178]: Active manager daemon y restarted 2024-06-11T13:28:50.953 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:50 smithi107 ceph-mon[33178]: Activating manager daemon y 2024-06-11T13:28:50.953 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:50 smithi107 ceph-mon[33178]: osdmap e70: 8 total, 8 up, 8 in 2024-06-11T13:28:50.953 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:50 smithi107 ceph-mon[33178]: mgrmap e19: y(active, starting, since 0.0117318s), standbys: x 2024-06-11T13:28:50.953 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:50 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-11T13:28:50.953 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:50 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-11T13:28:50.954 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:50 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-11T13:28:50.954 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:50 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-11T13:28:50.954 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:50 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-11T13:28:50.954 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:50 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-11T13:28:50.954 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:50 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-11T13:28:50.954 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:50 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-11T13:28:50.954 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:50 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-11T13:28:50.954 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:50 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-11T13:28:50.954 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:50 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-11T13:28:50.954 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:50 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-11T13:28:50.954 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:50 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-11T13:28:50.954 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:50 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-11T13:28:50.955 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:50 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-11T13:28:50.955 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:50 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-11T13:28:50.955 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:50 smithi107 ceph-mon[33178]: Manager daemon y is now available 2024-06-11T13:28:50.955 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:50 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:28:50.955 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:50 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:50.955 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:50 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:28:50.955 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:50 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-11T13:28:50.955 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:50 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-11T13:28:50.955 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:50 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-11T13:28:50.955 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:50 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-11T13:28:51.276 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:28:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:28:51.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.303+0000 7f23bebed640 1 -- 172.21.15.57:0/4129992439 >> v1:172.21.15.57:6790/0 conn(0x7f23b810e370 legacy=0x7f23b8110760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:51.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.303+0000 7f23bebed640 1 -- 172.21.15.57:0/4129992439 shutdown_connections 2024-06-11T13:28:51.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.303+0000 7f23bebed640 1 -- 172.21.15.57:0/4129992439 >> 172.21.15.57:0/4129992439 conn(0x7f23b80fc220 msgr2=0x7f23b80fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:28:51.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.303+0000 7f23bebed640 1 -- 172.21.15.57:0/4129992439 shutdown_connections 2024-06-11T13:28:51.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.303+0000 7f23bebed640 1 -- 172.21.15.57:0/4129992439 wait complete. 2024-06-11T13:28:51.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.304+0000 7f23bebed640 1 Processor -- start 2024-06-11T13:28:51.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.304+0000 7f23bebed640 1 -- start start 2024-06-11T13:28:51.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.304+0000 7f23bebed640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f23b81a65f0 con 0x7f23b810a860 2024-06-11T13:28:51.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.304+0000 7f23bebed640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f23b81a7830 con 0x7f23b8102a50 2024-06-11T13:28:51.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.304+0000 7f23bebed640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f23b81a8a70 con 0x7f23b810e370 2024-06-11T13:28:51.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.305+0000 7f23b7fff640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f23b810a860 0x7f23b81a16b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:46018/0 (socket says 172.21.15.57:46018) 2024-06-11T13:28:51.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.305+0000 7f23bc989640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f23b8102a50 0x7f23b8109e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.57:59016/0 (socket says 172.21.15.57:59016) 2024-06-11T13:28:51.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.305+0000 7f23b7fff640 1 -- 172.21.15.57:0/2099364759 learned_addr learned my addr 172.21.15.57:0/2099364759 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:28:51.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.305+0000 7f23bd18a640 1 --1- >> v1:172.21.15.57:6790/0 conn(0x7f23b810e370 0x7f23b81a4de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6790/0 says I am v1:172.21.15.57:45364/0 (socket says 172.21.15.57:45364) 2024-06-11T13:28:51.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.305+0000 7f23b5ffb640 1 -- 172.21.15.57:0/2099364759 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1632489139 0 0) 0x7f23b81a65f0 con 0x7f23b810a860 2024-06-11T13:28:51.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.305+0000 7f23b5ffb640 1 -- 172.21.15.57:0/2099364759 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23a8003160 con 0x7f23b810a860 2024-06-11T13:28:51.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.305+0000 7f23b5ffb640 1 -- 172.21.15.57:0/2099364759 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3400781775 0 0) 0x7f23b81a7830 con 0x7f23b8102a50 2024-06-11T13:28:51.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.305+0000 7f23b5ffb640 1 -- 172.21.15.57:0/2099364759 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23b81a65f0 con 0x7f23b8102a50 2024-06-11T13:28:51.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.305+0000 7f23b5ffb640 1 -- 172.21.15.57:0/2099364759 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1146457543 0 0) 0x7f23b81a8a70 con 0x7f23b810e370 2024-06-11T13:28:51.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.305+0000 7f23b5ffb640 1 -- 172.21.15.57:0/2099364759 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23b81a7830 con 0x7f23b810e370 2024-06-11T13:28:51.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.305+0000 7f23b5ffb640 1 -- 172.21.15.57:0/2099364759 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 896096849 0 0) 0x7f23a8003160 con 0x7f23b810a860 2024-06-11T13:28:51.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.305+0000 7f23b5ffb640 1 -- 172.21.15.57:0/2099364759 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f23b81a8a70 con 0x7f23b810a860 2024-06-11T13:28:51.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.305+0000 7f23b5ffb640 1 -- 172.21.15.57:0/2099364759 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1388496270 0 0) 0x7f23b81a65f0 con 0x7f23b8102a50 2024-06-11T13:28:51.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.305+0000 7f23b5ffb640 1 -- 172.21.15.57:0/2099364759 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f23a8003160 con 0x7f23b8102a50 2024-06-11T13:28:51.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.305+0000 7f23b5ffb640 1 -- 172.21.15.57:0/2099364759 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f23a4004000 con 0x7f23b810a860 2024-06-11T13:28:51.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.305+0000 7f23b5ffb640 1 -- 172.21.15.57:0/2099364759 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2953557405 0 0) 0x7f23b81a7830 con 0x7f23b810e370 2024-06-11T13:28:51.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.305+0000 7f23b5ffb640 1 -- 172.21.15.57:0/2099364759 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f23b81a65f0 con 0x7f23b810e370 2024-06-11T13:28:51.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.305+0000 7f23b5ffb640 1 -- 172.21.15.57:0/2099364759 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f23ac001890 con 0x7f23b8102a50 2024-06-11T13:28:51.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.305+0000 7f23b5ffb640 1 -- 172.21.15.57:0/2099364759 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f23b0001c90 con 0x7f23b810e370 2024-06-11T13:28:51.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.305+0000 7f23b5ffb640 1 -- 172.21.15.57:0/2099364759 <== mon.1 v1:172.21.15.107:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1700360244 0 0) 0x7f23a8003160 con 0x7f23b8102a50 2024-06-11T13:28:51.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.305+0000 7f23b5ffb640 1 -- 172.21.15.57:0/2099364759 >> v1:172.21.15.57:6790/0 conn(0x7f23b810e370 legacy=0x7f23b81a4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:51.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.305+0000 7f23b5ffb640 1 -- 172.21.15.57:0/2099364759 >> v1:172.21.15.57:6789/0 conn(0x7f23b810a860 legacy=0x7f23b81a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:51.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.305+0000 7f23b5ffb640 1 -- 172.21.15.57:0/2099364759 --> v1:172.21.15.107:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23b81a9cb0 con 0x7f23b8102a50 2024-06-11T13:28:51.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.305+0000 7f23bebed640 1 -- 172.21.15.57:0/2099364759 --> v1:172.21.15.107:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f23b81a7a60 con 0x7f23b8102a50 2024-06-11T13:28:51.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.306+0000 7f23b5ffb640 1 -- 172.21.15.57:0/2099364759 <== mon.1 v1:172.21.15.107:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f23ac003a80 con 0x7f23b8102a50 2024-06-11T13:28:51.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.306+0000 7f23b5ffb640 1 -- 172.21.15.57:0/2099364759 <== mon.1 v1:172.21.15.107:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f23ac005a80 con 0x7f23b8102a50 2024-06-11T13:28:51.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.306+0000 7f23bebed640 1 -- 172.21.15.57:0/2099364759 --> v1:172.21.15.107:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f23b81a7f20 con 0x7f23b8102a50 2024-06-11T13:28:51.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.307+0000 7f23bebed640 1 -- 172.21.15.57:0/2099364759 --> v1:172.21.15.107:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23b8069c30 con 0x7f23b8102a50 2024-06-11T13:28:51.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.307+0000 7f23b5ffb640 1 -- 172.21.15.57:0/2099364759 <== mon.1 v1:172.21.15.107:6789/0 7 ==== mgrmap(e 20) ==== 91890+0+0 (unknown 436626012 0 0) 0x7f23ac01c3a0 con 0x7f23b8102a50 2024-06-11T13:28:51.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.308+0000 7f23b5ffb640 1 -- 172.21.15.57:0/2099364759 <== mon.1 v1:172.21.15.107:6789/0 8 ==== osd_map(70..70 src has 1..70) ==== 6172+0+0 (unknown 3813449920 0 0) 0x7f23ac088e10 con 0x7f23b8102a50 2024-06-11T13:28:51.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.312+0000 7f23b5ffb640 1 -- 172.21.15.57:0/2099364759 <== mon.1 v1:172.21.15.107:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f23ac05edf0 con 0x7f23b8102a50 2024-06-11T13:28:51.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.504+0000 7f23bebed640 1 -- 172.21.15.57:0/2099364759 --> v1:172.21.15.107:6789/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f23b8068390 con 0x7f23b8102a50 2024-06-11T13:28:51.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.506+0000 7f23b5ffb640 1 -- 172.21.15.57:0/2099364759 <== mon.1 v1:172.21.15.107:6789/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+20892 (unknown 2276865964 0 2557683168) 0x7f23b8068390 con 0x7f23b8102a50 2024-06-11T13:28:51.502 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:28:51.502 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":70,"fsid":"98a6635e-27f5-11ef-bc9b-c7b262605968","created":"2024-06-11T13:22:52.358653+0000","modified":"2024-06-11T13:28:49.936249+0000","last_up_change":"2024-06-11T13:27:57.083673+0000","last_in_change":"2024-06-11T13:27:39.099229+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-11T13:25:58.933362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-06-11T13:28:11.715547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"datapool","create_time":"2024-06-11T13:28:12.149296+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":65,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-06-11T13:28:13.705298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-06-11T13:28:15.872269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-06-11T13:28:17.956597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b167b692-f22f-4806-ad75-e94e0ed1e57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6801","nonce":978735223}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6802","nonce":978735223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6804","nonce":978735223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6803","nonce":978735223}]},"public_addr":"172.21.15.57:6801/978735223","cluster_addr":"172.21.15.57:6802/978735223","heartbeat_back_addr":"172.21.15.57:6804/978735223","heartbeat_front_addr":"172.21.15.57:6803/978735223","state":["exists","up"]},{"osd":1,"uuid":"3e153beb-619d-47ba-ac9f-12d2fbc69b1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6805","nonce":137889538}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6806","nonce":137889538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6808","nonce":137889538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6807","nonce":137889538}]},"public_addr":"172.21.15.57:6805/137889538","cluster_addr":"172.21.15.57:6806/137889538","heartbeat_back_addr":"172.21.15.57:6808/137889538","heartbeat_front_addr":"172.21.15.57:6807/137889538","state":["exists","up"]},{"osd":2,"uuid":"0083f7a4-f96d-4efe-a2e9-4c9a4a66c0fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6809","nonce":385321295}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6810","nonce":385321295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6812","nonce":385321295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6811","nonce":385321295}]},"public_addr":"172.21.15.57:6809/385321295","cluster_addr":"172.21.15.57:6810/385321295","heartbeat_back_addr":"172.21.15.57:6812/385321295","heartbeat_front_addr":"172.21.15.57:6811/385321295","state":["exists","up"]},{"osd":3,"uuid":"21aeff39-ee19-45ed-bb85-51827aa0a89e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6813","nonce":1307945371}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6814","nonce":1307945371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6816","nonce":1307945371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6815","nonce":1307945371}]},"public_addr":"172.21.15.57:6813/1307945371","cluster_addr":"172.21.15.57:6814/1307945371","heartbeat_back_addr":"172.21.15.57:6816/1307945371","heartbeat_front_addr":"172.21.15.57:6815/1307945371","state":["exists","up"]},{"osd":4,"uuid":"efdccdfc-63d9-42f1-968e-0c411294b675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6800","nonce":2109046293}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6801","nonce":2109046293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6803","nonce":2109046293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6802","nonce":2109046293}]},"public_addr":"172.21.15.107:6800/2109046293","cluster_addr":"172.21.15.107:6801/2109046293","heartbeat_back_addr":"172.21.15.107:6803/2109046293","heartbeat_front_addr":"172.21.15.107:6802/2109046293","state":["exists","up"]},{"osd":5,"uuid":"beac608b-6839-4291-b866-69f354c0fc89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6804","nonce":2372371519}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6805","nonce":2372371519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6807","nonce":2372371519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6806","nonce":2372371519}]},"public_addr":"172.21.15.107:6804/2372371519","cluster_addr":"172.21.15.107:6805/2372371519","heartbeat_back_addr":"172.21.15.107:6807/2372371519","heartbeat_front_addr":"172.21.15.107:6806/2372371519","state":["exists","up"]},{"osd":6,"uuid":"d4f8d2d5-dabb-42ad-baea-cd5de8269914","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6808","nonce":1305047914}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6809","nonce":1305047914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6811","nonce":1305047914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6810","nonce":1305047914}]},"public_addr":"172.21.15.107:6808/1305047914","cluster_addr":"172.21.15.107:6809/1305047914","heartbeat_back_addr":"172.21.15.107:6811/1305047914","heartbeat_front_addr":"172.21.15.107:6810/1305047914","state":["exists","up"]},{"osd":7,"uuid":"57952071-ea76-4ec1-9462-a78264b11913","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6812","nonce":3207390156}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6813","nonce":3207390156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6815","nonce":3207390156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6814","nonce":3207390156}]},"public_addr":"172.21.15.107:6812/3207390156","cluster_addr":"172.21.15.107:6813/3207390156","heartbeat_back_addr":"172.21.15.107:6815/3207390156","heartbeat_front_addr":"172.21.15.107:6814/3207390156","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-11T13:25:06.508704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-11T13:25:31.399208+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-11T13:25:56.463834+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-11T13:26:21.380185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-11T13:26:42.749674+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-11T13:27:06.043909+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-11T13:27:29.706032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-11T13:27:54.608124+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/461546222":"2024-06-12T13:28:49.936204+0000","172.21.15.57:0/380379402":"2024-06-12T13:23:20.613278+0000","172.21.15.57:0/3067099028":"2024-06-12T13:23:20.613278+0000","172.21.15.57:6800/1064738157":"2024-06-12T13:23:46.423712+0000","172.21.15.57:0/482760167":"2024-06-12T13:23:46.423712+0000","172.21.15.57:0/2465286537":"2024-06-12T13:28:49.936204+0000","172.21.15.57:6800/262061228":"2024-06-12T13:23:20.613278+0000","172.21.15.57:0/1037722376":"2024-06-12T13:23:46.423712+0000","172.21.15.57:6800/2989665238":"2024-06-12T13:28:49.936204+0000","172.21.15.57:0/602127561":"2024-06-12T13:23:20.613278+0000","172.21.15.57:0/1586679112":"2024-06-12T13:28:49.936204+0000","172.21.15.57:0/4123200049":"2024-06-12T13:23:46.423712+0000","172.21.15.57:0/534678380":"2024-06-12T13:28:49.936204+0000","172.21.15.57:0/2279315136":"2024-06-12T13:28:49.936204+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-11T13:28:51.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.509+0000 7f23bebed640 1 -- 172.21.15.57:0/2099364759 >> v1:172.21.15.57:6800/2077416415 conn(0x7f23a806ec20 legacy=0x7f23a80710e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:51.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.509+0000 7f23bebed640 1 -- 172.21.15.57:0/2099364759 >> v1:172.21.15.107:6789/0 conn(0x7f23b8102a50 legacy=0x7f23b8109e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:51.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.509+0000 7f23bebed640 1 -- 172.21.15.57:0/2099364759 shutdown_connections 2024-06-11T13:28:51.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.509+0000 7f23bebed640 1 -- 172.21.15.57:0/2099364759 >> 172.21.15.57:0/2099364759 conn(0x7f23b80fc220 msgr2=0x7f23b80fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:28:51.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.509+0000 7f23bebed640 1 -- 172.21.15.57:0/2099364759 shutdown_connections 2024-06-11T13:28:51.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:51.509+0000 7f23bebed640 1 -- 172.21.15.57:0/2099364759 wait complete. 2024-06-11T13:28:52.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:51 smithi057 ceph-mon[40588]: mgrmap e20: y(active, since 1.01403s), standbys: x 2024-06-11T13:28:52.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:51 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:28:52.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:51 smithi057 ceph-mon[40588]: pgmap v3: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:28:52.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:51 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2099364759' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-11T13:28:52.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:51 smithi057 ceph-mon[29805]: mgrmap e20: y(active, since 1.01403s), standbys: x 2024-06-11T13:28:52.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:51 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:28:52.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:51 smithi057 ceph-mon[29805]: pgmap v3: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:28:52.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:51 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2099364759' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-11T13:28:52.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:51 smithi107 ceph-mon[33178]: mgrmap e20: y(active, since 1.01403s), standbys: x 2024-06-11T13:28:52.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:51 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:28:52.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:51 smithi107 ceph-mon[33178]: pgmap v3: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:28:52.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:51 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2099364759' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-11T13:28:52.460 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-06-11T13:28:52.460 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-11T13:28:52.783 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:52 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:28:52.557+0000 7feb48b39640 -1 log_channel(cephadm) log [ERR] : [11/Jun/2024:13:28:52] ENGINE Error in HTTPServer.serve 2024-06-11T13:28:52.784 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:52 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: Traceback (most recent call last): 2024-06-11T13:28:52.784 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:52 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-06-11T13:28:52.784 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:52 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: self._connections.run(self.expiration_interval) 2024-06-11T13:28:52.784 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:52 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-06-11T13:28:52.784 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:52 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: self._run(expiration_interval) 2024-06-11T13:28:52.784 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:52 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-06-11T13:28:52.784 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:52 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: new_conn = self._from_server_socket(self.server.socket) 2024-06-11T13:28:52.784 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:52 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-06-11T13:28:52.784 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:52 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-06-11T13:28:52.784 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:52 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-06-11T13:28:52.784 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:52 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: s = self.context.wrap_socket( 2024-06-11T13:28:52.784 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:52 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-06-11T13:28:52.785 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:52 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: return self.sslsocket_class._create( 2024-06-11T13:28:52.785 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:52 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-06-11T13:28:52.785 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:52 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: self.do_handshake() 2024-06-11T13:28:52.785 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:52 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-06-11T13:28:52.785 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:52 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: self._sslobj.do_handshake() 2024-06-11T13:28:52.785 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:52 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1129) 2024-06-11T13:28:52.785 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:52 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T13:28:52.844 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.a/config 2024-06-11T13:28:53.278 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:52 smithi057 ceph-mon[40588]: mgrmap e21: y(active, since 2s), standbys: x 2024-06-11T13:28:53.279 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:52 smithi057 ceph-mon[40588]: [11/Jun/2024:13:28:52] ENGINE Bus STARTING 2024-06-11T13:28:53.279 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:52 smithi057 ceph-mon[40588]: [11/Jun/2024:13:28:52] ENGINE Serving on http://172.21.15.57:8765 2024-06-11T13:28:53.279 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:52 smithi057 ceph-mon[40588]: [11/Jun/2024:13:28:52] ENGINE Serving on https://172.21.15.57:7150 2024-06-11T13:28:53.279 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:52 smithi057 ceph-mon[40588]: [11/Jun/2024:13:28:52] ENGINE Bus STARTED 2024-06-11T13:28:53.279 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:52 smithi057 ceph-mon[40588]: [11/Jun/2024:13:28:52] ENGINE Error in HTTPServer.serve 2024-06-11T13:28:53.279 INFO:journalctl@ceph.mon.c.smithi057.stdout: Traceback (most recent call last): 2024-06-11T13:28:53.279 INFO:journalctl@ceph.mon.c.smithi057.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-06-11T13:28:53.279 INFO:journalctl@ceph.mon.c.smithi057.stdout: self._connections.run(self.expiration_interval) 2024-06-11T13:28:53.279 INFO:journalctl@ceph.mon.c.smithi057.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-06-11T13:28:53.279 INFO:journalctl@ceph.mon.c.smithi057.stdout: self._run(expiration_interval) 2024-06-11T13:28:53.280 INFO:journalctl@ceph.mon.c.smithi057.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-06-11T13:28:53.280 INFO:journalctl@ceph.mon.c.smithi057.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-06-11T13:28:53.280 INFO:journalctl@ceph.mon.c.smithi057.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-06-11T13:28:53.280 INFO:journalctl@ceph.mon.c.smithi057.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-06-11T13:28:53.280 INFO:journalctl@ceph.mon.c.smithi057.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-06-11T13:28:53.280 INFO:journalctl@ceph.mon.c.smithi057.stdout: s = self.context.wrap_socket( 2024-06-11T13:28:53.280 INFO:journalctl@ceph.mon.c.smithi057.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-06-11T13:28:53.280 INFO:journalctl@ceph.mon.c.smithi057.stdout: return self.sslsocket_class._create( 2024-06-11T13:28:53.280 INFO:journalctl@ceph.mon.c.smithi057.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-06-11T13:28:53.280 INFO:journalctl@ceph.mon.c.smithi057.stdout: self.do_handshake() 2024-06-11T13:28:53.281 INFO:journalctl@ceph.mon.c.smithi057.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-06-11T13:28:53.281 INFO:journalctl@ceph.mon.c.smithi057.stdout: self._sslobj.do_handshake() 2024-06-11T13:28:53.281 INFO:journalctl@ceph.mon.c.smithi057.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1129) 2024-06-11T13:28:53.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:52 smithi057 ceph-mon[29805]: mgrmap e21: y(active, since 2s), standbys: x 2024-06-11T13:28:53.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:52 smithi057 ceph-mon[29805]: [11/Jun/2024:13:28:52] ENGINE Bus STARTING 2024-06-11T13:28:53.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:52 smithi057 ceph-mon[29805]: [11/Jun/2024:13:28:52] ENGINE Serving on http://172.21.15.57:8765 2024-06-11T13:28:53.281 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:52 smithi057 ceph-mon[29805]: [11/Jun/2024:13:28:52] ENGINE Serving on https://172.21.15.57:7150 2024-06-11T13:28:53.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:52 smithi057 ceph-mon[29805]: [11/Jun/2024:13:28:52] ENGINE Bus STARTED 2024-06-11T13:28:53.282 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:52 smithi057 ceph-mon[29805]: [11/Jun/2024:13:28:52] ENGINE Error in HTTPServer.serve 2024-06-11T13:28:53.282 INFO:journalctl@ceph.mon.a.smithi057.stdout: Traceback (most recent call last): 2024-06-11T13:28:53.282 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-06-11T13:28:53.282 INFO:journalctl@ceph.mon.a.smithi057.stdout: self._connections.run(self.expiration_interval) 2024-06-11T13:28:53.282 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-06-11T13:28:53.282 INFO:journalctl@ceph.mon.a.smithi057.stdout: self._run(expiration_interval) 2024-06-11T13:28:53.282 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-06-11T13:28:53.282 INFO:journalctl@ceph.mon.a.smithi057.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-06-11T13:28:53.282 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-06-11T13:28:53.282 INFO:journalctl@ceph.mon.a.smithi057.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-06-11T13:28:53.282 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-06-11T13:28:53.282 INFO:journalctl@ceph.mon.a.smithi057.stdout: s = self.context.wrap_socket( 2024-06-11T13:28:53.283 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-06-11T13:28:53.283 INFO:journalctl@ceph.mon.a.smithi057.stdout: return self.sslsocket_class._create( 2024-06-11T13:28:53.283 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-06-11T13:28:53.283 INFO:journalctl@ceph.mon.a.smithi057.stdout: self.do_handshake() 2024-06-11T13:28:53.283 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-06-11T13:28:53.283 INFO:journalctl@ceph.mon.a.smithi057.stdout: self._sslobj.do_handshake() 2024-06-11T13:28:53.283 INFO:journalctl@ceph.mon.a.smithi057.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1129) 2024-06-11T13:28:53.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:52 smithi107 ceph-mon[33178]: mgrmap e21: y(active, since 2s), standbys: x 2024-06-11T13:28:53.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:52 smithi107 ceph-mon[33178]: [11/Jun/2024:13:28:52] ENGINE Bus STARTING 2024-06-11T13:28:53.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:52 smithi107 ceph-mon[33178]: [11/Jun/2024:13:28:52] ENGINE Serving on http://172.21.15.57:8765 2024-06-11T13:28:53.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:52 smithi107 ceph-mon[33178]: [11/Jun/2024:13:28:52] ENGINE Serving on https://172.21.15.57:7150 2024-06-11T13:28:53.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:52 smithi107 ceph-mon[33178]: [11/Jun/2024:13:28:52] ENGINE Bus STARTED 2024-06-11T13:28:53.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:52 smithi107 ceph-mon[33178]: [11/Jun/2024:13:28:52] ENGINE Error in HTTPServer.serve 2024-06-11T13:28:53.297 INFO:journalctl@ceph.mon.b.smithi107.stdout: Traceback (most recent call last): 2024-06-11T13:28:53.297 INFO:journalctl@ceph.mon.b.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-06-11T13:28:53.297 INFO:journalctl@ceph.mon.b.smithi107.stdout: self._connections.run(self.expiration_interval) 2024-06-11T13:28:53.297 INFO:journalctl@ceph.mon.b.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-06-11T13:28:53.297 INFO:journalctl@ceph.mon.b.smithi107.stdout: self._run(expiration_interval) 2024-06-11T13:28:53.297 INFO:journalctl@ceph.mon.b.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-06-11T13:28:53.297 INFO:journalctl@ceph.mon.b.smithi107.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-06-11T13:28:53.297 INFO:journalctl@ceph.mon.b.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-06-11T13:28:53.297 INFO:journalctl@ceph.mon.b.smithi107.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-06-11T13:28:53.297 INFO:journalctl@ceph.mon.b.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-06-11T13:28:53.297 INFO:journalctl@ceph.mon.b.smithi107.stdout: s = self.context.wrap_socket( 2024-06-11T13:28:53.297 INFO:journalctl@ceph.mon.b.smithi107.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-06-11T13:28:53.297 INFO:journalctl@ceph.mon.b.smithi107.stdout: return self.sslsocket_class._create( 2024-06-11T13:28:53.298 INFO:journalctl@ceph.mon.b.smithi107.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-06-11T13:28:53.298 INFO:journalctl@ceph.mon.b.smithi107.stdout: self.do_handshake() 2024-06-11T13:28:53.298 INFO:journalctl@ceph.mon.b.smithi107.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-06-11T13:28:53.298 INFO:journalctl@ceph.mon.b.smithi107.stdout: self._sslobj.do_handshake() 2024-06-11T13:28:53.298 INFO:journalctl@ceph.mon.b.smithi107.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1129) 2024-06-11T13:28:54.030 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:28:53 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:28:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:28:54.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:54 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:54.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:54 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:54.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:54 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:54.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:54 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:54.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:54 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:54.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:54 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:54.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:54 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:54.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:54 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:54.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:54 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:54.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:54 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:54.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:54 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:54.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:54 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:54.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:54 smithi057 ceph-mon[40588]: Adjusting osd_memory_target on smithi107 to 3760M 2024-06-11T13:28:54.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:54 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:54.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:54 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:54.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:54 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:54.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:54 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:54.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:54 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:54.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:54 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:54.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:54 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:54.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:54 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:54.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:54 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:54.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:54 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:54.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:54 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:54.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:54 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:54.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:54 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:54.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:54 smithi057 ceph-mon[29805]: Adjusting osd_memory_target on smithi107 to 3760M 2024-06-11T13:28:54.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:54 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:54.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:54.511+0000 7fa2a1210640 1 -- 172.21.15.57:0/1424314757 >> v1:172.21.15.57:6789/0 conn(0x7fa29c1066f0 legacy=0x7fa29c106ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:54.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:54.511+0000 7fa2a1210640 1 -- 172.21.15.57:0/1424314757 shutdown_connections 2024-06-11T13:28:54.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:54.511+0000 7fa2a1210640 1 -- 172.21.15.57:0/1424314757 >> 172.21.15.57:0/1424314757 conn(0x7fa29c0fc220 msgr2=0x7fa29c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:28:54.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:54.511+0000 7fa2a1210640 1 -- 172.21.15.57:0/1424314757 shutdown_connections 2024-06-11T13:28:54.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:54.511+0000 7fa2a1210640 1 -- 172.21.15.57:0/1424314757 wait complete. 2024-06-11T13:28:54.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:54.512+0000 7fa2a1210640 1 Processor -- start 2024-06-11T13:28:54.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:54.512+0000 7fa2a1210640 1 -- start start 2024-06-11T13:28:54.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:54.512+0000 7fa2a1210640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fa29c1a5770 con 0x7fa29c1095a0 2024-06-11T13:28:54.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:54.512+0000 7fa2a1210640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fa29c1a69b0 con 0x7fa29c10d140 2024-06-11T13:28:54.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:54.512+0000 7fa2a1210640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fa29c1a7bf0 con 0x7fa29c1066f0 2024-06-11T13:28:54.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:54.512+0000 7fa29ad9d640 1 --1- >> v1:172.21.15.57:6790/0 conn(0x7fa29c1066f0 0x7fa29c10c900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6790/0 says I am v1:172.21.15.57:45406/0 (socket says 172.21.15.57:45406) 2024-06-11T13:28:54.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:54.512+0000 7fa29ad9d640 1 -- 172.21.15.57:0/1295597275 learned_addr learned my addr 172.21.15.57:0/1295597275 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:28:54.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:54.512+0000 7fa29a59c640 1 --1- 172.21.15.57:0/1295597275 >> v1:172.21.15.57:6789/0 conn(0x7fa29c1095a0 0x7fa29c1a05b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:46044/0 (socket says 172.21.15.57:46044) 2024-06-11T13:28:54.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:54.513+0000 7fa293fff640 1 -- 172.21.15.57:0/1295597275 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 203645393 0 0) 0x7fa29c1a7bf0 con 0x7fa29c1066f0 2024-06-11T13:28:54.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:54.513+0000 7fa293fff640 1 -- 172.21.15.57:0/1295597275 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa288003520 con 0x7fa29c1066f0 2024-06-11T13:28:54.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:54.513+0000 7fa293fff640 1 -- 172.21.15.57:0/1295597275 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 324824751 0 0) 0x7fa29c1a5770 con 0x7fa29c1095a0 2024-06-11T13:28:54.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:54.513+0000 7fa293fff640 1 -- 172.21.15.57:0/1295597275 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa29c1a7bf0 con 0x7fa29c1095a0 2024-06-11T13:28:54.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:54.513+0000 7fa293fff640 1 -- 172.21.15.57:0/1295597275 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 462787624 0 0) 0x7fa29c1a69b0 con 0x7fa29c10d140 2024-06-11T13:28:54.527 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:54.513+0000 7fa293fff640 1 -- 172.21.15.57:0/1295597275 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa29c1a5770 con 0x7fa29c10d140 2024-06-11T13:28:54.527 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:54.513+0000 7fa293fff640 1 -- 172.21.15.57:0/1295597275 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2363461181 0 0) 0x7fa288003520 con 0x7fa29c1066f0 2024-06-11T13:28:54.527 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:54.513+0000 7fa293fff640 1 -- 172.21.15.57:0/1295597275 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa29c1a69b0 con 0x7fa29c1066f0 2024-06-11T13:28:54.527 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:54.513+0000 7fa293fff640 1 -- 172.21.15.57:0/1295597275 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fa2940045b0 con 0x7fa29c1066f0 2024-06-11T13:28:54.527 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:54.513+0000 7fa293fff640 1 -- 172.21.15.57:0/1295597275 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2894180116 0 0) 0x7fa29c1a7bf0 con 0x7fa29c1095a0 2024-06-11T13:28:54.527 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:54.513+0000 7fa293fff640 1 -- 172.21.15.57:0/1295597275 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa288003520 con 0x7fa29c1095a0 2024-06-11T13:28:54.527 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:54.513+0000 7fa293fff640 1 -- 172.21.15.57:0/1295597275 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fa2840019b0 con 0x7fa29c1095a0 2024-06-11T13:28:54.527 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:54.513+0000 7fa293fff640 1 -- 172.21.15.57:0/1295597275 <== mon.2 v1:172.21.15.57:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4037461297 0 0) 0x7fa29c1a69b0 con 0x7fa29c1066f0 2024-06-11T13:28:54.527 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:54.513+0000 7fa293fff640 1 -- 172.21.15.57:0/1295597275 >> v1:172.21.15.107:6789/0 conn(0x7fa29c10d140 legacy=0x7fa29c1a3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:54.527 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:54.513+0000 7fa293fff640 1 -- 172.21.15.57:0/1295597275 >> v1:172.21.15.57:6789/0 conn(0x7fa29c1095a0 legacy=0x7fa29c1a05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:54.527 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:54.513+0000 7fa293fff640 1 -- 172.21.15.57:0/1295597275 --> v1:172.21.15.57:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa29c1a8e30 con 0x7fa29c1066f0 2024-06-11T13:28:54.527 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:54.513+0000 7fa2a1210640 1 -- 172.21.15.57:0/1295597275 --> v1:172.21.15.57:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fa29c1a59a0 con 0x7fa29c1066f0 2024-06-11T13:28:54.527 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:54.513+0000 7fa2a1210640 1 -- 172.21.15.57:0/1295597275 --> v1:172.21.15.57:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fa29c1a5e60 con 0x7fa29c1066f0 2024-06-11T13:28:54.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:54.513+0000 7fa293fff640 1 -- 172.21.15.57:0/1295597275 <== mon.2 v1:172.21.15.57:6790/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7fa294001b50 con 0x7fa29c1066f0 2024-06-11T13:28:54.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:54.513+0000 7fa293fff640 1 -- 172.21.15.57:0/1295597275 <== mon.2 v1:172.21.15.57:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fa294004d80 con 0x7fa29c1066f0 2024-06-11T13:28:54.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:54.514+0000 7fa2a1210640 1 -- 172.21.15.57:0/1295597275 --> v1:172.21.15.57:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa27c005080 con 0x7fa29c1066f0 2024-06-11T13:28:54.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:54.515+0000 7fa293fff640 1 -- 172.21.15.57:0/1295597275 <== mon.2 v1:172.21.15.57:6790/0 7 ==== mgrmap(e 21) ==== 92082+0+0 (unknown 3198199867 0 0) 0x7fa2940037d0 con 0x7fa29c1066f0 2024-06-11T13:28:54.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:54.516+0000 7fa293fff640 1 -- 172.21.15.57:0/1295597275 <== mon.2 v1:172.21.15.57:6790/0 8 ==== osd_map(70..70 src has 1..70) ==== 6172+0+0 (unknown 3813449920 0 0) 0x7fa294089220 con 0x7fa29c1066f0 2024-06-11T13:28:54.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:54.520+0000 7fa293fff640 1 -- 172.21.15.57:0/1295597275 <== mon.2 v1:172.21.15.57:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7fa294056ba0 con 0x7fa29c1066f0 2024-06-11T13:28:54.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:54 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:54.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:54 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:54.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:54 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:54.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:54 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:54.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:54 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:54.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:54 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:54.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:54 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:54.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:54 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:54.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:54 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:54.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:54 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:54.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:54 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:54.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:54 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:54.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:54 smithi107 ceph-mon[33178]: Adjusting osd_memory_target on smithi107 to 3760M 2024-06-11T13:28:54.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:54 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:54.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:54.732+0000 7fa2a1210640 1 -- 172.21.15.57:0/1295597275 --> v1:172.21.15.57:6790/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa27c005370 con 0x7fa29c1066f0 2024-06-11T13:28:54.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:54.736+0000 7fa293fff640 1 -- 172.21.15.57:0/1295597275 <== mon.2 v1:172.21.15.57:6790/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+20892 (unknown 2276865964 0 2557683168) 0x7fa29405a850 con 0x7fa29c1066f0 2024-06-11T13:28:54.732 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:28:54.732 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":70,"fsid":"98a6635e-27f5-11ef-bc9b-c7b262605968","created":"2024-06-11T13:22:52.358653+0000","modified":"2024-06-11T13:28:49.936249+0000","last_up_change":"2024-06-11T13:27:57.083673+0000","last_in_change":"2024-06-11T13:27:39.099229+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-11T13:25:58.933362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-06-11T13:28:11.715547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"datapool","create_time":"2024-06-11T13:28:12.149296+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":65,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-06-11T13:28:13.705298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-06-11T13:28:15.872269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-06-11T13:28:17.956597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b167b692-f22f-4806-ad75-e94e0ed1e57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6801","nonce":978735223}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6802","nonce":978735223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6804","nonce":978735223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6803","nonce":978735223}]},"public_addr":"172.21.15.57:6801/978735223","cluster_addr":"172.21.15.57:6802/978735223","heartbeat_back_addr":"172.21.15.57:6804/978735223","heartbeat_front_addr":"172.21.15.57:6803/978735223","state":["exists","up"]},{"osd":1,"uuid":"3e153beb-619d-47ba-ac9f-12d2fbc69b1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6805","nonce":137889538}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6806","nonce":137889538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6808","nonce":137889538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6807","nonce":137889538}]},"public_addr":"172.21.15.57:6805/137889538","cluster_addr":"172.21.15.57:6806/137889538","heartbeat_back_addr":"172.21.15.57:6808/137889538","heartbeat_front_addr":"172.21.15.57:6807/137889538","state":["exists","up"]},{"osd":2,"uuid":"0083f7a4-f96d-4efe-a2e9-4c9a4a66c0fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6809","nonce":385321295}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6810","nonce":385321295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6812","nonce":385321295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6811","nonce":385321295}]},"public_addr":"172.21.15.57:6809/385321295","cluster_addr":"172.21.15.57:6810/385321295","heartbeat_back_addr":"172.21.15.57:6812/385321295","heartbeat_front_addr":"172.21.15.57:6811/385321295","state":["exists","up"]},{"osd":3,"uuid":"21aeff39-ee19-45ed-bb85-51827aa0a89e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6813","nonce":1307945371}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6814","nonce":1307945371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6816","nonce":1307945371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.57:6815","nonce":1307945371}]},"public_addr":"172.21.15.57:6813/1307945371","cluster_addr":"172.21.15.57:6814/1307945371","heartbeat_back_addr":"172.21.15.57:6816/1307945371","heartbeat_front_addr":"172.21.15.57:6815/1307945371","state":["exists","up"]},{"osd":4,"uuid":"efdccdfc-63d9-42f1-968e-0c411294b675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6800","nonce":2109046293}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6801","nonce":2109046293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6803","nonce":2109046293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6802","nonce":2109046293}]},"public_addr":"172.21.15.107:6800/2109046293","cluster_addr":"172.21.15.107:6801/2109046293","heartbeat_back_addr":"172.21.15.107:6803/2109046293","heartbeat_front_addr":"172.21.15.107:6802/2109046293","state":["exists","up"]},{"osd":5,"uuid":"beac608b-6839-4291-b866-69f354c0fc89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6804","nonce":2372371519}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6805","nonce":2372371519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6807","nonce":2372371519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6806","nonce":2372371519}]},"public_addr":"172.21.15.107:6804/2372371519","cluster_addr":"172.21.15.107:6805/2372371519","heartbeat_back_addr":"172.21.15.107:6807/2372371519","heartbeat_front_addr":"172.21.15.107:6806/2372371519","state":["exists","up"]},{"osd":6,"uuid":"d4f8d2d5-dabb-42ad-baea-cd5de8269914","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6808","nonce":1305047914}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6809","nonce":1305047914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6811","nonce":1305047914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6810","nonce":1305047914}]},"public_addr":"172.21.15.107:6808/1305047914","cluster_addr":"172.21.15.107:6809/1305047914","heartbeat_back_addr":"172.21.15.107:6811/1305047914","heartbeat_front_addr":"172.21.15.107:6810/1305047914","state":["exists","up"]},{"osd":7,"uuid":"57952071-ea76-4ec1-9462-a78264b11913","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6812","nonce":3207390156}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6813","nonce":3207390156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6815","nonce":3207390156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6814","nonce":3207390156}]},"public_addr":"172.21.15.107:6812/3207390156","cluster_addr":"172.21.15.107:6813/3207390156","heartbeat_back_addr":"172.21.15.107:6815/3207390156","heartbeat_front_addr":"172.21.15.107:6814/3207390156","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-11T13:25:06.508704+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-11T13:25:31.399208+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-11T13:25:56.463834+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-11T13:26:21.380185+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-11T13:26:42.749674+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-11T13:27:06.043909+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-11T13:27:29.706032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-11T13:27:54.608124+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/461546222":"2024-06-12T13:28:49.936204+0000","172.21.15.57:0/380379402":"2024-06-12T13:23:20.613278+0000","172.21.15.57:0/3067099028":"2024-06-12T13:23:20.613278+0000","172.21.15.57:6800/1064738157":"2024-06-12T13:23:46.423712+0000","172.21.15.57:0/482760167":"2024-06-12T13:23:46.423712+0000","172.21.15.57:0/2465286537":"2024-06-12T13:28:49.936204+0000","172.21.15.57:6800/262061228":"2024-06-12T13:23:20.613278+0000","172.21.15.57:0/1037722376":"2024-06-12T13:23:46.423712+0000","172.21.15.57:6800/2989665238":"2024-06-12T13:28:49.936204+0000","172.21.15.57:0/602127561":"2024-06-12T13:23:20.613278+0000","172.21.15.57:0/1586679112":"2024-06-12T13:28:49.936204+0000","172.21.15.57:0/4123200049":"2024-06-12T13:23:46.423712+0000","172.21.15.57:0/534678380":"2024-06-12T13:28:49.936204+0000","172.21.15.57:0/2279315136":"2024-06-12T13:28:49.936204+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-11T13:28:54.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:54.738+0000 7fa2a1210640 1 -- 172.21.15.57:0/1295597275 >> v1:172.21.15.57:6800/2077416415 conn(0x7fa28806eab0 legacy=0x7fa288070f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:54.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:54.738+0000 7fa2a1210640 1 -- 172.21.15.57:0/1295597275 >> v1:172.21.15.57:6790/0 conn(0x7fa29c1066f0 legacy=0x7fa29c10c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:54.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:54.738+0000 7fa2a1210640 1 -- 172.21.15.57:0/1295597275 shutdown_connections 2024-06-11T13:28:54.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:54.738+0000 7fa2a1210640 1 -- 172.21.15.57:0/1295597275 >> 172.21.15.57:0/1295597275 conn(0x7fa29c0fc220 msgr2=0x7fa29c10fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:28:54.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:54.741+0000 7fa2a1210640 1 -- 172.21.15.57:0/1295597275 shutdown_connections 2024-06-11T13:28:54.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:54.742+0000 7fa2a1210640 1 -- 172.21.15.57:0/1295597275 wait complete. 2024-06-11T13:28:55.551 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-06-11T13:28:55.551 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-06-11T13:28:55.552 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-06-11T13:28:55.552 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-06-11T13:28:55.553 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-06-11T13:28:55.553 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-06-11T13:28:55.553 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-06-11T13:28:55.554 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-06-11T13:28:55.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:55 smithi057 ceph-mon[29805]: pgmap v5: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:28:55.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:55 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:55.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:55 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:55.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:55 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1295597275' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-11T13:28:55.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:55 smithi057 ceph-mon[29805]: mgrmap e22: y(active, since 4s), standbys: x 2024-06-11T13:28:55.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:55 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:55.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:55 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:55.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:55 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:55.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:55 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:55.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:55 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:55.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:55 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:55.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:55 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:55.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:55 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:55.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:55 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:55.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:55 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:55.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:55 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:55.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:55 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:28:55.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:55 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:28:55.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:55 smithi057 ceph-mon[40588]: pgmap v5: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:28:55.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:55 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:55.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:55 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:55.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:55 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1295597275' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-11T13:28:55.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:55 smithi057 ceph-mon[40588]: mgrmap e22: y(active, since 4s), standbys: x 2024-06-11T13:28:55.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:55 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:55.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:55 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:55.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:55 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:55.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:55 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:55.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:55 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:55.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:55 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:55.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:55 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:55.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:55 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:55.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:55 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:55.787 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:55 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:55.787 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:55 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:55.787 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:55 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:28:55.787 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:55 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:28:56.029 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.a/config 2024-06-11T13:28:56.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:55 smithi107 ceph-mon[33178]: pgmap v5: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:28:56.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:55 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:56.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:55 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:56.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:55 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1295597275' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-11T13:28:56.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:55 smithi107 ceph-mon[33178]: mgrmap e22: y(active, since 4s), standbys: x 2024-06-11T13:28:56.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:55 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:56.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:55 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:56.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:55 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:56.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:55 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:56.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:55 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:56.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:55 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:56.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:55 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:56.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:55 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:56.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:55 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:56.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:55 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:28:56.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:55 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:56.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:55 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:28:56.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:55 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:28:56.072 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.a/config 2024-06-11T13:28:56.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:56 smithi057 ceph-mon[29805]: Adjusting osd_memory_target on smithi057 to 3761M 2024-06-11T13:28:56.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:56 smithi057 ceph-mon[29805]: Updating smithi057:/etc/ceph/ceph.conf 2024-06-11T13:28:56.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:56 smithi057 ceph-mon[29805]: Updating smithi107:/etc/ceph/ceph.conf 2024-06-11T13:28:56.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:56 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:56.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:56 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:56.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:56 smithi057 ceph-mon[40588]: Adjusting osd_memory_target on smithi057 to 3761M 2024-06-11T13:28:56.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:56 smithi057 ceph-mon[40588]: Updating smithi057:/etc/ceph/ceph.conf 2024-06-11T13:28:56.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:56 smithi057 ceph-mon[40588]: Updating smithi107:/etc/ceph/ceph.conf 2024-06-11T13:28:56.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:56 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:56.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:56 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:57.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:56 smithi107 ceph-mon[33178]: Adjusting osd_memory_target on smithi057 to 3761M 2024-06-11T13:28:57.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:56 smithi107 ceph-mon[33178]: Updating smithi057:/etc/ceph/ceph.conf 2024-06-11T13:28:57.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:56 smithi107 ceph-mon[33178]: Updating smithi107:/etc/ceph/ceph.conf 2024-06-11T13:28:57.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:56 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:57.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:56 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:57.459 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.a/config 2024-06-11T13:28:57.460 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.a/config 2024-06-11T13:28:57.460 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.a/config 2024-06-11T13:28:57.461 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.a/config 2024-06-11T13:28:57.461 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.a/config 2024-06-11T13:28:57.464 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.a/config 2024-06-11T13:28:58.030 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:57 smithi057 ceph-mon[40588]: Updating smithi107:/var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-11T13:28:58.030 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:57 smithi057 ceph-mon[40588]: pgmap v6: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 42 KiB/s rd, 0 B/s wr, 24 op/s 2024-06-11T13:28:58.030 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:57 smithi057 ceph-mon[40588]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-06-11T13:28:58.030 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:57 smithi057 ceph-mon[40588]: Updating smithi057:/var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-11T13:28:58.031 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:57 smithi057 ceph-mon[40588]: Updating smithi107:/var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-11T13:28:58.031 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:57 smithi057 ceph-mon[40588]: Updating smithi057:/etc/ceph/ceph.client.admin.keyring 2024-06-11T13:28:58.031 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:57 smithi057 ceph-mon[40588]: Updating smithi057:/var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-11T13:28:58.031 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:57 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:58.031 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:57 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:58.031 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:57 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:58.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:57 smithi057 ceph-mon[29805]: Updating smithi107:/var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-11T13:28:58.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:57 smithi057 ceph-mon[29805]: pgmap v6: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 42 KiB/s rd, 0 B/s wr, 24 op/s 2024-06-11T13:28:58.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:57 smithi057 ceph-mon[29805]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-06-11T13:28:58.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:57 smithi057 ceph-mon[29805]: Updating smithi057:/var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-11T13:28:58.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:57 smithi057 ceph-mon[29805]: Updating smithi107:/var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-11T13:28:58.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:57 smithi057 ceph-mon[29805]: Updating smithi057:/etc/ceph/ceph.client.admin.keyring 2024-06-11T13:28:58.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:57 smithi057 ceph-mon[29805]: Updating smithi057:/var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-11T13:28:58.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:57 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:58.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:57 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:58.032 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:57 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:58.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:57 smithi107 ceph-mon[33178]: Updating smithi107:/var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-11T13:28:58.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:57 smithi107 ceph-mon[33178]: pgmap v6: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 42 KiB/s rd, 0 B/s wr, 24 op/s 2024-06-11T13:28:58.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:57 smithi107 ceph-mon[33178]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-06-11T13:28:58.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:57 smithi107 ceph-mon[33178]: Updating smithi057:/var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-11T13:28:58.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:57 smithi107 ceph-mon[33178]: Updating smithi107:/var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-11T13:28:58.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:57 smithi107 ceph-mon[33178]: Updating smithi057:/etc/ceph/ceph.client.admin.keyring 2024-06-11T13:28:58.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:57 smithi107 ceph-mon[33178]: Updating smithi057:/var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-11T13:28:58.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:57 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:58.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:57 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:58.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:57 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:28:58.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.119+0000 7f69f63e3640 1 -- 172.21.15.57:0/3773168335 >> v1:172.21.15.57:6789/0 conn(0x7f69f010c8e0 legacy=0x7f69f010ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:58.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.119+0000 7f69f63e3640 1 -- 172.21.15.57:0/3773168335 shutdown_connections 2024-06-11T13:28:58.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.119+0000 7f69f63e3640 1 -- 172.21.15.57:0/3773168335 >> 172.21.15.57:0/3773168335 conn(0x7f69f00ff4e0 msgr2=0x7f69f0101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:28:58.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.119+0000 7f69f63e3640 1 -- 172.21.15.57:0/3773168335 shutdown_connections 2024-06-11T13:28:58.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.119+0000 7f69f63e3640 1 -- 172.21.15.57:0/3773168335 wait complete. 2024-06-11T13:28:58.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.120+0000 7f69f63e3640 1 Processor -- start 2024-06-11T13:28:58.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.120+0000 7f69f63e3640 1 -- start start 2024-06-11T13:28:58.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.120+0000 7f69f63e3640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f69f01a8ab0 con 0x7f69f0109a30 2024-06-11T13:28:58.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.120+0000 7f69f63e3640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f69f01a9cf0 con 0x7f69f0110480 2024-06-11T13:28:58.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.120+0000 7f69f63e3640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f69f01aaf30 con 0x7f69f010c8e0 2024-06-11T13:28:58.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.120+0000 7f69effff640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f69f0109a30 0x7f69f010fcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:46060/0 (socket says 172.21.15.57:46060) 2024-06-11T13:28:58.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.120+0000 7f69effff640 1 -- 172.21.15.57:0/851494882 learned_addr learned my addr 172.21.15.57:0/851494882 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:28:58.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.121+0000 7f69edffb640 1 -- 172.21.15.57:0/851494882 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2488514631 0 0) 0x7f69f01aaf30 con 0x7f69f010c8e0 2024-06-11T13:28:58.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.121+0000 7f69edffb640 1 -- 172.21.15.57:0/851494882 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f69dc003520 con 0x7f69f010c8e0 2024-06-11T13:28:58.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.121+0000 7f69edffb640 1 -- 172.21.15.57:0/851494882 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1638590455 0 0) 0x7f69f01a8ab0 con 0x7f69f0109a30 2024-06-11T13:28:58.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.121+0000 7f69edffb640 1 -- 172.21.15.57:0/851494882 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f69f01aaf30 con 0x7f69f0109a30 2024-06-11T13:28:58.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.121+0000 7f69edffb640 1 -- 172.21.15.57:0/851494882 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2548550645 0 0) 0x7f69f01a9cf0 con 0x7f69f0110480 2024-06-11T13:28:58.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.121+0000 7f69edffb640 1 -- 172.21.15.57:0/851494882 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f69f01a8ab0 con 0x7f69f0110480 2024-06-11T13:28:58.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.121+0000 7f69edffb640 1 -- 172.21.15.57:0/851494882 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1638191172 0 0) 0x7f69dc003520 con 0x7f69f010c8e0 2024-06-11T13:28:58.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.121+0000 7f69edffb640 1 -- 172.21.15.57:0/851494882 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f69f01a9cf0 con 0x7f69f010c8e0 2024-06-11T13:28:58.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.121+0000 7f69edffb640 1 -- 172.21.15.57:0/851494882 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f69d80043d0 con 0x7f69f010c8e0 2024-06-11T13:28:58.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.121+0000 7f69edffb640 1 -- 172.21.15.57:0/851494882 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2079841061 0 0) 0x7f69f01aaf30 con 0x7f69f0109a30 2024-06-11T13:28:58.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.121+0000 7f69edffb640 1 -- 172.21.15.57:0/851494882 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f69dc003520 con 0x7f69f0109a30 2024-06-11T13:28:58.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.121+0000 7f69edffb640 1 -- 172.21.15.57:0/851494882 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f69e00018b0 con 0x7f69f0109a30 2024-06-11T13:28:58.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.121+0000 7f69edffb640 1 -- 172.21.15.57:0/851494882 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2991911110 0 0) 0x7f69f01a8ab0 con 0x7f69f0110480 2024-06-11T13:28:58.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.121+0000 7f69edffb640 1 -- 172.21.15.57:0/851494882 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f69f01aaf30 con 0x7f69f0110480 2024-06-11T13:28:58.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.121+0000 7f69edffb640 1 -- 172.21.15.57:0/851494882 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f69e8001470 con 0x7f69f0110480 2024-06-11T13:28:58.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.121+0000 7f69edffb640 1 -- 172.21.15.57:0/851494882 <== mon.2 v1:172.21.15.57:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1790928775 0 0) 0x7f69f01a9cf0 con 0x7f69f010c8e0 2024-06-11T13:28:58.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.121+0000 7f69edffb640 1 -- 172.21.15.57:0/851494882 >> v1:172.21.15.107:6789/0 conn(0x7f69f0110480 legacy=0x7f69f01a71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:58.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.121+0000 7f69edffb640 1 -- 172.21.15.57:0/851494882 >> v1:172.21.15.57:6789/0 conn(0x7f69f0109a30 legacy=0x7f69f010fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:58.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.121+0000 7f69edffb640 1 -- 172.21.15.57:0/851494882 --> v1:172.21.15.57:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69f01ac170 con 0x7f69f010c8e0 2024-06-11T13:28:58.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.121+0000 7f69f63e3640 1 -- 172.21.15.57:0/851494882 --> v1:172.21.15.57:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f69f01a8ce0 con 0x7f69f010c8e0 2024-06-11T13:28:58.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.121+0000 7f69f63e3640 1 -- 172.21.15.57:0/851494882 --> v1:172.21.15.57:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f69f01a91a0 con 0x7f69f010c8e0 2024-06-11T13:28:58.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.121+0000 7f69edffb640 1 -- 172.21.15.57:0/851494882 <== mon.2 v1:172.21.15.57:6790/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f69d8001af0 con 0x7f69f010c8e0 2024-06-11T13:28:58.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.121+0000 7f69edffb640 1 -- 172.21.15.57:0/851494882 <== mon.2 v1:172.21.15.57:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f69d8004d30 con 0x7f69f010c8e0 2024-06-11T13:28:58.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.122+0000 7f69f63e3640 1 -- 172.21.15.57:0/851494882 --> v1:172.21.15.57:6790/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f69d0000f80 con 0x7f69f010c8e0 2024-06-11T13:28:58.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.123+0000 7f69edffb640 1 -- 172.21.15.57:0/851494882 <== mon.2 v1:172.21.15.57:6790/0 7 ==== mgrmap(e 22) ==== 92138+0+0 (unknown 311317106 0 0) 0x7f69d80037d0 con 0x7f69f010c8e0 2024-06-11T13:28:58.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.123+0000 7f69edffb640 1 -- 172.21.15.57:0/851494882 <== mon.2 v1:172.21.15.57:6790/0 8 ==== osd_map(70..70 src has 1..70) ==== 6172+0+0 (unknown 3813449920 0 0) 0x7f69d8088470 con 0x7f69f010c8e0 2024-06-11T13:28:58.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.123+0000 7f69edffb640 1 -- 172.21.15.57:0/851494882 --> v1:172.21.15.57:6805/137889538 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f69d8001080 con 0x7f69dc079630 2024-06-11T13:28:58.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.124+0000 7f69edffb640 1 -- 172.21.15.57:0/851494882 <== mon.2 v1:172.21.15.57:6790/0 9 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (unknown 2350636211 0 0) 0x7f69d8088940 con 0x7f69f010c8e0 2024-06-11T13:28:58.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.124+0000 7f69edffb640 1 -- 172.21.15.57:0/851494882 <== osd.1 v1:172.21.15.57:6805/137889538 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f69d8001080 con 0x7f69dc079630 2024-06-11T13:28:58.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.146+0000 7f69f63e3640 1 -- 172.21.15.57:0/851494882 --> v1:172.21.15.57:6805/137889538 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f69d0002c50 con 0x7f69dc079630 2024-06-11T13:28:58.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.146+0000 7f69edffb640 1 -- 172.21.15.57:0/851494882 <== osd.1 v1:172.21.15.57:6805/137889538 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 3910016750) 0x7f69d0002c50 con 0x7f69dc079630 2024-06-11T13:28:58.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.147+0000 7f69f63e3640 1 -- 172.21.15.57:0/851494882 >> v1:172.21.15.57:6805/137889538 conn(0x7f69dc079630 legacy=0x7f69dc07ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:58.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.147+0000 7f69f63e3640 1 -- 172.21.15.57:0/851494882 >> v1:172.21.15.57:6800/2077416415 conn(0x7f69dc06eec0 legacy=0x7f69dc071380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:58.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.147+0000 7f69f63e3640 1 -- 172.21.15.57:0/851494882 >> v1:172.21.15.57:6790/0 conn(0x7f69f010c8e0 legacy=0x7f69f01a38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:28:58.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.147+0000 7f69f4980640 1 -- 172.21.15.57:0/851494882 reap_dead start 2024-06-11T13:28:58.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.147+0000 7f69f63e3640 1 -- 172.21.15.57:0/851494882 shutdown_connections 2024-06-11T13:28:58.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.147+0000 7f69f63e3640 1 -- 172.21.15.57:0/851494882 >> 172.21.15.57:0/851494882 conn(0x7f69f00ff4e0 msgr2=0x7f69f01020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:28:58.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.147+0000 7f69f63e3640 1 -- 172.21.15.57:0/851494882 shutdown_connections 2024-06-11T13:28:58.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:28:58.147+0000 7f69f63e3640 1 -- 172.21.15.57:0/851494882 wait complete. 2024-06-11T13:28:58.876 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:58 smithi057 ceph-mon[40588]: Deploying daemon alertmanager.a on smithi057 2024-06-11T13:28:58.876 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:58 smithi057 ceph-mon[29805]: Deploying daemon alertmanager.a on smithi057 2024-06-11T13:28:59.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:58 smithi107 ceph-mon[33178]: Deploying daemon alertmanager.a on smithi057 2024-06-11T13:29:00.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:28:59 smithi057 ceph-mon[29805]: pgmap v7: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-06-11T13:29:00.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:28:59 smithi057 ceph-mon[40588]: pgmap v7: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-06-11T13:29:00.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:28:59 smithi107 ceph-mon[33178]: pgmap v7: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-06-11T13:29:01.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:00 smithi057 ceph-mon[40588]: pgmap v8: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 0 B/s wr, 13 op/s 2024-06-11T13:29:01.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:00 smithi057 ceph-mon[29805]: pgmap v8: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 0 B/s wr, 13 op/s 2024-06-11T13:29:01.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:00 smithi107 ceph-mon[33178]: pgmap v8: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 0 B/s wr, 13 op/s 2024-06-11T13:29:01.046 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:29:00 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:29:02.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:01 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:29:02.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:01 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:29:02.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:01 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:29:02.327 INFO:teuthology.orchestra.run.smithi057.stdout:55834574891 2024-06-11T13:29:02.328 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-06-11T13:29:03.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:02 smithi107 ceph-mon[33178]: pgmap v9: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-06-11T13:29:03.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:02 smithi057 ceph-mon[29805]: pgmap v9: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-06-11T13:29:03.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:02 smithi057 ceph-mon[40588]: pgmap v9: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-06-11T13:29:04.033 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:29:03 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:29:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:29:05.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:05 smithi057 ceph-mon[29805]: pgmap v10: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-06-11T13:29:05.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:05 smithi057 ceph-mon[40588]: pgmap v10: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-06-11T13:29:05.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:05 smithi107 ceph-mon[33178]: pgmap v10: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-06-11T13:29:06.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:06 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:29:06.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:06 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:29:06.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:06 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:29:06.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.624+0000 7ff184f67640 1 -- 172.21.15.57:0/3707239284 >> v1:172.21.15.57:6790/0 conn(0x7ff1801005a0 legacy=0x7ff180100980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:06.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.624+0000 7ff184f67640 1 -- 172.21.15.57:0/3707239284 shutdown_connections 2024-06-11T13:29:06.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.624+0000 7ff184f67640 1 -- 172.21.15.57:0/3707239284 >> 172.21.15.57:0/3707239284 conn(0x7ff1800fc220 msgr2=0x7ff1800fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:29:06.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.624+0000 7ff184f67640 1 -- 172.21.15.57:0/3707239284 shutdown_connections 2024-06-11T13:29:06.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.624+0000 7ff184f67640 1 -- 172.21.15.57:0/3707239284 wait complete. 2024-06-11T13:29:06.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.625+0000 7ff184f67640 1 Processor -- start 2024-06-11T13:29:06.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.625+0000 7ff184f67640 1 -- start start 2024-06-11T13:29:06.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.625+0000 7ff184f67640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ff1801a8a60 con 0x7ff180101180 2024-06-11T13:29:06.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.625+0000 7ff184f67640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ff1801a9ca0 con 0x7ff180110470 2024-06-11T13:29:06.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.625+0000 7ff184f67640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ff1801aaee0 con 0x7ff1801005a0 2024-06-11T13:29:06.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.625+0000 7ff17e59c640 1 --1- >> v1:172.21.15.57:6790/0 conn(0x7ff1801005a0 0x7ff18010eb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6790/0 says I am v1:172.21.15.57:60424/0 (socket says 172.21.15.57:60424) 2024-06-11T13:29:06.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.625+0000 7ff177fff640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7ff180101180 0x7ff1801a38d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:45766/0 (socket says 172.21.15.57:45766) 2024-06-11T13:29:06.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.625+0000 7ff17e59c640 1 -- 172.21.15.57:0/3222106665 learned_addr learned my addr 172.21.15.57:0/3222106665 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:29:06.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.625+0000 7ff1777fe640 1 -- 172.21.15.57:0/3222106665 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2425309450 0 0) 0x7ff1801a8a60 con 0x7ff180101180 2024-06-11T13:29:06.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.625+0000 7ff1777fe640 1 -- 172.21.15.57:0/3222106665 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff16c003520 con 0x7ff180101180 2024-06-11T13:29:06.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.625+0000 7ff1777fe640 1 -- 172.21.15.57:0/3222106665 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2728276047 0 0) 0x7ff1801aaee0 con 0x7ff1801005a0 2024-06-11T13:29:06.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.625+0000 7ff1777fe640 1 -- 172.21.15.57:0/3222106665 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff1801a8a60 con 0x7ff1801005a0 2024-06-11T13:29:06.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.625+0000 7ff1777fe640 1 -- 172.21.15.57:0/3222106665 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4042445706 0 0) 0x7ff1801a9ca0 con 0x7ff180110470 2024-06-11T13:29:06.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.625+0000 7ff1777fe640 1 -- 172.21.15.57:0/3222106665 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff1801aaee0 con 0x7ff180110470 2024-06-11T13:29:06.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.626+0000 7ff1777fe640 1 -- 172.21.15.57:0/3222106665 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2148997577 0 0) 0x7ff16c003520 con 0x7ff180101180 2024-06-11T13:29:06.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.626+0000 7ff1777fe640 1 -- 172.21.15.57:0/3222106665 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff1801a9ca0 con 0x7ff180101180 2024-06-11T13:29:06.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.626+0000 7ff1777fe640 1 -- 172.21.15.57:0/3222106665 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7ff168001c30 con 0x7ff180101180 2024-06-11T13:29:06.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.626+0000 7ff1777fe640 1 -- 172.21.15.57:0/3222106665 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 415490677 0 0) 0x7ff1801a8a60 con 0x7ff1801005a0 2024-06-11T13:29:06.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.626+0000 7ff1777fe640 1 -- 172.21.15.57:0/3222106665 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff16c003520 con 0x7ff1801005a0 2024-06-11T13:29:06.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.626+0000 7ff1777fe640 1 -- 172.21.15.57:0/3222106665 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7ff170004450 con 0x7ff1801005a0 2024-06-11T13:29:06.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.626+0000 7ff1777fe640 1 -- 172.21.15.57:0/3222106665 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1684045219 0 0) 0x7ff1801aaee0 con 0x7ff180110470 2024-06-11T13:29:06.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.626+0000 7ff1777fe640 1 -- 172.21.15.57:0/3222106665 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff1801a8a60 con 0x7ff180110470 2024-06-11T13:29:06.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.626+0000 7ff1777fe640 1 -- 172.21.15.57:0/3222106665 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3591964613 0 0) 0x7ff1801a9ca0 con 0x7ff180101180 2024-06-11T13:29:06.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.626+0000 7ff1777fe640 1 -- 172.21.15.57:0/3222106665 >> v1:172.21.15.57:6790/0 conn(0x7ff1801005a0 legacy=0x7ff18010eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:06.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.626+0000 7ff1777fe640 1 -- 172.21.15.57:0/3222106665 >> v1:172.21.15.107:6789/0 conn(0x7ff180110470 legacy=0x7ff1801a7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:06.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.626+0000 7ff1777fe640 1 -- 172.21.15.57:0/3222106665 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1801ac120 con 0x7ff180101180 2024-06-11T13:29:06.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.626+0000 7ff184f67640 1 -- 172.21.15.57:0/3222106665 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7ff1801a8c90 con 0x7ff180101180 2024-06-11T13:29:06.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.626+0000 7ff184f67640 1 -- 172.21.15.57:0/3222106665 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7ff1801a9150 con 0x7ff180101180 2024-06-11T13:29:06.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.626+0000 7ff1777fe640 1 -- 172.21.15.57:0/3222106665 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7ff168003e20 con 0x7ff180101180 2024-06-11T13:29:06.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.626+0000 7ff1777fe640 1 -- 172.21.15.57:0/3222106665 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7ff168005f80 con 0x7ff180101180 2024-06-11T13:29:06.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.627+0000 7ff184f67640 1 -- 172.21.15.57:0/3222106665 --> v1:172.21.15.57:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7ff160000f80 con 0x7ff180101180 2024-06-11T13:29:06.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.628+0000 7ff1777fe640 1 -- 172.21.15.57:0/3222106665 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 22) ==== 92138+0+0 (unknown 311317106 0 0) 0x7ff16801c920 con 0x7ff180101180 2024-06-11T13:29:06.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.628+0000 7ff1777fe640 1 -- 172.21.15.57:0/3222106665 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(70..70 src has 1..70) ==== 6172+0+0 (unknown 3813449920 0 0) 0x7ff168089870 con 0x7ff180101180 2024-06-11T13:29:06.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.628+0000 7ff1777fe640 1 -- 172.21.15.57:0/3222106665 --> v1:172.21.15.107:6808/1305047914 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff1680047b0 con 0x7ff16c0797d0 2024-06-11T13:29:06.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.628+0000 7ff1777fe640 1 -- 172.21.15.57:0/3222106665 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (unknown 2350636211 0 0) 0x7ff168089c80 con 0x7ff180101180 2024-06-11T13:29:06.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.630+0000 7ff1777fe640 1 -- 172.21.15.57:0/3222106665 <== osd.6 v1:172.21.15.107:6808/1305047914 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7ff1680047b0 con 0x7ff16c0797d0 2024-06-11T13:29:06.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.644+0000 7ff184f67640 1 -- 172.21.15.57:0/3222106665 --> v1:172.21.15.107:6808/1305047914 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff160002c50 con 0x7ff16c0797d0 2024-06-11T13:29:06.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.645+0000 7ff1777fe640 1 -- 172.21.15.57:0/3222106665 <== osd.6 v1:172.21.15.107:6808/1305047914 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (unknown 0 0 3659474624) 0x7ff160002c50 con 0x7ff16c0797d0 2024-06-11T13:29:06.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.645+0000 7ff184f67640 1 -- 172.21.15.57:0/3222106665 >> v1:172.21.15.107:6808/1305047914 conn(0x7ff16c0797d0 legacy=0x7ff16c07bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:06.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.645+0000 7ff184f67640 1 -- 172.21.15.57:0/3222106665 >> v1:172.21.15.57:6800/2077416415 conn(0x7ff16c06ef60 legacy=0x7ff16c071420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:06.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.645+0000 7ff184f67640 1 -- 172.21.15.57:0/3222106665 >> v1:172.21.15.57:6789/0 conn(0x7ff180101180 legacy=0x7ff1801a38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:06.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.645+0000 7ff17ed9d640 1 -- 172.21.15.57:0/3222106665 reap_dead start 2024-06-11T13:29:06.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.645+0000 7ff184f67640 1 -- 172.21.15.57:0/3222106665 shutdown_connections 2024-06-11T13:29:06.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.645+0000 7ff184f67640 1 -- 172.21.15.57:0/3222106665 >> 172.21.15.57:0/3222106665 conn(0x7ff1800fc220 msgr2=0x7ff1800fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:29:06.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.645+0000 7ff184f67640 1 -- 172.21.15.57:0/3222106665 shutdown_connections 2024-06-11T13:29:06.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:06.645+0000 7ff184f67640 1 -- 172.21.15.57:0/3222106665 wait complete. 2024-06-11T13:29:07.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:07 smithi057 ceph-mon[29805]: pgmap v11: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-06-11T13:29:07.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:07 smithi057 ceph-mon[40588]: pgmap v11: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-06-11T13:29:07.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:07 smithi107 ceph-mon[33178]: pgmap v11: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-06-11T13:29:08.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.554+0000 7f3d11f29640 1 -- 172.21.15.57:0/3238522214 >> v1:172.21.15.57:6790/0 conn(0x7f3d0c1095a0 legacy=0x7f3d0c10b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:08.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.555+0000 7f3d11f29640 1 -- 172.21.15.57:0/3238522214 shutdown_connections 2024-06-11T13:29:08.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.555+0000 7f3d11f29640 1 -- 172.21.15.57:0/3238522214 >> 172.21.15.57:0/3238522214 conn(0x7f3d0c0fc220 msgr2=0x7f3d0c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:29:08.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.555+0000 7f3d11f29640 1 -- 172.21.15.57:0/3238522214 shutdown_connections 2024-06-11T13:29:08.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.555+0000 7f3d11f29640 1 -- 172.21.15.57:0/3238522214 wait complete. 2024-06-11T13:29:08.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.555+0000 7f3d11f29640 1 Processor -- start 2024-06-11T13:29:08.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.555+0000 7f3d11f29640 1 -- start start 2024-06-11T13:29:08.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.555+0000 7f3d11f29640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f3d0c10cad0 con 0x7f3d0c10d140 2024-06-11T13:29:08.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.555+0000 7f3d11f29640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f3d0c10cd00 con 0x7f3d0c1095a0 2024-06-11T13:29:08.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.555+0000 7f3d11f29640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f3d0c10cf30 con 0x7f3d0c1066f0 2024-06-11T13:29:08.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.555+0000 7f3d0b7fe640 1 --1- >> v1:172.21.15.57:6790/0 conn(0x7f3d0c1066f0 0x7f3d0c10d6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6790/0 says I am v1:172.21.15.57:60438/0 (socket says 172.21.15.57:60438) 2024-06-11T13:29:08.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.556+0000 7f3d0b7fe640 1 -- 172.21.15.57:0/3031410281 learned_addr learned my addr 172.21.15.57:0/3031410281 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:29:08.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.556+0000 7f3d097fa640 1 -- 172.21.15.57:0/3031410281 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2465277200 0 0) 0x7f3d0c10cad0 con 0x7f3d0c10d140 2024-06-11T13:29:08.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.556+0000 7f3d097fa640 1 -- 172.21.15.57:0/3031410281 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3cf8003520 con 0x7f3d0c10d140 2024-06-11T13:29:08.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.556+0000 7f3d097fa640 1 -- 172.21.15.57:0/3031410281 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 619175529 0 0) 0x7f3d0c10cf30 con 0x7f3d0c1066f0 2024-06-11T13:29:08.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.556+0000 7f3d097fa640 1 -- 172.21.15.57:0/3031410281 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d0c10cad0 con 0x7f3d0c1066f0 2024-06-11T13:29:08.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.556+0000 7f3d097fa640 1 -- 172.21.15.57:0/3031410281 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3753826757 0 0) 0x7f3d0c10cd00 con 0x7f3d0c1095a0 2024-06-11T13:29:08.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.556+0000 7f3d097fa640 1 -- 172.21.15.57:0/3031410281 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d0c10cf30 con 0x7f3d0c1095a0 2024-06-11T13:29:08.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.556+0000 7f3d097fa640 1 -- 172.21.15.57:0/3031410281 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2854095775 0 0) 0x7f3cf8003520 con 0x7f3d0c10d140 2024-06-11T13:29:08.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.556+0000 7f3d097fa640 1 -- 172.21.15.57:0/3031410281 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f3d0c10cd00 con 0x7f3d0c10d140 2024-06-11T13:29:08.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.556+0000 7f3d097fa640 1 -- 172.21.15.57:0/3031410281 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f3d04001b70 con 0x7f3d0c10d140 2024-06-11T13:29:08.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.556+0000 7f3d097fa640 1 -- 172.21.15.57:0/3031410281 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1427338638 0 0) 0x7f3d0c10cad0 con 0x7f3d0c1066f0 2024-06-11T13:29:08.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.556+0000 7f3d097fa640 1 -- 172.21.15.57:0/3031410281 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f3cf8003520 con 0x7f3d0c1066f0 2024-06-11T13:29:08.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.556+0000 7f3d097fa640 1 -- 172.21.15.57:0/3031410281 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f3cfc0019b0 con 0x7f3d0c1066f0 2024-06-11T13:29:08.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.556+0000 7f3d097fa640 1 -- 172.21.15.57:0/3031410281 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3061043561 0 0) 0x7f3d0c10cf30 con 0x7f3d0c1095a0 2024-06-11T13:29:08.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.556+0000 7f3d097fa640 1 -- 172.21.15.57:0/3031410281 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f3d0c10cad0 con 0x7f3d0c1095a0 2024-06-11T13:29:08.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.556+0000 7f3d097fa640 1 -- 172.21.15.57:0/3031410281 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f3cf40044c0 con 0x7f3d0c1095a0 2024-06-11T13:29:08.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.556+0000 7f3d097fa640 1 -- 172.21.15.57:0/3031410281 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4124601113 0 0) 0x7f3d0c10cd00 con 0x7f3d0c10d140 2024-06-11T13:29:08.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.556+0000 7f3d097fa640 1 -- 172.21.15.57:0/3031410281 >> v1:172.21.15.57:6790/0 conn(0x7f3d0c1066f0 legacy=0x7f3d0c10d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:08.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.556+0000 7f3d097fa640 1 -- 172.21.15.57:0/3031410281 >> v1:172.21.15.107:6789/0 conn(0x7f3d0c1095a0 legacy=0x7f3d0c10ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:08.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.556+0000 7f3d097fa640 1 -- 172.21.15.57:0/3031410281 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d0c1ac170 con 0x7f3d0c10d140 2024-06-11T13:29:08.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.556+0000 7f3d11f29640 1 -- 172.21.15.57:0/3031410281 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f3d0c1a9160 con 0x7f3d0c10d140 2024-06-11T13:29:08.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.556+0000 7f3d11f29640 1 -- 172.21.15.57:0/3031410281 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f3d0c1a9620 con 0x7f3d0c10d140 2024-06-11T13:29:08.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.556+0000 7f3d097fa640 1 -- 172.21.15.57:0/3031410281 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f3d04003e20 con 0x7f3d0c10d140 2024-06-11T13:29:08.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.556+0000 7f3d097fa640 1 -- 172.21.15.57:0/3031410281 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f3d04004ca0 con 0x7f3d0c10d140 2024-06-11T13:29:08.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.557+0000 7f3d11f29640 1 -- 172.21.15.57:0/3031410281 --> v1:172.21.15.57:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f3cec000f80 con 0x7f3d0c10d140 2024-06-11T13:29:08.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.558+0000 7f3d097fa640 1 -- 172.21.15.57:0/3031410281 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 22) ==== 92138+0+0 (unknown 311317106 0 0) 0x7f3d04003990 con 0x7f3d0c10d140 2024-06-11T13:29:08.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.558+0000 7f3d097fa640 1 -- 172.21.15.57:0/3031410281 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(70..70 src has 1..70) ==== 6172+0+0 (unknown 3813449920 0 0) 0x7f3d04088440 con 0x7f3d0c10d140 2024-06-11T13:29:08.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.559+0000 7f3d097fa640 1 -- 172.21.15.57:0/3031410281 --> v1:172.21.15.107:6804/2372371519 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3cf807d2c0 con 0x7f3cf8079610 2024-06-11T13:29:08.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.559+0000 7f3d097fa640 1 -- 172.21.15.57:0/3031410281 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (unknown 2350636211 0 0) 0x7f3d04088910 con 0x7f3d0c10d140 2024-06-11T13:29:08.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.560+0000 7f3d097fa640 1 -- 172.21.15.57:0/3031410281 <== osd.5 v1:172.21.15.107:6804/2372371519 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f3cf807d2c0 con 0x7f3cf8079610 2024-06-11T13:29:08.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.574+0000 7f3d11f29640 1 -- 172.21.15.57:0/3031410281 --> v1:172.21.15.107:6804/2372371519 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3cec002c50 con 0x7f3cf8079610 2024-06-11T13:29:08.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.575+0000 7f3d097fa640 1 -- 172.21.15.57:0/3031410281 <== osd.5 v1:172.21.15.107:6804/2372371519 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (unknown 0 0 3853432738) 0x7f3cec002c50 con 0x7f3cf8079610 2024-06-11T13:29:08.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.575+0000 7f3d11f29640 1 -- 172.21.15.57:0/3031410281 >> v1:172.21.15.107:6804/2372371519 conn(0x7f3cf8079610 legacy=0x7f3cf807ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:08.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.575+0000 7f3d11f29640 1 -- 172.21.15.57:0/3031410281 >> v1:172.21.15.57:6800/2077416415 conn(0x7f3cf806eea0 legacy=0x7f3cf8071360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:08.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.575+0000 7f3d11f29640 1 -- 172.21.15.57:0/3031410281 >> v1:172.21.15.57:6789/0 conn(0x7f3d0c10d140 legacy=0x7f3d0c1a79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:08.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.575+0000 7f3d0bfff640 1 -- 172.21.15.57:0/3031410281 reap_dead start 2024-06-11T13:29:08.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.575+0000 7f3d11f29640 1 -- 172.21.15.57:0/3031410281 shutdown_connections 2024-06-11T13:29:08.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.575+0000 7f3d11f29640 1 -- 172.21.15.57:0/3031410281 >> 172.21.15.57:0/3031410281 conn(0x7f3d0c0fc220 msgr2=0x7f3d0c069c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:29:08.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.575+0000 7f3d11f29640 1 -- 172.21.15.57:0/3031410281 shutdown_connections 2024-06-11T13:29:08.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:08.575+0000 7f3d11f29640 1 -- 172.21.15.57:0/3031410281 wait complete. 2024-06-11T13:29:09.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:09 smithi057 ceph-mon[40588]: pgmap v12: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:29:09.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:09 smithi057 ceph-mon[29805]: pgmap v12: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:29:09.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:09 smithi107 ceph-mon[33178]: pgmap v12: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:29:09.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.470+0000 7fc7f6705640 1 -- 172.21.15.57:0/3649796418 >> v1:172.21.15.57:6789/0 conn(0x7fc7f0102a80 legacy=0x7fc7f0102e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:09.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.470+0000 7fc7f6705640 1 -- 172.21.15.57:0/3649796418 shutdown_connections 2024-06-11T13:29:09.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.470+0000 7fc7f6705640 1 -- 172.21.15.57:0/3649796418 >> 172.21.15.57:0/3649796418 conn(0x7fc7f00fc230 msgr2=0x7fc7f00fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:29:09.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.470+0000 7fc7f6705640 1 -- 172.21.15.57:0/3649796418 shutdown_connections 2024-06-11T13:29:09.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.470+0000 7fc7f6705640 1 -- 172.21.15.57:0/3649796418 wait complete. 2024-06-11T13:29:09.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.471+0000 7fc7f6705640 1 Processor -- start 2024-06-11T13:29:09.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.471+0000 7fc7f6705640 1 -- start start 2024-06-11T13:29:09.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.471+0000 7fc7f6705640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fc7f01a8a70 con 0x7fc7f010e3a0 2024-06-11T13:29:09.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.471+0000 7fc7f6705640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fc7f01a9cb0 con 0x7fc7f0102a80 2024-06-11T13:29:09.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.471+0000 7fc7f6705640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fc7f01aaef0 con 0x7fc7f010a890 2024-06-11T13:29:09.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.471+0000 7fc7ef7fe640 1 --1- >> v1:172.21.15.57:6790/0 conn(0x7fc7f010a890 0x7fc7f01a3ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6790/0 says I am v1:172.21.15.57:52278/0 (socket says 172.21.15.57:52278) 2024-06-11T13:29:09.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.471+0000 7fc7ef7fe640 1 -- 172.21.15.57:0/1794300415 learned_addr learned my addr 172.21.15.57:0/1794300415 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:29:09.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.472+0000 7fc7ed7fa640 1 -- 172.21.15.57:0/1794300415 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3740481123 0 0) 0x7fc7f01a8a70 con 0x7fc7f010e3a0 2024-06-11T13:29:09.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.472+0000 7fc7ed7fa640 1 -- 172.21.15.57:0/1794300415 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc7d8003520 con 0x7fc7f010e3a0 2024-06-11T13:29:09.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.472+0000 7fc7ed7fa640 1 -- 172.21.15.57:0/1794300415 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2638971692 0 0) 0x7fc7f01aaef0 con 0x7fc7f010a890 2024-06-11T13:29:09.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.472+0000 7fc7ed7fa640 1 -- 172.21.15.57:0/1794300415 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc7f01a8a70 con 0x7fc7f010a890 2024-06-11T13:29:09.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.472+0000 7fc7ed7fa640 1 -- 172.21.15.57:0/1794300415 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3108959987 0 0) 0x7fc7f01a9cb0 con 0x7fc7f0102a80 2024-06-11T13:29:09.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.472+0000 7fc7ed7fa640 1 -- 172.21.15.57:0/1794300415 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc7f01aaef0 con 0x7fc7f0102a80 2024-06-11T13:29:09.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.472+0000 7fc7ed7fa640 1 -- 172.21.15.57:0/1794300415 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3550984191 0 0) 0x7fc7d8003520 con 0x7fc7f010e3a0 2024-06-11T13:29:09.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.472+0000 7fc7ed7fa640 1 -- 172.21.15.57:0/1794300415 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc7f01a9cb0 con 0x7fc7f010e3a0 2024-06-11T13:29:09.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.472+0000 7fc7ed7fa640 1 -- 172.21.15.57:0/1794300415 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fc7e8001b70 con 0x7fc7f010e3a0 2024-06-11T13:29:09.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.472+0000 7fc7ed7fa640 1 -- 172.21.15.57:0/1794300415 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4026245197 0 0) 0x7fc7f01a8a70 con 0x7fc7f010a890 2024-06-11T13:29:09.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.472+0000 7fc7ed7fa640 1 -- 172.21.15.57:0/1794300415 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc7d8003520 con 0x7fc7f010a890 2024-06-11T13:29:09.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.472+0000 7fc7ed7fa640 1 -- 172.21.15.57:0/1794300415 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fc7e4001890 con 0x7fc7f010a890 2024-06-11T13:29:09.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.472+0000 7fc7ed7fa640 1 -- 172.21.15.57:0/1794300415 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3953787339 0 0) 0x7fc7f01aaef0 con 0x7fc7f0102a80 2024-06-11T13:29:09.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.472+0000 7fc7ed7fa640 1 -- 172.21.15.57:0/1794300415 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc7f01a8a70 con 0x7fc7f0102a80 2024-06-11T13:29:09.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.472+0000 7fc7ed7fa640 1 -- 172.21.15.57:0/1794300415 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fc7e0004410 con 0x7fc7f0102a80 2024-06-11T13:29:09.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.472+0000 7fc7ed7fa640 1 -- 172.21.15.57:0/1794300415 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3498339868 0 0) 0x7fc7f01a9cb0 con 0x7fc7f010e3a0 2024-06-11T13:29:09.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.472+0000 7fc7ed7fa640 1 -- 172.21.15.57:0/1794300415 >> v1:172.21.15.57:6790/0 conn(0x7fc7f010a890 legacy=0x7fc7f01a3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:09.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.472+0000 7fc7ed7fa640 1 -- 172.21.15.57:0/1794300415 >> v1:172.21.15.107:6789/0 conn(0x7fc7f0102a80 legacy=0x7fc7f010a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:09.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.472+0000 7fc7ed7fa640 1 -- 172.21.15.57:0/1794300415 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f01ac130 con 0x7fc7f010e3a0 2024-06-11T13:29:09.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.472+0000 7fc7f6705640 1 -- 172.21.15.57:0/1794300415 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fc7f01ab120 con 0x7fc7f010e3a0 2024-06-11T13:29:09.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.472+0000 7fc7f6705640 1 -- 172.21.15.57:0/1794300415 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fc7f01ab5e0 con 0x7fc7f010e3a0 2024-06-11T13:29:09.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.472+0000 7fc7ed7fa640 1 -- 172.21.15.57:0/1794300415 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7fc7e8003e00 con 0x7fc7f010e3a0 2024-06-11T13:29:09.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.472+0000 7fc7ed7fa640 1 -- 172.21.15.57:0/1794300415 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fc7e8004d20 con 0x7fc7f010e3a0 2024-06-11T13:29:09.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.473+0000 7fc7f6705640 1 -- 172.21.15.57:0/1794300415 --> v1:172.21.15.57:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7fc7dc000f80 con 0x7fc7f010e3a0 2024-06-11T13:29:09.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.474+0000 7fc7ed7fa640 1 -- 172.21.15.57:0/1794300415 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 22) ==== 92138+0+0 (unknown 311317106 0 0) 0x7fc7e8003970 con 0x7fc7f010e3a0 2024-06-11T13:29:09.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.474+0000 7fc7ed7fa640 1 -- 172.21.15.57:0/1794300415 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(70..70 src has 1..70) ==== 6172+0+0 (unknown 3813449920 0 0) 0x7fc7e8088870 con 0x7fc7f010e3a0 2024-06-11T13:29:09.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.475+0000 7fc7ed7fa640 1 -- 172.21.15.57:0/1794300415 --> v1:172.21.15.57:6801/978735223 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc7e0001080 con 0x7fc7d80796e0 2024-06-11T13:29:09.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.475+0000 7fc7ed7fa640 1 -- 172.21.15.57:0/1794300415 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (unknown 2350636211 0 0) 0x7fc7e8088ce0 con 0x7fc7f010e3a0 2024-06-11T13:29:09.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.476+0000 7fc7ed7fa640 1 -- 172.21.15.57:0/1794300415 <== osd.0 v1:172.21.15.57:6801/978735223 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7fc7e0001080 con 0x7fc7d80796e0 2024-06-11T13:29:09.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.492+0000 7fc7f6705640 1 -- 172.21.15.57:0/1794300415 --> v1:172.21.15.57:6801/978735223 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc7dc002c50 con 0x7fc7d80796e0 2024-06-11T13:29:09.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.493+0000 7fc7ed7fa640 1 -- 172.21.15.57:0/1794300415 <== osd.0 v1:172.21.15.57:6801/978735223 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 1002017476) 0x7fc7dc002c50 con 0x7fc7d80796e0 2024-06-11T13:29:09.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.493+0000 7fc7f6705640 1 -- 172.21.15.57:0/1794300415 >> v1:172.21.15.57:6801/978735223 conn(0x7fc7d80796e0 legacy=0x7fc7d807bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:09.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.493+0000 7fc7f6705640 1 -- 172.21.15.57:0/1794300415 >> v1:172.21.15.57:6800/2077416415 conn(0x7fc7d806eef0 legacy=0x7fc7d80713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:09.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.493+0000 7fc7f6705640 1 -- 172.21.15.57:0/1794300415 >> v1:172.21.15.57:6789/0 conn(0x7fc7f010e3a0 legacy=0x7fc7f01a71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:09.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.493+0000 7fc7f4ca2640 1 -- 172.21.15.57:0/1794300415 reap_dead start 2024-06-11T13:29:09.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.493+0000 7fc7f6705640 1 -- 172.21.15.57:0/1794300415 shutdown_connections 2024-06-11T13:29:09.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.493+0000 7fc7f6705640 1 -- 172.21.15.57:0/1794300415 >> 172.21.15.57:0/1794300415 conn(0x7fc7f00fc230 msgr2=0x7fc7f00fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:29:09.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.493+0000 7fc7f6705640 1 -- 172.21.15.57:0/1794300415 shutdown_connections 2024-06-11T13:29:09.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:09.493+0000 7fc7f6705640 1 -- 172.21.15.57:0/1794300415 wait complete. 2024-06-11T13:29:10.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.295+0000 7ffa1ed1e640 1 -- 172.21.15.57:0/2664764379 >> v1:172.21.15.57:6789/0 conn(0x7ffa181066f0 legacy=0x7ffa18106ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:10.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.295+0000 7ffa1ed1e640 1 -- 172.21.15.57:0/2664764379 shutdown_connections 2024-06-11T13:29:10.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.295+0000 7ffa1ed1e640 1 -- 172.21.15.57:0/2664764379 >> 172.21.15.57:0/2664764379 conn(0x7ffa180fc220 msgr2=0x7ffa180fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:29:10.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.295+0000 7ffa1ed1e640 1 -- 172.21.15.57:0/2664764379 shutdown_connections 2024-06-11T13:29:10.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.295+0000 7ffa1ed1e640 1 -- 172.21.15.57:0/2664764379 wait complete. 2024-06-11T13:29:10.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.296+0000 7ffa1ed1e640 1 Processor -- start 2024-06-11T13:29:10.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.296+0000 7ffa1ed1e640 1 -- start start 2024-06-11T13:29:10.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.296+0000 7ffa1caba640 1 --1- >> v1:172.21.15.57:6790/0 conn(0x7ffa181066f0 0x7ffa1810ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6790/0 says I am v1:172.21.15.57:52288/0 (socket says 172.21.15.57:52288) 2024-06-11T13:29:10.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.296+0000 7ffa1caba640 1 -- 172.21.15.57:0/1266302660 learned_addr learned my addr 172.21.15.57:0/1266302660 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:29:10.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.297+0000 7ffa1ed1e640 1 -- 172.21.15.57:0/1266302660 --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ffa1810cea0 con 0x7ffa181095a0 2024-06-11T13:29:10.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.297+0000 7ffa1ed1e640 1 -- 172.21.15.57:0/1266302660 --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ffa181a7d20 con 0x7ffa1810d140 2024-06-11T13:29:10.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.297+0000 7ffa1ed1e640 1 -- 172.21.15.57:0/1266302660 --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ffa181a8f00 con 0x7ffa181066f0 2024-06-11T13:29:10.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.297+0000 7ffa0dffb640 1 -- 172.21.15.57:0/1266302660 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 96498744 0 0) 0x7ffa1810cea0 con 0x7ffa181095a0 2024-06-11T13:29:10.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.297+0000 7ffa0dffb640 1 -- 172.21.15.57:0/1266302660 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa00003160 con 0x7ffa181095a0 2024-06-11T13:29:10.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.297+0000 7ffa0dffb640 1 -- 172.21.15.57:0/1266302660 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2358798465 0 0) 0x7ffa181a8f00 con 0x7ffa181066f0 2024-06-11T13:29:10.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.297+0000 7ffa0dffb640 1 -- 172.21.15.57:0/1266302660 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa1810cea0 con 0x7ffa181066f0 2024-06-11T13:29:10.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.297+0000 7ffa0dffb640 1 -- 172.21.15.57:0/1266302660 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4209377352 0 0) 0x7ffa181a7d20 con 0x7ffa1810d140 2024-06-11T13:29:10.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.297+0000 7ffa0dffb640 1 -- 172.21.15.57:0/1266302660 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa181a8f00 con 0x7ffa1810d140 2024-06-11T13:29:10.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.297+0000 7ffa0dffb640 1 -- 172.21.15.57:0/1266302660 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2755489179 0 0) 0x7ffa00003160 con 0x7ffa181095a0 2024-06-11T13:29:10.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.297+0000 7ffa0dffb640 1 -- 172.21.15.57:0/1266302660 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ffa181a7d20 con 0x7ffa181095a0 2024-06-11T13:29:10.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.297+0000 7ffa0dffb640 1 -- 172.21.15.57:0/1266302660 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1266380725 0 0) 0x7ffa181a8f00 con 0x7ffa1810d140 2024-06-11T13:29:10.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.297+0000 7ffa0dffb640 1 -- 172.21.15.57:0/1266302660 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ffa00003160 con 0x7ffa1810d140 2024-06-11T13:29:10.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.297+0000 7ffa0dffb640 1 -- 172.21.15.57:0/1266302660 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3693920734 0 0) 0x7ffa1810cea0 con 0x7ffa181066f0 2024-06-11T13:29:10.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.297+0000 7ffa0dffb640 1 -- 172.21.15.57:0/1266302660 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ffa181a8f00 con 0x7ffa181066f0 2024-06-11T13:29:10.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.297+0000 7ffa0dffb640 1 -- 172.21.15.57:0/1266302660 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7ffa080018d0 con 0x7ffa181095a0 2024-06-11T13:29:10.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.297+0000 7ffa0dffb640 1 -- 172.21.15.57:0/1266302660 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7ffa10001af0 con 0x7ffa1810d140 2024-06-11T13:29:10.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.297+0000 7ffa0dffb640 1 -- 172.21.15.57:0/1266302660 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7ffa14001a00 con 0x7ffa181066f0 2024-06-11T13:29:10.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.298+0000 7ffa0dffb640 1 -- 172.21.15.57:0/1266302660 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 790935997 0 0) 0x7ffa181a7d20 con 0x7ffa181095a0 2024-06-11T13:29:10.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.298+0000 7ffa0dffb640 1 -- 172.21.15.57:0/1266302660 >> v1:172.21.15.57:6790/0 conn(0x7ffa181066f0 legacy=0x7ffa1810ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:10.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.298+0000 7ffa0dffb640 1 -- 172.21.15.57:0/1266302660 >> v1:172.21.15.107:6789/0 conn(0x7ffa1810d140 legacy=0x7ffa181a55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:10.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.298+0000 7ffa0dffb640 1 -- 172.21.15.57:0/1266302660 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa181aa140 con 0x7ffa181095a0 2024-06-11T13:29:10.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.298+0000 7ffa1ed1e640 1 -- 172.21.15.57:0/1266302660 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7ffa181a9130 con 0x7ffa181095a0 2024-06-11T13:29:10.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.298+0000 7ffa1ed1e640 1 -- 172.21.15.57:0/1266302660 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7ffa181a95f0 con 0x7ffa181095a0 2024-06-11T13:29:10.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.298+0000 7ffa0dffb640 1 -- 172.21.15.57:0/1266302660 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7ffa08003aa0 con 0x7ffa181095a0 2024-06-11T13:29:10.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.299+0000 7ffa0dffb640 1 -- 172.21.15.57:0/1266302660 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7ffa080049e0 con 0x7ffa181095a0 2024-06-11T13:29:10.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.299+0000 7ffa1ed1e640 1 -- 172.21.15.57:0/1266302660 --> v1:172.21.15.57:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7ffa04000f80 con 0x7ffa181095a0 2024-06-11T13:29:10.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.300+0000 7ffa0dffb640 1 -- 172.21.15.57:0/1266302660 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 22) ==== 92138+0+0 (unknown 311317106 0 0) 0x7ffa08003610 con 0x7ffa181095a0 2024-06-11T13:29:10.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.301+0000 7ffa0dffb640 1 -- 172.21.15.57:0/1266302660 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(70..70 src has 1..70) ==== 6172+0+0 (unknown 3813449920 0 0) 0x7ffa08087a80 con 0x7ffa181095a0 2024-06-11T13:29:10.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.301+0000 7ffa0dffb640 1 -- 172.21.15.57:0/1266302660 --> v1:172.21.15.57:6809/385321295 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffa00081d00 con 0x7ffa0007e050 2024-06-11T13:29:10.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.301+0000 7ffa0dffb640 1 -- 172.21.15.57:0/1266302660 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (unknown 2350636211 0 0) 0x7ffa08087d00 con 0x7ffa181095a0 2024-06-11T13:29:10.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.302+0000 7ffa0dffb640 1 -- 172.21.15.57:0/1266302660 <== osd.2 v1:172.21.15.57:6809/385321295 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7ffa00081d00 con 0x7ffa0007e050 2024-06-11T13:29:10.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.319+0000 7ffa1ed1e640 1 -- 172.21.15.57:0/1266302660 --> v1:172.21.15.57:6809/385321295 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffa04002c50 con 0x7ffa0007e050 2024-06-11T13:29:10.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.320+0000 7ffa0dffb640 1 -- 172.21.15.57:0/1266302660 <== osd.2 v1:172.21.15.57:6809/385321295 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 1652615586) 0x7ffa04002c50 con 0x7ffa0007e050 2024-06-11T13:29:10.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.320+0000 7ffa1ed1e640 1 -- 172.21.15.57:0/1266302660 >> v1:172.21.15.57:6809/385321295 conn(0x7ffa0007e050 legacy=0x7ffa00080490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:10.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.320+0000 7ffa1ed1e640 1 -- 172.21.15.57:0/1266302660 >> v1:172.21.15.57:6800/2077416415 conn(0x7ffa000737e0 legacy=0x7ffa00075ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:10.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.320+0000 7ffa1ed1e640 1 -- 172.21.15.57:0/1266302660 >> v1:172.21.15.57:6789/0 conn(0x7ffa181095a0 legacy=0x7ffa1810b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:10.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.320+0000 7ffa1d2bb640 1 -- 172.21.15.57:0/1266302660 reap_dead start 2024-06-11T13:29:10.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.320+0000 7ffa1ed1e640 1 -- 172.21.15.57:0/1266302660 shutdown_connections 2024-06-11T13:29:10.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.320+0000 7ffa1ed1e640 1 -- 172.21.15.57:0/1266302660 >> 172.21.15.57:0/1266302660 conn(0x7ffa180fc220 msgr2=0x7ffa18100790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:29:10.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.320+0000 7ffa1ed1e640 1 -- 172.21.15.57:0/1266302660 shutdown_connections 2024-06-11T13:29:10.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.320+0000 7ffa1ed1e640 1 -- 172.21.15.57:0/1266302660 wait complete. 2024-06-11T13:29:10.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.446+0000 7f118d8ec640 1 -- 172.21.15.57:0/1111598098 >> v1:172.21.15.57:6790/0 conn(0x7f118810a7b0 legacy=0x7f118810cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:10.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.446+0000 7f118d8ec640 1 -- 172.21.15.57:0/1111598098 shutdown_connections 2024-06-11T13:29:10.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.446+0000 7f118d8ec640 1 -- 172.21.15.57:0/1111598098 >> 172.21.15.57:0/1111598098 conn(0x7f11880fc1f0 msgr2=0x7f11880fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:29:10.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.446+0000 7f118d8ec640 1 -- 172.21.15.57:0/1111598098 shutdown_connections 2024-06-11T13:29:10.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.446+0000 7f118d8ec640 1 -- 172.21.15.57:0/1111598098 wait complete. 2024-06-11T13:29:10.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.447+0000 7f118d8ec640 1 Processor -- start 2024-06-11T13:29:10.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.447+0000 7f118d8ec640 1 -- start start 2024-06-11T13:29:10.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.448+0000 7f118d8ec640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f11881a67e0 con 0x7f118810a7b0 2024-06-11T13:29:10.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.448+0000 7f118d8ec640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f11881a7a20 con 0x7f11881029a0 2024-06-11T13:29:10.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.448+0000 7f118d8ec640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f11881a8c60 con 0x7f118810e2c0 2024-06-11T13:29:10.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.448+0000 7f11867fc640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f118810a7b0 0x7f11881a16a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:42986/0 (socket says 172.21.15.57:42986) 2024-06-11T13:29:10.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.448+0000 7f11867fc640 1 -- 172.21.15.57:0/408511994 learned_addr learned my addr 172.21.15.57:0/408511994 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:29:10.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.448+0000 7f11877fe640 1 --1- 172.21.15.57:0/408511994 >> v1:172.21.15.57:6790/0 conn(0x7f118810e2c0 0x7f11881a4ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6790/0 says I am v1:172.21.15.57:52300/0 (socket says 172.21.15.57:52300) 2024-06-11T13:29:10.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.448+0000 7f118c8ea640 1 -- 172.21.15.57:0/408511994 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4180647655 0 0) 0x7f11881a8c60 con 0x7f118810e2c0 2024-06-11T13:29:10.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.448+0000 7f118c8ea640 1 -- 172.21.15.57:0/408511994 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1170003520 con 0x7f118810e2c0 2024-06-11T13:29:10.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.448+0000 7f118c8ea640 1 -- 172.21.15.57:0/408511994 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3383347747 0 0) 0x7f11881a67e0 con 0x7f118810a7b0 2024-06-11T13:29:10.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.448+0000 7f118c8ea640 1 -- 172.21.15.57:0/408511994 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f11881a8c60 con 0x7f118810a7b0 2024-06-11T13:29:10.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.448+0000 7f118c8ea640 1 -- 172.21.15.57:0/408511994 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2963700409 0 0) 0x7f11881a8c60 con 0x7f118810a7b0 2024-06-11T13:29:10.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.448+0000 7f118c8ea640 1 -- 172.21.15.57:0/408511994 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f11881a67e0 con 0x7f118810a7b0 2024-06-11T13:29:10.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.448+0000 7f118c8ea640 1 -- 172.21.15.57:0/408511994 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f117c004410 con 0x7f118810a7b0 2024-06-11T13:29:10.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.448+0000 7f118c8ea640 1 -- 172.21.15.57:0/408511994 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2048302681 0 0) 0x7f1170003520 con 0x7f118810e2c0 2024-06-11T13:29:10.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.448+0000 7f118c8ea640 1 -- 172.21.15.57:0/408511994 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f11881a8c60 con 0x7f118810e2c0 2024-06-11T13:29:10.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.448+0000 7f118c8ea640 1 -- 172.21.15.57:0/408511994 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2639682658 0 0) 0x7f11881a7a20 con 0x7f11881029a0 2024-06-11T13:29:10.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.448+0000 7f118c8ea640 1 -- 172.21.15.57:0/408511994 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1170003520 con 0x7f11881029a0 2024-06-11T13:29:10.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.448+0000 7f118c8ea640 1 -- 172.21.15.57:0/408511994 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f1180001b30 con 0x7f118810e2c0 2024-06-11T13:29:10.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.448+0000 7f118c8ea640 1 -- 172.21.15.57:0/408511994 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 714756739 0 0) 0x7f11881a67e0 con 0x7f118810a7b0 2024-06-11T13:29:10.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.449+0000 7f118c8ea640 1 -- 172.21.15.57:0/408511994 >> v1:172.21.15.57:6790/0 conn(0x7f118810e2c0 legacy=0x7f11881a4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:10.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.449+0000 7f118c8ea640 1 -- 172.21.15.57:0/408511994 >> v1:172.21.15.107:6789/0 conn(0x7f11881029a0 legacy=0x7f1188109f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:10.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.449+0000 7f118c8ea640 1 -- 172.21.15.57:0/408511994 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11881a9ea0 con 0x7f118810a7b0 2024-06-11T13:29:10.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.449+0000 7f118d8ec640 1 -- 172.21.15.57:0/408511994 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f11881a6a10 con 0x7f118810a7b0 2024-06-11T13:29:10.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.449+0000 7f118d8ec640 1 -- 172.21.15.57:0/408511994 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f11881a6ed0 con 0x7f118810a7b0 2024-06-11T13:29:10.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.449+0000 7f118c8ea640 1 -- 172.21.15.57:0/408511994 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f117c004670 con 0x7f118810a7b0 2024-06-11T13:29:10.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.449+0000 7f118c8ea640 1 -- 172.21.15.57:0/408511994 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f117c004c80 con 0x7f118810a7b0 2024-06-11T13:29:10.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.449+0000 7f118d8ec640 1 -- 172.21.15.57:0/408511994 --> v1:172.21.15.57:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f1174000f80 con 0x7f118810a7b0 2024-06-11T13:29:10.938 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.450+0000 7f118c8ea640 1 -- 172.21.15.57:0/408511994 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 22) ==== 92138+0+0 (unknown 311317106 0 0) 0x7f117c0037d0 con 0x7f118810a7b0 2024-06-11T13:29:10.938 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.451+0000 7f118c8ea640 1 -- 172.21.15.57:0/408511994 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(70..70 src has 1..70) ==== 6172+0+0 (unknown 3813449920 0 0) 0x7f117c089490 con 0x7f118810a7b0 2024-06-11T13:29:10.938 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.451+0000 7f118c8ea640 1 -- 172.21.15.57:0/408511994 --> v1:172.21.15.57:6813/1307945371 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f117c001080 con 0x7f1170079940 2024-06-11T13:29:10.938 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.451+0000 7f118c8ea640 1 -- 172.21.15.57:0/408511994 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (unknown 2350636211 0 0) 0x7f117c0898d0 con 0x7f118810a7b0 2024-06-11T13:29:10.938 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.452+0000 7f118c8ea640 1 -- 172.21.15.57:0/408511994 <== osd.3 v1:172.21.15.57:6813/1307945371 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f117c001080 con 0x7f1170079940 2024-06-11T13:29:10.938 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.473+0000 7f118d8ec640 1 -- 172.21.15.57:0/408511994 --> v1:172.21.15.57:6813/1307945371 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1174002c50 con 0x7f1170079940 2024-06-11T13:29:10.938 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.473+0000 7f118c8ea640 1 -- 172.21.15.57:0/408511994 <== osd.3 v1:172.21.15.57:6813/1307945371 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (unknown 0 0 3775874274) 0x7f1174002c50 con 0x7f1170079940 2024-06-11T13:29:10.938 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.473+0000 7f118d8ec640 1 -- 172.21.15.57:0/408511994 >> v1:172.21.15.57:6813/1307945371 conn(0x7f1170079940 legacy=0x7f117007bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:10.938 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.473+0000 7f118d8ec640 1 -- 172.21.15.57:0/408511994 >> v1:172.21.15.57:6800/2077416415 conn(0x7f117006f0d0 legacy=0x7f1170071590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:10.938 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.473+0000 7f118d8ec640 1 -- 172.21.15.57:0/408511994 >> v1:172.21.15.57:6789/0 conn(0x7f118810a7b0 legacy=0x7f11881a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:10.938 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.473+0000 7f11877fe640 1 -- 172.21.15.57:0/408511994 reap_dead start 2024-06-11T13:29:10.938 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.473+0000 7f118d8ec640 1 -- 172.21.15.57:0/408511994 shutdown_connections 2024-06-11T13:29:10.938 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.473+0000 7f118d8ec640 1 -- 172.21.15.57:0/408511994 >> 172.21.15.57:0/408511994 conn(0x7f11880fc1f0 msgr2=0x7f11880fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:29:10.938 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.474+0000 7f118d8ec640 1 -- 172.21.15.57:0/408511994 shutdown_connections 2024-06-11T13:29:10.939 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:10.474+0000 7f118d8ec640 1 -- 172.21.15.57:0/408511994 wait complete. 2024-06-11T13:29:11.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:11 smithi057 ceph-mon[29805]: pgmap v13: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:29:11.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:11 smithi057 ceph-mon[40588]: pgmap v13: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:29:11.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:11 smithi107 ceph-mon[33178]: pgmap v13: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:29:11.296 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:29:10 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:29:11.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.536+0000 7fc0333c3640 1 -- 172.21.15.57:0/815750327 >> v1:172.21.15.57:6790/0 conn(0x7fc02c10e290 legacy=0x7fc02c110750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:11.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.537+0000 7fc0333c3640 1 -- 172.21.15.57:0/815750327 shutdown_connections 2024-06-11T13:29:11.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.537+0000 7fc0333c3640 1 -- 172.21.15.57:0/815750327 >> 172.21.15.57:0/815750327 conn(0x7fc02c0fd350 msgr2=0x7fc02c0ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:29:11.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.537+0000 7fc0333c3640 1 -- 172.21.15.57:0/815750327 shutdown_connections 2024-06-11T13:29:11.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.537+0000 7fc0333c3640 1 -- 172.21.15.57:0/815750327 wait complete. 2024-06-11T13:29:11.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.537+0000 7fc0333c3640 1 Processor -- start 2024-06-11T13:29:11.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.537+0000 7fc0333c3640 1 -- start start 2024-06-11T13:29:11.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.538+0000 7fc0333c3640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fc02c1a6770 con 0x7fc02c10e290 2024-06-11T13:29:11.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.538+0000 7fc0333c3640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fc02c1a79b0 con 0x7fc02c107840 2024-06-11T13:29:11.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.538+0000 7fc0333c3640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fc02c1a8bf0 con 0x7fc02c10a6f0 2024-06-11T13:29:11.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.538+0000 7fc03095e640 1 --1- >> v1:172.21.15.57:6790/0 conn(0x7fc02c10a6f0 0x7fc02c1a1700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6790/0 says I am v1:172.21.15.57:52310/0 (socket says 172.21.15.57:52310) 2024-06-11T13:29:11.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.538+0000 7fc031960640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fc02c10e290 0x7fc02c1a4e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:43008/0 (socket says 172.21.15.57:43008) 2024-06-11T13:29:11.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.538+0000 7fc03095e640 1 -- 172.21.15.57:0/327122045 learned_addr learned my addr 172.21.15.57:0/327122045 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:29:11.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.538+0000 7fc0227fc640 1 -- 172.21.15.57:0/327122045 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4050404875 0 0) 0x7fc02c1a6770 con 0x7fc02c10e290 2024-06-11T13:29:11.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.538+0000 7fc0227fc640 1 -- 172.21.15.57:0/327122045 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc018003520 con 0x7fc02c10e290 2024-06-11T13:29:11.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.538+0000 7fc0227fc640 1 -- 172.21.15.57:0/327122045 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3838763643 0 0) 0x7fc02c1a8bf0 con 0x7fc02c10a6f0 2024-06-11T13:29:11.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.538+0000 7fc0227fc640 1 -- 172.21.15.57:0/327122045 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc02c1a6770 con 0x7fc02c10a6f0 2024-06-11T13:29:11.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.538+0000 7fc0227fc640 1 -- 172.21.15.57:0/327122045 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1659198393 0 0) 0x7fc02c1a6770 con 0x7fc02c10a6f0 2024-06-11T13:29:11.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.538+0000 7fc0227fc640 1 -- 172.21.15.57:0/327122045 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc02c1a8bf0 con 0x7fc02c10a6f0 2024-06-11T13:29:11.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.538+0000 7fc0227fc640 1 -- 172.21.15.57:0/327122045 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3876477821 0 0) 0x7fc018003520 con 0x7fc02c10e290 2024-06-11T13:29:11.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.538+0000 7fc0227fc640 1 -- 172.21.15.57:0/327122045 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc02c1a6770 con 0x7fc02c10e290 2024-06-11T13:29:11.626 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.538+0000 7fc0227fc640 1 -- 172.21.15.57:0/327122045 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fc01c001930 con 0x7fc02c10a6f0 2024-06-11T13:29:11.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.538+0000 7fc0227fc640 1 -- 172.21.15.57:0/327122045 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fc024004660 con 0x7fc02c10e290 2024-06-11T13:29:11.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.538+0000 7fc0227fc640 1 -- 172.21.15.57:0/327122045 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2725438796 0 0) 0x7fc02c1a79b0 con 0x7fc02c107840 2024-06-11T13:29:11.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.538+0000 7fc0227fc640 1 -- 172.21.15.57:0/327122045 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc018003520 con 0x7fc02c107840 2024-06-11T13:29:11.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.538+0000 7fc0227fc640 1 -- 172.21.15.57:0/327122045 <== mon.2 v1:172.21.15.57:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1922527671 0 0) 0x7fc02c1a8bf0 con 0x7fc02c10a6f0 2024-06-11T13:29:11.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.538+0000 7fc0227fc640 1 -- 172.21.15.57:0/327122045 >> v1:172.21.15.107:6789/0 conn(0x7fc02c107840 legacy=0x7fc02c10d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:11.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.538+0000 7fc0227fc640 1 -- 172.21.15.57:0/327122045 >> v1:172.21.15.57:6789/0 conn(0x7fc02c10e290 legacy=0x7fc02c1a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:11.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.538+0000 7fc0227fc640 1 -- 172.21.15.57:0/327122045 --> v1:172.21.15.57:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc02c1a9e30 con 0x7fc02c10a6f0 2024-06-11T13:29:11.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.538+0000 7fc0333c3640 1 -- 172.21.15.57:0/327122045 --> v1:172.21.15.57:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fc02c1a8e20 con 0x7fc02c10a6f0 2024-06-11T13:29:11.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.539+0000 7fc0333c3640 1 -- 172.21.15.57:0/327122045 --> v1:172.21.15.57:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fc02c1a92e0 con 0x7fc02c10a6f0 2024-06-11T13:29:11.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.539+0000 7fc0227fc640 1 -- 172.21.15.57:0/327122045 <== mon.2 v1:172.21.15.57:6790/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7fc01c001cf0 con 0x7fc02c10a6f0 2024-06-11T13:29:11.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.539+0000 7fc0227fc640 1 -- 172.21.15.57:0/327122045 <== mon.2 v1:172.21.15.57:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fc01c005d10 con 0x7fc02c10a6f0 2024-06-11T13:29:11.627 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.539+0000 7fc0333c3640 1 -- 172.21.15.57:0/327122045 --> v1:172.21.15.57:6790/0 -- mon_get_version(what=osdmap handle=1) -- 0x7fc010000fb0 con 0x7fc02c10a6f0 2024-06-11T13:29:11.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.540+0000 7fc0227fc640 1 -- 172.21.15.57:0/327122045 <== mon.2 v1:172.21.15.57:6790/0 7 ==== mgrmap(e 22) ==== 92138+0+0 (unknown 311317106 0 0) 0x7fc01c01c6b0 con 0x7fc02c10a6f0 2024-06-11T13:29:11.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.541+0000 7fc0227fc640 1 -- 172.21.15.57:0/327122045 <== mon.2 v1:172.21.15.57:6790/0 8 ==== osd_map(70..70 src has 1..70) ==== 6172+0+0 (unknown 3813449920 0 0) 0x7fc01c0894a0 con 0x7fc02c10a6f0 2024-06-11T13:29:11.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.541+0000 7fc0227fc640 1 -- 172.21.15.57:0/327122045 --> v1:172.21.15.107:6800/2109046293 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc01807d1e0 con 0x7fc0180795d0 2024-06-11T13:29:11.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.541+0000 7fc0227fc640 1 -- 172.21.15.57:0/327122045 <== mon.2 v1:172.21.15.57:6790/0 9 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (unknown 2350636211 0 0) 0x7fc01c089910 con 0x7fc02c10a6f0 2024-06-11T13:29:11.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.542+0000 7fc0227fc640 1 -- 172.21.15.57:0/327122045 <== osd.4 v1:172.21.15.107:6800/2109046293 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7fc024008e50 con 0x7fc0180795d0 2024-06-11T13:29:11.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.557+0000 7fc0333c3640 1 -- 172.21.15.57:0/327122045 --> v1:172.21.15.107:6800/2109046293 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc010002c80 con 0x7fc0180795d0 2024-06-11T13:29:11.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.557+0000 7fc0227fc640 1 -- 172.21.15.57:0/327122045 <== osd.4 v1:172.21.15.107:6800/2109046293 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (unknown 0 0 3062051976) 0x7fc010002c80 con 0x7fc0180795d0 2024-06-11T13:29:11.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.557+0000 7fc0333c3640 1 -- 172.21.15.57:0/327122045 >> v1:172.21.15.107:6800/2109046293 conn(0x7fc0180795d0 legacy=0x7fc01807ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:11.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.558+0000 7fc0333c3640 1 -- 172.21.15.57:0/327122045 >> v1:172.21.15.57:6800/2077416415 conn(0x7fc01806ee60 legacy=0x7fc018071320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:11.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.558+0000 7fc0333c3640 1 -- 172.21.15.57:0/327122045 >> v1:172.21.15.57:6790/0 conn(0x7fc02c10a6f0 legacy=0x7fc02c1a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:11.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.558+0000 7fc031960640 1 -- 172.21.15.57:0/327122045 reap_dead start 2024-06-11T13:29:11.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.558+0000 7fc0333c3640 1 -- 172.21.15.57:0/327122045 shutdown_connections 2024-06-11T13:29:11.629 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.558+0000 7fc0333c3640 1 -- 172.21.15.57:0/327122045 >> 172.21.15.57:0/327122045 conn(0x7fc02c0fd350 msgr2=0x7fc02c110f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:29:11.629 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.558+0000 7fc0333c3640 1 -- 172.21.15.57:0/327122045 shutdown_connections 2024-06-11T13:29:11.629 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:11.558+0000 7fc0333c3640 1 -- 172.21.15.57:0/327122045 wait complete. 2024-06-11T13:29:12.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:12 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:29:12.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:12 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:29:12.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:12 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:29:12.785 INFO:teuthology.orchestra.run.smithi057.stdout:201863462933 2024-06-11T13:29:12.787 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-06-11T13:29:13.135 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:13 smithi057 ceph-mon[29805]: pgmap v14: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-06-11T13:29:13.135 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:13 smithi057 ceph-mon[40588]: pgmap v14: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-06-11T13:29:13.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:13 smithi107 ceph-mon[33178]: pgmap v14: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-06-11T13:29:14.033 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:29:13 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:29:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:29:14.168 INFO:teuthology.orchestra.run.smithi057.stdout:167503724569 2024-06-11T13:29:14.168 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-06-11T13:29:15.177 INFO:teuthology.orchestra.run.smithi057.stdout:77309411367 2024-06-11T13:29:15.177 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-06-11T13:29:15.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:15 smithi057 ceph-mon[29805]: pgmap v15: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:29:15.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:15 smithi057 ceph-mon[40588]: pgmap v15: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:29:15.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:15 smithi107 ceph-mon[33178]: pgmap v15: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:29:15.606 INFO:teuthology.orchestra.run.smithi057.stdout:128849018911 2024-06-11T13:29:15.606 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-06-11T13:29:16.370 INFO:teuthology.orchestra.run.smithi057.stdout:34359738418 2024-06-11T13:29:16.370 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-06-11T13:29:17.053 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.a/config 2024-06-11T13:29:17.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:17 smithi057 ceph-mon[29805]: pgmap v16: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:29:17.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:17 smithi057 ceph-mon[40588]: pgmap v16: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:29:17.468 INFO:teuthology.orchestra.run.smithi057.stdout:107374182435 2024-06-11T13:29:17.468 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-06-11T13:29:17.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:17 smithi107 ceph-mon[33178]: pgmap v16: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:29:17.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:17.662+0000 7f7ab4d21640 1 -- 172.21.15.57:0/1837753197 >> v1:172.21.15.57:6790/0 conn(0x7f7ab010c8d0 legacy=0x7f7ab010ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:17.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:17.663+0000 7f7ab4d21640 1 -- 172.21.15.57:0/1837753197 shutdown_connections 2024-06-11T13:29:17.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:17.663+0000 7f7ab4d21640 1 -- 172.21.15.57:0/1837753197 >> 172.21.15.57:0/1837753197 conn(0x7f7ab0076b60 msgr2=0x7f7ab00fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:29:17.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:17.663+0000 7f7ab4d21640 1 -- 172.21.15.57:0/1837753197 shutdown_connections 2024-06-11T13:29:17.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:17.663+0000 7f7ab4d21640 1 -- 172.21.15.57:0/1837753197 wait complete. 2024-06-11T13:29:17.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:17.664+0000 7f7ab4d21640 1 Processor -- start 2024-06-11T13:29:17.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:17.664+0000 7f7ab4d21640 1 -- start start 2024-06-11T13:29:17.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:17.665+0000 7f7aae59c640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f7ab01000d0 0x7f7ab0103e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:43016/0 (socket says 172.21.15.57:43016) 2024-06-11T13:29:17.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:17.665+0000 7f7aa7fff640 1 --1- >> v1:172.21.15.57:6790/0 conn(0x7f7ab0108d60 0x7f7ab0104550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6790/0 says I am v1:172.21.15.57:52336/0 (socket says 172.21.15.57:52336) 2024-06-11T13:29:17.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:17.665+0000 7f7aa7fff640 1 -- 172.21.15.57:0/3907482715 learned_addr learned my addr 172.21.15.57:0/3907482715 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:29:17.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:17.665+0000 7f7ab4d21640 1 -- 172.21.15.57:0/3907482715 --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f7ab019cec0 con 0x7f7ab01000d0 2024-06-11T13:29:17.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:17.665+0000 7f7ab4d21640 1 -- 172.21.15.57:0/3907482715 --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f7ab01ae060 con 0x7f7ab010c8d0 2024-06-11T13:29:17.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:17.665+0000 7f7ab4d21640 1 -- 172.21.15.57:0/3907482715 --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f7ab01af280 con 0x7f7ab0108d60 2024-06-11T13:29:17.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:17.666+0000 7f7aa77fe640 1 -- 172.21.15.57:0/3907482715 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1856262147 0 0) 0x7f7ab01af280 con 0x7f7ab0108d60 2024-06-11T13:29:17.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:17.666+0000 7f7aa77fe640 1 -- 172.21.15.57:0/3907482715 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a9c003160 con 0x7f7ab0108d60 2024-06-11T13:29:17.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:17.666+0000 7f7aa77fe640 1 -- 172.21.15.57:0/3907482715 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 704029154 0 0) 0x7f7ab01ae060 con 0x7f7ab010c8d0 2024-06-11T13:29:17.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:17.666+0000 7f7aa77fe640 1 -- 172.21.15.57:0/3907482715 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ab01af280 con 0x7f7ab010c8d0 2024-06-11T13:29:17.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:17.666+0000 7f7aa77fe640 1 -- 172.21.15.57:0/3907482715 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3810861159 0 0) 0x7f7a9c003160 con 0x7f7ab0108d60 2024-06-11T13:29:17.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:17.666+0000 7f7aa77fe640 1 -- 172.21.15.57:0/3907482715 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7ab01ae060 con 0x7f7ab0108d60 2024-06-11T13:29:17.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:17.666+0000 7f7aa77fe640 1 -- 172.21.15.57:0/3907482715 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f7a98001930 con 0x7f7ab0108d60 2024-06-11T13:29:17.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:17.666+0000 7f7aa77fe640 1 -- 172.21.15.57:0/3907482715 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3304080864 0 0) 0x7f7ab01af280 con 0x7f7ab010c8d0 2024-06-11T13:29:17.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:17.666+0000 7f7aa77fe640 1 -- 172.21.15.57:0/3907482715 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7a9c003160 con 0x7f7ab010c8d0 2024-06-11T13:29:17.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:17.666+0000 7f7aa77fe640 1 -- 172.21.15.57:0/3907482715 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f7aa8001d20 con 0x7f7ab010c8d0 2024-06-11T13:29:17.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:17.666+0000 7f7aa77fe640 1 -- 172.21.15.57:0/3907482715 <== mon.2 v1:172.21.15.57:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4058145867 0 0) 0x7f7ab01ae060 con 0x7f7ab0108d60 2024-06-11T13:29:17.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:17.666+0000 7f7aa77fe640 1 -- 172.21.15.57:0/3907482715 >> v1:172.21.15.107:6789/0 conn(0x7f7ab010c8d0 legacy=0x7f7ab01ab930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:17.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:17.666+0000 7f7aa77fe640 1 -- 172.21.15.57:0/3907482715 >> v1:172.21.15.57:6789/0 conn(0x7f7ab01000d0 legacy=0x7f7ab0103e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:17.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:17.666+0000 7f7aa77fe640 1 -- 172.21.15.57:0/3907482715 --> v1:172.21.15.57:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ab01ad050 con 0x7f7ab0108d60 2024-06-11T13:29:17.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:17.666+0000 7f7ab4d21640 1 -- 172.21.15.57:0/3907482715 --> v1:172.21.15.57:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f7ab01ad340 con 0x7f7ab0108d60 2024-06-11T13:29:17.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:17.667+0000 7f7ab4d21640 1 -- 172.21.15.57:0/3907482715 --> v1:172.21.15.57:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f7ab01ad840 con 0x7f7ab0108d60 2024-06-11T13:29:17.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:17.667+0000 7f7aa77fe640 1 -- 172.21.15.57:0/3907482715 <== mon.2 v1:172.21.15.57:6790/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f7a98001d00 con 0x7f7ab0108d60 2024-06-11T13:29:17.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:17.667+0000 7f7aa77fe640 1 -- 172.21.15.57:0/3907482715 <== mon.2 v1:172.21.15.57:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f7a98005d10 con 0x7f7ab0108d60 2024-06-11T13:29:17.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:17.668+0000 7f7ab4d21640 1 -- 172.21.15.57:0/3907482715 --> v1:172.21.15.57:6790/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f7a90000f80 con 0x7f7ab0108d60 2024-06-11T13:29:17.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:17.669+0000 7f7aa77fe640 1 -- 172.21.15.57:0/3907482715 <== mon.2 v1:172.21.15.57:6790/0 7 ==== mgrmap(e 22) ==== 92138+0+0 (unknown 311317106 0 0) 0x7f7a9801c6b0 con 0x7f7ab0108d60 2024-06-11T13:29:17.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:17.669+0000 7f7aa77fe640 1 -- 172.21.15.57:0/3907482715 <== mon.2 v1:172.21.15.57:6790/0 8 ==== osd_map(70..70 src has 1..70) ==== 6172+0+0 (unknown 3813449920 0 0) 0x7f7a98089510 con 0x7f7ab0108d60 2024-06-11T13:29:17.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:17.669+0000 7f7aa77fe640 1 -- 172.21.15.57:0/3907482715 --> v1:172.21.15.107:6812/3207390156 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7a9801d020 con 0x7f7a9c079450 2024-06-11T13:29:17.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:17.669+0000 7f7aa77fe640 1 -- 172.21.15.57:0/3907482715 <== mon.2 v1:172.21.15.57:6790/0 9 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (unknown 2350636211 0 0) 0x7f7a98089950 con 0x7f7ab0108d60 2024-06-11T13:29:17.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:17.671+0000 7f7aa77fe640 1 -- 172.21.15.57:0/3907482715 <== osd.7 v1:172.21.15.107:6812/3207390156 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f7a9801d020 con 0x7f7a9c079450 2024-06-11T13:29:17.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:17.686+0000 7f7ab4d21640 1 -- 172.21.15.57:0/3907482715 --> v1:172.21.15.107:6812/3207390156 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7a90002bf0 con 0x7f7a9c079450 2024-06-11T13:29:17.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:17.687+0000 7f7aa77fe640 1 -- 172.21.15.57:0/3907482715 <== osd.7 v1:172.21.15.107:6812/3207390156 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (unknown 0 0 4044699597) 0x7f7a90002bf0 con 0x7f7a9c079450 2024-06-11T13:29:17.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:17.688+0000 7f7ab4d21640 1 -- 172.21.15.57:0/3907482715 >> v1:172.21.15.107:6812/3207390156 conn(0x7f7a9c079450 legacy=0x7f7a9c07b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:17.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:17.688+0000 7f7ab4d21640 1 -- 172.21.15.57:0/3907482715 >> v1:172.21.15.57:6800/2077416415 conn(0x7f7a9c06ece0 legacy=0x7f7a9c0711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:17.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:17.688+0000 7f7ab4d21640 1 -- 172.21.15.57:0/3907482715 >> v1:172.21.15.57:6790/0 conn(0x7f7ab0108d60 legacy=0x7f7ab0104550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:17.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:17.688+0000 7f7aaed9d640 1 -- 172.21.15.57:0/3907482715 reap_dead start 2024-06-11T13:29:17.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:17.688+0000 7f7ab4d21640 1 -- 172.21.15.57:0/3907482715 shutdown_connections 2024-06-11T13:29:17.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:17.688+0000 7f7ab4d21640 1 -- 172.21.15.57:0/3907482715 >> 172.21.15.57:0/3907482715 conn(0x7f7ab0076b60 msgr2=0x7f7ab00fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:29:17.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:17.688+0000 7f7ab4d21640 1 -- 172.21.15.57:0/3907482715 shutdown_connections 2024-06-11T13:29:17.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:17.688+0000 7f7ab4d21640 1 -- 172.21.15.57:0/3907482715 wait complete. 2024-06-11T13:29:18.868 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.a/config 2024-06-11T13:29:19.147 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.a/config 2024-06-11T13:29:19.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:19 smithi057 ceph-mon[29805]: pgmap v17: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:29:19.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:19 smithi057 ceph-mon[40588]: pgmap v17: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:29:19.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:19 smithi107 ceph-mon[33178]: pgmap v17: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:29:19.851 INFO:teuthology.orchestra.run.smithi057.stdout:236223201299 2024-06-11T13:29:19.852 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-06-11T13:29:20.264 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.a/config 2024-06-11T13:29:20.265 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.a/config 2024-06-11T13:29:20.265 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.a/config 2024-06-11T13:29:20.475 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:20 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:29:20.475 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:20 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:29:20.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:20 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:29:21.272 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:21 smithi057 ceph-mon[29805]: pgmap v18: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:29:21.272 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:21 smithi057 ceph-mon[40588]: pgmap v18: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:29:21.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:21 smithi107 ceph-mon[33178]: pgmap v18: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:29:21.296 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:29:20 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:29:22.448 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:22 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:29:22.449 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:22 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:29:22.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:22 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:29:23.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:23 smithi057 ceph-mon[29805]: pgmap v19: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:29:23.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:23 smithi057 ceph-mon[40588]: pgmap v19: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:29:23.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:23 smithi107 ceph-mon[33178]: pgmap v19: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:29:23.918 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:29:23 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:29:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:29:25.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.129+0000 7fd9204db640 1 -- 172.21.15.57:0/2092822666 >> v1:172.21.15.57:6790/0 conn(0x7fd91810a830 legacy=0x7fd91810cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:25.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.129+0000 7fd9204db640 1 -- 172.21.15.57:0/2092822666 shutdown_connections 2024-06-11T13:29:25.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.129+0000 7fd9204db640 1 -- 172.21.15.57:0/2092822666 >> 172.21.15.57:0/2092822666 conn(0x7fd9180fc1f0 msgr2=0x7fd9180fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:29:25.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.129+0000 7fd9204db640 1 -- 172.21.15.57:0/2092822666 shutdown_connections 2024-06-11T13:29:25.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.129+0000 7fd9204db640 1 -- 172.21.15.57:0/2092822666 wait complete. 2024-06-11T13:29:25.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.130+0000 7fd9204db640 1 Processor -- start 2024-06-11T13:29:25.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.130+0000 7fd9204db640 1 -- start start 2024-06-11T13:29:25.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.131+0000 7fd9204db640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fd9181ad040 con 0x7fd91810e340 2024-06-11T13:29:25.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.131+0000 7fd9204db640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fd9181ae280 con 0x7fd91810a830 2024-06-11T13:29:25.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.131+0000 7fd9204db640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fd9181af4c0 con 0x7fd918102a20 2024-06-11T13:29:25.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.131+0000 7fd91e277640 1 --1- >> v1:172.21.15.57:6790/0 conn(0x7fd918102a20 0x7fd91810a040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6790/0 says I am v1:172.21.15.57:39394/0 (socket says 172.21.15.57:39394) 2024-06-11T13:29:25.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.131+0000 7fd91ea78640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fd91810e340 0x7fd9181ab740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:42350/0 (socket says 172.21.15.57:42350) 2024-06-11T13:29:25.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.131+0000 7fd91e277640 1 -- 172.21.15.57:0/3330234232 learned_addr learned my addr 172.21.15.57:0/3330234232 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:29:25.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.132+0000 7fd90f7fe640 1 -- 172.21.15.57:0/3330234232 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4081523701 0 0) 0x7fd9181ad040 con 0x7fd91810e340 2024-06-11T13:29:25.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.132+0000 7fd90f7fe640 1 -- 172.21.15.57:0/3330234232 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd904003520 con 0x7fd91810e340 2024-06-11T13:29:25.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.132+0000 7fd90f7fe640 1 -- 172.21.15.57:0/3330234232 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2728364302 0 0) 0x7fd9181ae280 con 0x7fd91810a830 2024-06-11T13:29:25.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.132+0000 7fd90f7fe640 1 -- 172.21.15.57:0/3330234232 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd9181ad040 con 0x7fd91810a830 2024-06-11T13:29:25.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.132+0000 7fd90f7fe640 1 -- 172.21.15.57:0/3330234232 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2608542256 0 0) 0x7fd904003520 con 0x7fd91810e340 2024-06-11T13:29:25.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.132+0000 7fd90f7fe640 1 -- 172.21.15.57:0/3330234232 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd9181ae280 con 0x7fd91810e340 2024-06-11T13:29:25.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.133+0000 7fd90f7fe640 1 -- 172.21.15.57:0/3330234232 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fd910001aa0 con 0x7fd91810e340 2024-06-11T13:29:25.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.133+0000 7fd90f7fe640 1 -- 172.21.15.57:0/3330234232 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 757433071 0 0) 0x7fd9181ad040 con 0x7fd91810a830 2024-06-11T13:29:25.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.133+0000 7fd90f7fe640 1 -- 172.21.15.57:0/3330234232 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd904003520 con 0x7fd91810a830 2024-06-11T13:29:25.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.133+0000 7fd90f7fe640 1 -- 172.21.15.57:0/3330234232 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fd908001430 con 0x7fd91810a830 2024-06-11T13:29:25.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.133+0000 7fd90f7fe640 1 -- 172.21.15.57:0/3330234232 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2522359593 0 0) 0x7fd9181ae280 con 0x7fd91810e340 2024-06-11T13:29:25.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.133+0000 7fd90f7fe640 1 -- 172.21.15.57:0/3330234232 >> v1:172.21.15.57:6790/0 conn(0x7fd918102a20 legacy=0x7fd91810a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:25.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.133+0000 7fd90f7fe640 1 -- 172.21.15.57:0/3330234232 >> v1:172.21.15.107:6789/0 conn(0x7fd91810a830 legacy=0x7fd9181a7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:25.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.133+0000 7fd90f7fe640 1 -- 172.21.15.57:0/3330234232 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9181b0700 con 0x7fd91810e340 2024-06-11T13:29:25.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.133+0000 7fd9204db640 1 -- 172.21.15.57:0/3330234232 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fd9181af6f0 con 0x7fd91810e340 2024-06-11T13:29:25.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.133+0000 7fd9204db640 1 -- 172.21.15.57:0/3330234232 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fd9181afbb0 con 0x7fd91810e340 2024-06-11T13:29:25.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.133+0000 7fd90f7fe640 1 -- 172.21.15.57:0/3330234232 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7fd910002070 con 0x7fd91810e340 2024-06-11T13:29:25.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.133+0000 7fd90f7fe640 1 -- 172.21.15.57:0/3330234232 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fd910005e20 con 0x7fd91810e340 2024-06-11T13:29:25.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.134+0000 7fd9204db640 1 -- 172.21.15.57:0/3330234232 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8fc005080 con 0x7fd91810e340 2024-06-11T13:29:25.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.135+0000 7fd90f7fe640 1 -- 172.21.15.57:0/3330234232 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 22) ==== 92138+0+0 (unknown 311317106 0 0) 0x7fd91001c7c0 con 0x7fd91810e340 2024-06-11T13:29:25.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.136+0000 7fd90f7fe640 1 -- 172.21.15.57:0/3330234232 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(70..70 src has 1..70) ==== 6172+0+0 (unknown 3813449920 0 0) 0x7fd910089760 con 0x7fd91810e340 2024-06-11T13:29:25.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.140+0000 7fd90f7fe640 1 -- 172.21.15.57:0/3330234232 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7fd91005abd0 con 0x7fd91810e340 2024-06-11T13:29:25.218 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:25 smithi057 ceph-mon[40588]: pgmap v20: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:29:25.218 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:25 smithi057 ceph-mon[29805]: pgmap v20: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:29:25.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.331+0000 7fd9204db640 1 -- 172.21.15.57:0/3330234232 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fd8fc005370 con 0x7fd91810e340 2024-06-11T13:29:25.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.332+0000 7fd90f7fe640 1 -- 172.21.15.57:0/3330234232 <== mon.0 v1:172.21.15.57:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (unknown 1274345170 0 4120766577) 0x7fd91005f490 con 0x7fd91810e340 2024-06-11T13:29:25.329 INFO:teuthology.orchestra.run.smithi057.stdout:201863462936 2024-06-11T13:29:25.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.334+0000 7fd9204db640 1 -- 172.21.15.57:0/3330234232 >> v1:172.21.15.57:6800/2077416415 conn(0x7fd90406ee50 legacy=0x7fd904071310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:25.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.334+0000 7fd9204db640 1 -- 172.21.15.57:0/3330234232 >> v1:172.21.15.57:6789/0 conn(0x7fd91810e340 legacy=0x7fd9181ab740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:25.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.334+0000 7fd9204db640 1 -- 172.21.15.57:0/3330234232 shutdown_connections 2024-06-11T13:29:25.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.334+0000 7fd9204db640 1 -- 172.21.15.57:0/3330234232 >> 172.21.15.57:0/3330234232 conn(0x7fd9180fc1f0 msgr2=0x7fd91810efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:29:25.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.334+0000 7fd9204db640 1 -- 172.21.15.57:0/3330234232 shutdown_connections 2024-06-11T13:29:25.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.334+0000 7fd9204db640 1 -- 172.21.15.57:0/3330234232 wait complete. 2024-06-11T13:29:25.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:25 smithi107 ceph-mon[33178]: pgmap v20: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:29:25.680 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.675+0000 7f8788c3b640 1 -- 172.21.15.57:0/2849890840 >> v1:172.21.15.57:6789/0 conn(0x7f878410a7c0 legacy=0x7f878410cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:25.680 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.675+0000 7f8788c3b640 1 -- 172.21.15.57:0/2849890840 shutdown_connections 2024-06-11T13:29:25.680 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.675+0000 7f8788c3b640 1 -- 172.21.15.57:0/2849890840 >> 172.21.15.57:0/2849890840 conn(0x7f87840fc220 msgr2=0x7f87840fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:29:25.680 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.675+0000 7f8788c3b640 1 -- 172.21.15.57:0/2849890840 shutdown_connections 2024-06-11T13:29:25.680 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.675+0000 7f8788c3b640 1 -- 172.21.15.57:0/2849890840 wait complete. 2024-06-11T13:29:25.680 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.676+0000 7f8788c3b640 1 Processor -- start 2024-06-11T13:29:25.680 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.676+0000 7f8788c3b640 1 -- start start 2024-06-11T13:29:25.680 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.676+0000 7f8788c3b640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f878419e9c0 con 0x7f878410a7c0 2024-06-11T13:29:25.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.676+0000 7f8788c3b640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f87841a9c60 con 0x7f878410e2d0 2024-06-11T13:29:25.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.676+0000 7f8788c3b640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f87841aae80 con 0x7f87841029b0 2024-06-11T13:29:25.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.676+0000 7f878259c640 1 --1- >> v1:172.21.15.57:6790/0 conn(0x7f87841029b0 0x7f878419dee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6790/0 says I am v1:172.21.15.57:39420/0 (socket says 172.21.15.57:39420) 2024-06-11T13:29:25.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.676+0000 7f878259c640 1 -- 172.21.15.57:0/96194509 learned_addr learned my addr 172.21.15.57:0/96194509 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:29:25.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.677+0000 7f87737fe640 1 -- 172.21.15.57:0/96194509 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 26348500 0 0) 0x7f87841aae80 con 0x7f87841029b0 2024-06-11T13:29:25.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.677+0000 7f87737fe640 1 -- 172.21.15.57:0/96194509 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f876c003520 con 0x7f87841029b0 2024-06-11T13:29:25.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.677+0000 7f87737fe640 1 -- 172.21.15.57:0/96194509 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 601102849 0 0) 0x7f878419e9c0 con 0x7f878410a7c0 2024-06-11T13:29:25.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.677+0000 7f87737fe640 1 -- 172.21.15.57:0/96194509 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f87841aae80 con 0x7f878410a7c0 2024-06-11T13:29:25.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.677+0000 7f87737fe640 1 -- 172.21.15.57:0/96194509 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2628326122 0 0) 0x7f87841a9c60 con 0x7f878410e2d0 2024-06-11T13:29:25.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.677+0000 7f87737fe640 1 -- 172.21.15.57:0/96194509 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f878419e9c0 con 0x7f878410e2d0 2024-06-11T13:29:25.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.677+0000 7f87737fe640 1 -- 172.21.15.57:0/96194509 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 216771357 0 0) 0x7f876c003520 con 0x7f87841029b0 2024-06-11T13:29:25.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.677+0000 7f87737fe640 1 -- 172.21.15.57:0/96194509 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f87841a9c60 con 0x7f87841029b0 2024-06-11T13:29:25.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.677+0000 7f87737fe640 1 -- 172.21.15.57:0/96194509 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f8774001db0 con 0x7f87841029b0 2024-06-11T13:29:25.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.677+0000 7f87737fe640 1 -- 172.21.15.57:0/96194509 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2346042623 0 0) 0x7f87841aae80 con 0x7f878410a7c0 2024-06-11T13:29:25.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.677+0000 7f87737fe640 1 -- 172.21.15.57:0/96194509 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f876c003520 con 0x7f878410a7c0 2024-06-11T13:29:25.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.677+0000 7f87737fe640 1 -- 172.21.15.57:0/96194509 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f8778004370 con 0x7f878410a7c0 2024-06-11T13:29:25.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.677+0000 7f87737fe640 1 -- 172.21.15.57:0/96194509 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3473938879 0 0) 0x7f878419e9c0 con 0x7f878410e2d0 2024-06-11T13:29:25.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.677+0000 7f87737fe640 1 -- 172.21.15.57:0/96194509 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f87841aae80 con 0x7f878410e2d0 2024-06-11T13:29:25.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.677+0000 7f87737fe640 1 -- 172.21.15.57:0/96194509 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f877c0017f0 con 0x7f878410e2d0 2024-06-11T13:29:25.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.677+0000 7f87737fe640 1 -- 172.21.15.57:0/96194509 <== mon.2 v1:172.21.15.57:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2497132609 0 0) 0x7f87841a9c60 con 0x7f87841029b0 2024-06-11T13:29:25.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.677+0000 7f87737fe640 1 -- 172.21.15.57:0/96194509 >> v1:172.21.15.107:6789/0 conn(0x7f878410e2d0 legacy=0x7f87841a7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:25.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.677+0000 7f87737fe640 1 -- 172.21.15.57:0/96194509 >> v1:172.21.15.57:6789/0 conn(0x7f878410a7c0 legacy=0x7f87841a3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:25.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.677+0000 7f87737fe640 1 -- 172.21.15.57:0/96194509 --> v1:172.21.15.57:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87841ac0e0 con 0x7f87841029b0 2024-06-11T13:29:25.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.677+0000 7f8788c3b640 1 -- 172.21.15.57:0/96194509 --> v1:172.21.15.57:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f87841ab0b0 con 0x7f87841029b0 2024-06-11T13:29:25.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.677+0000 7f8788c3b640 1 -- 172.21.15.57:0/96194509 --> v1:172.21.15.57:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f87841ab570 con 0x7f87841029b0 2024-06-11T13:29:25.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.677+0000 7f87737fe640 1 -- 172.21.15.57:0/96194509 <== mon.2 v1:172.21.15.57:6790/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f8774002010 con 0x7f87841029b0 2024-06-11T13:29:25.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.677+0000 7f87737fe640 1 -- 172.21.15.57:0/96194509 <== mon.2 v1:172.21.15.57:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f8774005ed0 con 0x7f87841029b0 2024-06-11T13:29:25.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.678+0000 7f8788c3b640 1 -- 172.21.15.57:0/96194509 --> v1:172.21.15.57:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8764005080 con 0x7f87841029b0 2024-06-11T13:29:25.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.679+0000 7f87737fe640 1 -- 172.21.15.57:0/96194509 <== mon.2 v1:172.21.15.57:6790/0 7 ==== mgrmap(e 22) ==== 92138+0+0 (unknown 311317106 0 0) 0x7f8774003c20 con 0x7f87841029b0 2024-06-11T13:29:25.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.679+0000 7f87737fe640 1 -- 172.21.15.57:0/96194509 <== mon.2 v1:172.21.15.57:6790/0 8 ==== osd_map(70..70 src has 1..70) ==== 6172+0+0 (unknown 3813449920 0 0) 0x7f8774089660 con 0x7f87841029b0 2024-06-11T13:29:25.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.684+0000 7f87737fe640 1 -- 172.21.15.57:0/96194509 <== mon.2 v1:172.21.15.57:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f8774056dc0 con 0x7f87841029b0 2024-06-11T13:29:25.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.882+0000 7f8788c3b640 1 -- 172.21.15.57:0/96194509 --> v1:172.21.15.57:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f8764005370 con 0x7f87841029b0 2024-06-11T13:29:25.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.882+0000 7f87737fe640 1 -- 172.21.15.57:0/96194509 <== mon.2 v1:172.21.15.57:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (unknown 832126871 0 1580136542) 0x7f877405aa70 con 0x7f87841029b0 2024-06-11T13:29:25.879 INFO:teuthology.orchestra.run.smithi057.stdout:55834574896 2024-06-11T13:29:25.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.884+0000 7f8788c3b640 1 -- 172.21.15.57:0/96194509 >> v1:172.21.15.57:6800/2077416415 conn(0x7f876c06ef40 legacy=0x7f876c071400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:25.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.884+0000 7f8788c3b640 1 -- 172.21.15.57:0/96194509 >> v1:172.21.15.57:6790/0 conn(0x7f87841029b0 legacy=0x7f878419dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:25.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.884+0000 7f8788c3b640 1 -- 172.21.15.57:0/96194509 shutdown_connections 2024-06-11T13:29:25.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.884+0000 7f8788c3b640 1 -- 172.21.15.57:0/96194509 >> 172.21.15.57:0/96194509 conn(0x7f87840fc220 msgr2=0x7f8784110740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:29:25.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.884+0000 7f8788c3b640 1 -- 172.21.15.57:0/96194509 shutdown_connections 2024-06-11T13:29:25.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:25.884+0000 7f8788c3b640 1 -- 172.21.15.57:0/96194509 wait complete. 2024-06-11T13:29:26.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:26.322+0000 7f0834d13640 1 -- 172.21.15.57:0/2033313111 >> v1:172.21.15.57:6789/0 conn(0x7f083010a700 legacy=0x7f083010caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:26.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:26.323+0000 7f0834d13640 1 -- 172.21.15.57:0/2033313111 shutdown_connections 2024-06-11T13:29:26.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:26.323+0000 7f0834d13640 1 -- 172.21.15.57:0/2033313111 >> 172.21.15.57:0/2033313111 conn(0x7f08300fd340 msgr2=0x7f08300ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:29:26.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:26.323+0000 7f0834d13640 1 -- 172.21.15.57:0/2033313111 shutdown_connections 2024-06-11T13:29:26.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:26.323+0000 7f0834d13640 1 -- 172.21.15.57:0/2033313111 wait complete. 2024-06-11T13:29:26.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:26.323+0000 7f0834d13640 1 Processor -- start 2024-06-11T13:29:26.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:26.323+0000 7f0834d13640 1 -- start start 2024-06-11T13:29:26.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:26.324+0000 7f0834d13640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f08301a6850 con 0x7f0830107850 2024-06-11T13:29:26.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:26.324+0000 7f0834d13640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f08301a7a90 con 0x7f083010a700 2024-06-11T13:29:26.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:26.324+0000 7f0834d13640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f08301a8cd0 con 0x7f083010e2a0 2024-06-11T13:29:26.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:26.324+0000 7f082e59c640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f0830107850 0x7f083010da80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:42400/0 (socket says 172.21.15.57:42400) 2024-06-11T13:29:26.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:26.324+0000 7f082ed9d640 1 --1- >> v1:172.21.15.57:6790/0 conn(0x7f083010e2a0 0x7f08301a4f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6790/0 says I am v1:172.21.15.57:39432/0 (socket says 172.21.15.57:39432) 2024-06-11T13:29:26.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:26.324+0000 7f082e59c640 1 -- 172.21.15.57:0/455358295 learned_addr learned my addr 172.21.15.57:0/455358295 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:29:26.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:26.324+0000 7f081f7fe640 1 -- 172.21.15.57:0/455358295 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 670873420 0 0) 0x7f08301a6850 con 0x7f0830107850 2024-06-11T13:29:26.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:26.324+0000 7f081f7fe640 1 -- 172.21.15.57:0/455358295 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0818003520 con 0x7f0830107850 2024-06-11T13:29:26.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:26.324+0000 7f081f7fe640 1 -- 172.21.15.57:0/455358295 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 905415301 0 0) 0x7f08301a7a90 con 0x7f083010a700 2024-06-11T13:29:26.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:26.324+0000 7f081f7fe640 1 -- 172.21.15.57:0/455358295 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f08301a6850 con 0x7f083010a700 2024-06-11T13:29:26.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:26.324+0000 7f081f7fe640 1 -- 172.21.15.57:0/455358295 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2522577012 0 0) 0x7f08301a8cd0 con 0x7f083010e2a0 2024-06-11T13:29:26.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:26.324+0000 7f081f7fe640 1 -- 172.21.15.57:0/455358295 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f08301a7a90 con 0x7f083010e2a0 2024-06-11T13:29:26.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:26.324+0000 7f081f7fe640 1 -- 172.21.15.57:0/455358295 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3477383981 0 0) 0x7f0818003520 con 0x7f0830107850 2024-06-11T13:29:26.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:26.325+0000 7f081f7fe640 1 -- 172.21.15.57:0/455358295 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f08301a8cd0 con 0x7f0830107850 2024-06-11T13:29:26.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:26.325+0000 7f081f7fe640 1 -- 172.21.15.57:0/455358295 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f0820001a30 con 0x7f0830107850 2024-06-11T13:29:26.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:26.325+0000 7f081f7fe640 1 -- 172.21.15.57:0/455358295 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3470702575 0 0) 0x7f08301a7a90 con 0x7f083010e2a0 2024-06-11T13:29:26.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:26.325+0000 7f081f7fe640 1 -- 172.21.15.57:0/455358295 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0818003520 con 0x7f083010e2a0 2024-06-11T13:29:26.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:26.325+0000 7f081f7fe640 1 -- 172.21.15.57:0/455358295 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f0828001b30 con 0x7f083010e2a0 2024-06-11T13:29:26.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:26.325+0000 7f081f7fe640 1 -- 172.21.15.57:0/455358295 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2383023959 0 0) 0x7f08301a8cd0 con 0x7f0830107850 2024-06-11T13:29:26.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:26.325+0000 7f081f7fe640 1 -- 172.21.15.57:0/455358295 >> v1:172.21.15.57:6790/0 conn(0x7f083010e2a0 legacy=0x7f08301a4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:26.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:26.325+0000 7f081f7fe640 1 -- 172.21.15.57:0/455358295 >> v1:172.21.15.107:6789/0 conn(0x7f083010a700 legacy=0x7f08301a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:26.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:26.325+0000 7f081f7fe640 1 -- 172.21.15.57:0/455358295 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08301a9f10 con 0x7f0830107850 2024-06-11T13:29:26.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:26.325+0000 7f0834d13640 1 -- 172.21.15.57:0/455358295 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f08301a8f00 con 0x7f0830107850 2024-06-11T13:29:26.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:26.325+0000 7f0834d13640 1 -- 172.21.15.57:0/455358295 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f08301a93c0 con 0x7f0830107850 2024-06-11T13:29:26.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:26.325+0000 7f081f7fe640 1 -- 172.21.15.57:0/455358295 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f0820001ce0 con 0x7f0830107850 2024-06-11T13:29:26.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:26.325+0000 7f081f7fe640 1 -- 172.21.15.57:0/455358295 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f0820005bc0 con 0x7f0830107850 2024-06-11T13:29:26.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:26.326+0000 7f0834d13640 1 -- 172.21.15.57:0/455358295 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08100050b0 con 0x7f0830107850 2024-06-11T13:29:26.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:26.327+0000 7f081f7fe640 1 -- 172.21.15.57:0/455358295 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 22) ==== 92138+0+0 (unknown 311317106 0 0) 0x7f0820003fc0 con 0x7f0830107850 2024-06-11T13:29:26.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:26.327+0000 7f081f7fe640 1 -- 172.21.15.57:0/455358295 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(70..70 src has 1..70) ==== 6172+0+0 (unknown 3813449920 0 0) 0x7f0820089540 con 0x7f0830107850 2024-06-11T13:29:26.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:26.332+0000 7f081f7fe640 1 -- 172.21.15.57:0/455358295 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f0820056ca0 con 0x7f0830107850 2024-06-11T13:29:26.413 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:26 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3330234232' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-11T13:29:26.414 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:26 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/96194509' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-11T13:29:26.414 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:26 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3330234232' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-11T13:29:26.414 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:26 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/96194509' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-11T13:29:26.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:26.519+0000 7f0834d13640 1 -- 172.21.15.57:0/455358295 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f08100053a0 con 0x7f0830107850 2024-06-11T13:29:26.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:26.520+0000 7f081f7fe640 1 -- 172.21.15.57:0/455358295 <== mon.0 v1:172.21.15.57:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (unknown 92182286 0 3833507532) 0x7f082005a950 con 0x7f0830107850 2024-06-11T13:29:26.516 INFO:teuthology.orchestra.run.smithi057.stdout:77309411371 2024-06-11T13:29:26.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:26.522+0000 7f0834d13640 1 -- 172.21.15.57:0/455358295 >> v1:172.21.15.57:6800/2077416415 conn(0x7f081806ec30 legacy=0x7f08180710f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:26.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:26.522+0000 7f0834d13640 1 -- 172.21.15.57:0/455358295 >> v1:172.21.15.57:6789/0 conn(0x7f0830107850 legacy=0x7f083010da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:26.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:26.522+0000 7f0834d13640 1 -- 172.21.15.57:0/455358295 shutdown_connections 2024-06-11T13:29:26.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:26.522+0000 7f0834d13640 1 -- 172.21.15.57:0/455358295 >> 172.21.15.57:0/455358295 conn(0x7f08300fd340 msgr2=0x7f08300fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:29:26.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:26.522+0000 7f0834d13640 1 -- 172.21.15.57:0/455358295 shutdown_connections 2024-06-11T13:29:26.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:26.522+0000 7f0834d13640 1 -- 172.21.15.57:0/455358295 wait complete. 2024-06-11T13:29:26.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:26 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3330234232' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-11T13:29:26.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:26 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/96194509' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-11T13:29:26.758 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574891 got 55834574896 for osd.1 2024-06-11T13:29:26.759 DEBUG:teuthology.parallel:result is None 2024-06-11T13:29:27.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:27 smithi057 ceph-mon[29805]: pgmap v21: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:29:27.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:27 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/455358295' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-11T13:29:27.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:27 smithi057 ceph-mon[40588]: pgmap v21: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:29:27.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:27 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/455358295' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-11T13:29:27.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:27 smithi107 ceph-mon[33178]: pgmap v21: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:29:27.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:27 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/455358295' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-11T13:29:28.033 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 13:29:27 smithi057 podman[74388]: 2024-06-11 13:29:27.893206712 +0000 UTC m=+2.169539528 volume create 424b7d248055bfb7c82fb5ee2c32e0eefd512d0582cc2690a7d86d07a14c02a5 2024-06-11T13:29:28.687 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462933 got 201863462936 for osd.6 2024-06-11T13:29:28.687 DEBUG:teuthology.parallel:result is None 2024-06-11T13:29:28.719 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.a/config 2024-06-11T13:29:28.720 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.a/config 2024-06-11T13:29:29.426 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:29 smithi057 ceph-mon[29805]: pgmap v22: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:29:29.427 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:29 smithi057 ceph-mon[40588]: pgmap v22: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:29:29.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:29 smithi107 ceph-mon[33178]: pgmap v22: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:29:29.783 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 13:29:29 smithi057 podman[74388]: 2024-06-11 13:29:29.537881117 +0000 UTC m=+3.814213883 container create 86927cc63d428c7777ba9b29d132b989e2740c976814c98ed7d5b4c06dc30a43 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-11T13:29:29.929 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411367 got 77309411371 for osd.2 2024-06-11T13:29:29.929 DEBUG:teuthology.parallel:result is None 2024-06-11T13:29:30.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:30.829+0000 7f7478cf9640 1 -- 172.21.15.57:0/3513198209 >> v1:172.21.15.57:6790/0 conn(0x7f747410d170 legacy=0x7f747410f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:30.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:30.829+0000 7f7478cf9640 1 -- 172.21.15.57:0/3513198209 shutdown_connections 2024-06-11T13:29:30.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:30.829+0000 7f7478cf9640 1 -- 172.21.15.57:0/3513198209 >> 172.21.15.57:0/3513198209 conn(0x7f74740fc230 msgr2=0x7f74740fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:29:30.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:30.829+0000 7f7478cf9640 1 -- 172.21.15.57:0/3513198209 shutdown_connections 2024-06-11T13:29:30.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:30.829+0000 7f7478cf9640 1 -- 172.21.15.57:0/3513198209 wait complete. 2024-06-11T13:29:30.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:30.830+0000 7f7478cf9640 1 Processor -- start 2024-06-11T13:29:30.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:30.830+0000 7f7478cf9640 1 -- start start 2024-06-11T13:29:30.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:30.830+0000 7f7478cf9640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f74741a8ae0 con 0x7f747410d170 2024-06-11T13:29:30.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:30.830+0000 7f7478cf9640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f74741a9d20 con 0x7f7474106720 2024-06-11T13:29:30.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:30.830+0000 7f7478cf9640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f74741aaf60 con 0x7f74741095d0 2024-06-11T13:29:30.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:30.830+0000 7f7471d9b640 1 --1- >> v1:172.21.15.57:6790/0 conn(0x7f74741095d0 0x7f74741a39e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6790/0 says I am v1:172.21.15.57:48466/0 (socket says 172.21.15.57:48466) 2024-06-11T13:29:30.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:30.830+0000 7f7471d9b640 1 -- 172.21.15.57:0/2789156365 learned_addr learned my addr 172.21.15.57:0/2789156365 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:29:30.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:30.830+0000 7f74637fe640 1 -- 172.21.15.57:0/2789156365 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1666515646 0 0) 0x7f74741aaf60 con 0x7f74741095d0 2024-06-11T13:29:30.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:30.830+0000 7f74637fe640 1 -- 172.21.15.57:0/2789156365 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f745c003520 con 0x7f74741095d0 2024-06-11T13:29:30.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:30.830+0000 7f74637fe640 1 -- 172.21.15.57:0/2789156365 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 662071605 0 0) 0x7f74741a8ae0 con 0x7f747410d170 2024-06-11T13:29:30.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:30.830+0000 7f74637fe640 1 -- 172.21.15.57:0/2789156365 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f74741aaf60 con 0x7f747410d170 2024-06-11T13:29:30.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:30.830+0000 7f74637fe640 1 -- 172.21.15.57:0/2789156365 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3937185670 0 0) 0x7f74741a9d20 con 0x7f7474106720 2024-06-11T13:29:30.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:30.830+0000 7f74637fe640 1 -- 172.21.15.57:0/2789156365 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f74741a8ae0 con 0x7f7474106720 2024-06-11T13:29:30.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:30.831+0000 7f74637fe640 1 -- 172.21.15.57:0/2789156365 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1302672407 0 0) 0x7f745c003520 con 0x7f74741095d0 2024-06-11T13:29:30.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:30.831+0000 7f74637fe640 1 -- 172.21.15.57:0/2789156365 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f74741a9d20 con 0x7f74741095d0 2024-06-11T13:29:30.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:30.831+0000 7f74637fe640 1 -- 172.21.15.57:0/2789156365 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f7468001a30 con 0x7f74741095d0 2024-06-11T13:29:30.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:30.831+0000 7f74637fe640 1 -- 172.21.15.57:0/2789156365 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2798172383 0 0) 0x7f74741aaf60 con 0x7f747410d170 2024-06-11T13:29:30.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:30.831+0000 7f74637fe640 1 -- 172.21.15.57:0/2789156365 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f745c003520 con 0x7f747410d170 2024-06-11T13:29:30.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:30.831+0000 7f74637fe640 1 -- 172.21.15.57:0/2789156365 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 72793769 0 0) 0x7f74741a8ae0 con 0x7f7474106720 2024-06-11T13:29:30.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:30.831+0000 7f74637fe640 1 -- 172.21.15.57:0/2789156365 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f74741aaf60 con 0x7f7474106720 2024-06-11T13:29:30.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:30.831+0000 7f74637fe640 1 -- 172.21.15.57:0/2789156365 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f746c0016f0 con 0x7f747410d170 2024-06-11T13:29:30.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:30.831+0000 7f74637fe640 1 -- 172.21.15.57:0/2789156365 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f74640017f0 con 0x7f7474106720 2024-06-11T13:29:30.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:30.831+0000 7f74637fe640 1 -- 172.21.15.57:0/2789156365 <== mon.2 v1:172.21.15.57:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 347031162 0 0) 0x7f74741a9d20 con 0x7f74741095d0 2024-06-11T13:29:30.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:30.831+0000 7f74637fe640 1 -- 172.21.15.57:0/2789156365 >> v1:172.21.15.107:6789/0 conn(0x7f7474106720 legacy=0x7f747410c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:30.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:30.831+0000 7f74637fe640 1 -- 172.21.15.57:0/2789156365 >> v1:172.21.15.57:6789/0 conn(0x7f747410d170 legacy=0x7f74741a71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:30.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:30.831+0000 7f74637fe640 1 -- 172.21.15.57:0/2789156365 --> v1:172.21.15.57:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74741ac1a0 con 0x7f74741095d0 2024-06-11T13:29:30.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:30.831+0000 7f7478cf9640 1 -- 172.21.15.57:0/2789156365 --> v1:172.21.15.57:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f74741a8d10 con 0x7f74741095d0 2024-06-11T13:29:30.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:30.831+0000 7f7478cf9640 1 -- 172.21.15.57:0/2789156365 --> v1:172.21.15.57:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f74741a91d0 con 0x7f74741095d0 2024-06-11T13:29:30.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:30.831+0000 7f74637fe640 1 -- 172.21.15.57:0/2789156365 <== mon.2 v1:172.21.15.57:6790/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f7468001bd0 con 0x7f74741095d0 2024-06-11T13:29:30.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:30.831+0000 7f74637fe640 1 -- 172.21.15.57:0/2789156365 <== mon.2 v1:172.21.15.57:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f7468005ba0 con 0x7f74741095d0 2024-06-11T13:29:30.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:30.832+0000 7f7478cf9640 1 -- 172.21.15.57:0/2789156365 --> v1:172.21.15.57:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7454005080 con 0x7f74741095d0 2024-06-11T13:29:30.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:30.833+0000 7f74637fe640 1 -- 172.21.15.57:0/2789156365 <== mon.2 v1:172.21.15.57:6790/0 7 ==== mgrmap(e 22) ==== 92138+0+0 (unknown 311317106 0 0) 0x7f74680038a0 con 0x7f74741095d0 2024-06-11T13:29:30.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:30.833+0000 7f74637fe640 1 -- 172.21.15.57:0/2789156365 <== mon.2 v1:172.21.15.57:6790/0 8 ==== osd_map(70..70 src has 1..70) ==== 6172+0+0 (unknown 3813449920 0 0) 0x7f7468089470 con 0x7f74741095d0 2024-06-11T13:29:30.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:30.838+0000 7f74637fe640 1 -- 172.21.15.57:0/2789156365 <== mon.2 v1:172.21.15.57:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f7468056c30 con 0x7f74741095d0 2024-06-11T13:29:31.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:31.031+0000 7f7478cf9640 1 -- 172.21.15.57:0/2789156365 --> v1:172.21.15.57:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f7454005370 con 0x7f74741095d0 2024-06-11T13:29:31.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:31.032+0000 7f74637fe640 1 -- 172.21.15.57:0/2789156365 <== mon.2 v1:172.21.15.57:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (unknown 574334944 0 1068569428) 0x7f746805a8e0 con 0x7f74741095d0 2024-06-11T13:29:31.029 INFO:teuthology.orchestra.run.smithi057.stdout:34359738422 2024-06-11T13:29:31.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:31.034+0000 7f7478cf9640 1 -- 172.21.15.57:0/2789156365 >> v1:172.21.15.57:6800/2077416415 conn(0x7f745c06ee60 legacy=0x7f745c071320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:31.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:31.034+0000 7f7478cf9640 1 -- 172.21.15.57:0/2789156365 >> v1:172.21.15.57:6790/0 conn(0x7f74741095d0 legacy=0x7f74741a39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:31.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:31.034+0000 7f7478cf9640 1 -- 172.21.15.57:0/2789156365 shutdown_connections 2024-06-11T13:29:31.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:31.034+0000 7f7478cf9640 1 -- 172.21.15.57:0/2789156365 >> 172.21.15.57:0/2789156365 conn(0x7f74740fc230 msgr2=0x7f7474110690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:29:31.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:31.034+0000 7f7478cf9640 1 -- 172.21.15.57:0/2789156365 shutdown_connections 2024-06-11T13:29:31.031 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:31.034+0000 7f7478cf9640 1 -- 172.21.15.57:0/2789156365 wait complete. 2024-06-11T13:29:31.159 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 13:29:30 smithi057 podman[74388]: 2024-06-11 13:29:30.942538978 +0000 UTC m=+5.218871755 container init 86927cc63d428c7777ba9b29d132b989e2740c976814c98ed7d5b4c06dc30a43 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-11T13:29:31.159 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 13:29:30 smithi057 podman[74388]: 2024-06-11 13:29:30.945667586 +0000 UTC m=+5.222000367 container start 86927cc63d428c7777ba9b29d132b989e2740c976814c98ed7d5b4c06dc30a43 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-11T13:29:31.160 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 13:29:30 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-alertmanager-a[74565]: ts=2024-06-11T13:29:30.961Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-06-11T13:29:31.160 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 13:29:30 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-alertmanager-a[74565]: ts=2024-06-11T13:29:30.961Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-06-11T13:29:31.160 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 13:29:30 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-alertmanager-a[74565]: ts=2024-06-11T13:29:30.962Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.57 port=9094 2024-06-11T13:29:31.160 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 13:29:30 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-alertmanager-a[74565]: ts=2024-06-11T13:29:30.964Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-06-11T13:29:31.160 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 13:29:30 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-alertmanager-a[74565]: ts=2024-06-11T13:29:30.986Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-06-11T13:29:31.160 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 13:29:30 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-alertmanager-a[74565]: ts=2024-06-11T13:29:30.987Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-06-11T13:29:31.160 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 13:29:30 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-alertmanager-a[74565]: ts=2024-06-11T13:29:30.991Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-06-11T13:29:31.160 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 13:29:30 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-alertmanager-a[74565]: ts=2024-06-11T13:29:30.991Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-06-11T13:29:31.160 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 13:29:31 smithi057 bash[74388]: 86927cc63d428c7777ba9b29d132b989e2740c976814c98ed7d5b4c06dc30a43 2024-06-11T13:29:31.160 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 13:29:31 smithi057 systemd[1]: Started Ceph alertmanager.a for 98a6635e-27f5-11ef-bc9b-c7b262605968. 2024-06-11T13:29:31.296 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:29:30 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:29:31.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:31 smithi107 ceph-mon[33178]: pgmap v23: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:29:31.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:31 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2789156365' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-11T13:29:31.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:31 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:31.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:31 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:31.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:31 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:31.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:31 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:31.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:31 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:31.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:31 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:31.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:31 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-06-11T13:29:31.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:31 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:31.466 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:31 smithi057 ceph-mon[40588]: pgmap v23: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:29:31.467 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:31 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2789156365' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-11T13:29:31.467 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:31 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:31.467 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:31 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:31.467 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:31 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:31.467 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:31 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:31.467 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:31 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:31.467 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:31 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:31.467 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:31 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-06-11T13:29:31.467 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:31 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:31.468 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:31 smithi057 ceph-mon[29805]: pgmap v23: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:29:31.468 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:31 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2789156365' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-11T13:29:31.468 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:31 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:31.468 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:31 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:31.468 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:31 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:31.468 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:31 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:31.468 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:31 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:31.468 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:31 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:31.468 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:31 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-06-11T13:29:31.469 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:31 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:31.469 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:29:31 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: [11/Jun/2024:13:29:31] ENGINE Bus STOPPING 2024-06-11T13:29:31.469 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:29:31 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: [11/Jun/2024:13:29:31] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-06-11T13:29:31.469 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:29:31 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: [11/Jun/2024:13:29:31] ENGINE Bus STOPPED 2024-06-11T13:29:31.469 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:29:31 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: [11/Jun/2024:13:29:31] ENGINE Bus STARTING 2024-06-11T13:29:31.469 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:29:31 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: [11/Jun/2024:13:29:31] ENGINE Serving on http://:::9283 2024-06-11T13:29:31.469 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:29:31 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: [11/Jun/2024:13:29:31] ENGINE Bus STARTED 2024-06-11T13:29:32.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:32.310+0000 7fe16fb6a640 1 -- 172.21.15.57:0/4006798041 >> v1:172.21.15.57:6789/0 conn(0x7fe16810e290 legacy=0x7fe168110750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:32.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:32.310+0000 7fe16fb6a640 1 -- 172.21.15.57:0/4006798041 shutdown_connections 2024-06-11T13:29:32.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:32.310+0000 7fe16fb6a640 1 -- 172.21.15.57:0/4006798041 >> 172.21.15.57:0/4006798041 conn(0x7fe1680fd350 msgr2=0x7fe1680ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:29:32.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:32.311+0000 7fe16fb6a640 1 -- 172.21.15.57:0/4006798041 shutdown_connections 2024-06-11T13:29:32.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:32.311+0000 7fe16fb6a640 1 -- 172.21.15.57:0/4006798041 wait complete. 2024-06-11T13:29:32.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:32.311+0000 7fe16fb6a640 1 Processor -- start 2024-06-11T13:29:32.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:32.311+0000 7fe16fb6a640 1 -- start start 2024-06-11T13:29:32.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:32.311+0000 7fe16fb6a640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fe1681a6860 con 0x7fe16810e290 2024-06-11T13:29:32.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:32.311+0000 7fe16fb6a640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fe1681a7a60 con 0x7fe16810a6f0 2024-06-11T13:29:32.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:32.311+0000 7fe16fb6a640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fe1681a8ca0 con 0x7fe168107840 2024-06-11T13:29:32.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:32.311+0000 7fe16d906640 1 --1- >> v1:172.21.15.57:6790/0 conn(0x7fe168107840 0x7fe16810da60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6790/0 says I am v1:172.21.15.57:48480/0 (socket says 172.21.15.57:48480) 2024-06-11T13:29:32.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:32.311+0000 7fe16d906640 1 -- 172.21.15.57:0/1445190122 learned_addr learned my addr 172.21.15.57:0/1445190122 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:29:32.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:32.312+0000 7fe15effd640 1 -- 172.21.15.57:0/1445190122 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2673575215 0 0) 0x7fe1681a8ca0 con 0x7fe168107840 2024-06-11T13:29:32.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:32.312+0000 7fe15effd640 1 -- 172.21.15.57:0/1445190122 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe154003160 con 0x7fe168107840 2024-06-11T13:29:32.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:32.312+0000 7fe15effd640 1 -- 172.21.15.57:0/1445190122 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3366779028 0 0) 0x7fe1681a7a60 con 0x7fe16810a6f0 2024-06-11T13:29:32.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:32.312+0000 7fe15effd640 1 -- 172.21.15.57:0/1445190122 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1681a8ca0 con 0x7fe16810a6f0 2024-06-11T13:29:32.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:32.312+0000 7fe15effd640 1 -- 172.21.15.57:0/1445190122 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1674199493 0 0) 0x7fe1681a6860 con 0x7fe16810e290 2024-06-11T13:29:32.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:32.312+0000 7fe15effd640 1 -- 172.21.15.57:0/1445190122 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1681a7a60 con 0x7fe16810e290 2024-06-11T13:29:32.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:32.312+0000 7fe15effd640 1 -- 172.21.15.57:0/1445190122 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 150191192 0 0) 0x7fe1681a8ca0 con 0x7fe16810a6f0 2024-06-11T13:29:32.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:32.312+0000 7fe15effd640 1 -- 172.21.15.57:0/1445190122 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe1681a6860 con 0x7fe16810a6f0 2024-06-11T13:29:32.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:32.312+0000 7fe15effd640 1 -- 172.21.15.57:0/1445190122 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3709737776 0 0) 0x7fe154003160 con 0x7fe168107840 2024-06-11T13:29:32.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:32.312+0000 7fe15effd640 1 -- 172.21.15.57:0/1445190122 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe1681a8ca0 con 0x7fe168107840 2024-06-11T13:29:32.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:32.312+0000 7fe15effd640 1 -- 172.21.15.57:0/1445190122 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fe158001870 con 0x7fe16810a6f0 2024-06-11T13:29:32.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:32.312+0000 7fe15effd640 1 -- 172.21.15.57:0/1445190122 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3591912881 0 0) 0x7fe1681a7a60 con 0x7fe16810e290 2024-06-11T13:29:32.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:32.312+0000 7fe15effd640 1 -- 172.21.15.57:0/1445190122 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe154003160 con 0x7fe16810e290 2024-06-11T13:29:32.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:32.312+0000 7fe15effd640 1 -- 172.21.15.57:0/1445190122 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fe164001890 con 0x7fe168107840 2024-06-11T13:29:32.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:32.312+0000 7fe15effd640 1 -- 172.21.15.57:0/1445190122 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fe160001c80 con 0x7fe16810e290 2024-06-11T13:29:32.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:32.312+0000 7fe15effd640 1 -- 172.21.15.57:0/1445190122 <== mon.1 v1:172.21.15.107:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2969458325 0 0) 0x7fe1681a6860 con 0x7fe16810a6f0 2024-06-11T13:29:32.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:32.312+0000 7fe15effd640 1 -- 172.21.15.57:0/1445190122 >> v1:172.21.15.57:6790/0 conn(0x7fe168107840 legacy=0x7fe16810da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:32.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:32.312+0000 7fe15effd640 1 -- 172.21.15.57:0/1445190122 >> v1:172.21.15.57:6789/0 conn(0x7fe16810e290 legacy=0x7fe1681a4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:32.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:32.312+0000 7fe15effd640 1 -- 172.21.15.57:0/1445190122 --> v1:172.21.15.107:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1681a9ee0 con 0x7fe16810a6f0 2024-06-11T13:29:32.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:32.312+0000 7fe16fb6a640 1 -- 172.21.15.57:0/1445190122 --> v1:172.21.15.107:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fe1681a7c90 con 0x7fe16810a6f0 2024-06-11T13:29:32.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:32.313+0000 7fe15effd640 1 -- 172.21.15.57:0/1445190122 <== mon.1 v1:172.21.15.107:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7fe158001a10 con 0x7fe16810a6f0 2024-06-11T13:29:32.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:32.313+0000 7fe15effd640 1 -- 172.21.15.57:0/1445190122 <== mon.1 v1:172.21.15.107:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fe158004950 con 0x7fe16810a6f0 2024-06-11T13:29:32.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:32.313+0000 7fe16fb6a640 1 -- 172.21.15.57:0/1445190122 --> v1:172.21.15.107:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fe1681a8150 con 0x7fe16810a6f0 2024-06-11T13:29:32.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:32.314+0000 7fe16fb6a640 1 -- 172.21.15.57:0/1445190122 --> v1:172.21.15.107:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe168102f80 con 0x7fe16810a6f0 2024-06-11T13:29:32.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:32.314+0000 7fe15effd640 1 -- 172.21.15.57:0/1445190122 <== mon.1 v1:172.21.15.107:6789/0 7 ==== mgrmap(e 22) ==== 92138+0+0 (unknown 311317106 0 0) 0x7fe15801b130 con 0x7fe16810a6f0 2024-06-11T13:29:32.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:32.315+0000 7fe15effd640 1 -- 172.21.15.57:0/1445190122 <== mon.1 v1:172.21.15.107:6789/0 8 ==== osd_map(70..70 src has 1..70) ==== 6172+0+0 (unknown 3813449920 0 0) 0x7fe158088150 con 0x7fe16810a6f0 2024-06-11T13:29:32.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:32.319+0000 7fe15effd640 1 -- 172.21.15.57:0/1445190122 <== mon.1 v1:172.21.15.107:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7fe1580558b0 con 0x7fe16810a6f0 2024-06-11T13:29:32.451 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:32 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:29:32.451 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:32 smithi057 ceph-mon[40588]: Regenerating cephadm self-signed grafana TLS certificates 2024-06-11T13:29:32.451 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:32 smithi057 ceph-mon[40588]: from='mon.? v1:172.21.15.57:0/45435016' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-06-11T13:29:32.451 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:32 smithi057 ceph-mon[40588]: Deploying daemon grafana.a on smithi107 2024-06-11T13:29:32.452 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:32 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:29:32.452 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:32 smithi057 ceph-mon[29805]: Regenerating cephadm self-signed grafana TLS certificates 2024-06-11T13:29:32.452 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:32 smithi057 ceph-mon[29805]: from='mon.? v1:172.21.15.57:0/45435016' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-06-11T13:29:32.452 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:32 smithi057 ceph-mon[29805]: Deploying daemon grafana.a on smithi107 2024-06-11T13:29:32.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:32.513+0000 7fe16fb6a640 1 -- 172.21.15.57:0/1445190122 --> v1:172.21.15.107:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fe168068360 con 0x7fe16810a6f0 2024-06-11T13:29:32.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:32.513+0000 7fe15effd640 1 -- 172.21.15.57:0/1445190122 <== mon.1 v1:172.21.15.107:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (unknown 2131975755 0 1292561331) 0x7fe158059560 con 0x7fe16810a6f0 2024-06-11T13:29:32.510 INFO:teuthology.orchestra.run.smithi057.stdout:167503724574 2024-06-11T13:29:32.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:32.516+0000 7fe16fb6a640 1 -- 172.21.15.57:0/1445190122 >> v1:172.21.15.57:6800/2077416415 conn(0x7fe15406ef50 legacy=0x7fe154071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:32.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:32.516+0000 7fe16fb6a640 1 -- 172.21.15.57:0/1445190122 >> v1:172.21.15.107:6789/0 conn(0x7fe16810a6f0 legacy=0x7fe1681a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:32.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:32.516+0000 7fe16fb6a640 1 -- 172.21.15.57:0/1445190122 shutdown_connections 2024-06-11T13:29:32.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:32.516+0000 7fe16fb6a640 1 -- 172.21.15.57:0/1445190122 >> 172.21.15.57:0/1445190122 conn(0x7fe1680fd350 msgr2=0x7fe1680fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:29:32.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:32.516+0000 7fe16fb6a640 1 -- 172.21.15.57:0/1445190122 shutdown_connections 2024-06-11T13:29:32.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:32.516+0000 7fe16fb6a640 1 -- 172.21.15.57:0/1445190122 wait complete. 2024-06-11T13:29:32.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:32 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:29:32.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:32 smithi107 ceph-mon[33178]: Regenerating cephadm self-signed grafana TLS certificates 2024-06-11T13:29:32.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:32 smithi107 ceph-mon[33178]: from='mon.? v1:172.21.15.57:0/45435016' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-06-11T13:29:32.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:32 smithi107 ceph-mon[33178]: Deploying daemon grafana.a on smithi107 2024-06-11T13:29:32.756 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738418 got 34359738422 for osd.0 2024-06-11T13:29:32.756 DEBUG:teuthology.parallel:result is None 2024-06-11T13:29:33.033 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 13:29:32 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-alertmanager-a[74565]: ts=2024-06-11T13:29:32.964Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000594129s 2024-06-11T13:29:33.341 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724569 got 167503724574 for osd.5 2024-06-11T13:29:33.341 DEBUG:teuthology.parallel:result is None 2024-06-11T13:29:33.441 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:33 smithi057 ceph-mon[29805]: pgmap v24: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:29:33.441 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:33 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1445190122' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-11T13:29:33.441 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:33 smithi057 ceph-mon[40588]: pgmap v24: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:29:33.441 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:33 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1445190122' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-11T13:29:33.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:33 smithi107 ceph-mon[33178]: pgmap v24: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:29:33.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:33 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1445190122' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-11T13:29:34.033 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:29:33 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:29:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:29:34.310 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:34 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3649977071' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-11T13:29:34.311 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:34 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3649977071' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-11T13:29:34.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:34 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3649977071' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-11T13:29:34.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.536+0000 7f2c2cbf0640 1 -- 172.21.15.57:0/3383099638 >> v1:172.21.15.107:6789/0 conn(0x7f2c28106720 legacy=0x7f2c28106b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:34.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.536+0000 7f2c2cbf0640 1 -- 172.21.15.57:0/3383099638 shutdown_connections 2024-06-11T13:29:34.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.536+0000 7f2c2cbf0640 1 -- 172.21.15.57:0/3383099638 >> 172.21.15.57:0/3383099638 conn(0x7f2c280fc230 msgr2=0x7f2c280fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:29:34.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.536+0000 7f2c2cbf0640 1 -- 172.21.15.57:0/3383099638 shutdown_connections 2024-06-11T13:29:34.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.536+0000 7f2c2cbf0640 1 -- 172.21.15.57:0/3383099638 wait complete. 2024-06-11T13:29:34.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.537+0000 7f2c2cbf0640 1 Processor -- start 2024-06-11T13:29:34.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.537+0000 7f2c2cbf0640 1 -- start start 2024-06-11T13:29:34.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.537+0000 7f2c2cbf0640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f2c281a89f0 con 0x7f2c28106720 2024-06-11T13:29:34.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.537+0000 7f2c2cbf0640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f2c281a9c30 con 0x7f2c2810d170 2024-06-11T13:29:34.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.537+0000 7f2c2cbf0640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f2c281aae70 con 0x7f2c281095d0 2024-06-11T13:29:34.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.537+0000 7f2c2659c640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f2c28106720 0x7f2c2810c990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:35332/0 (socket says 172.21.15.57:35332) 2024-06-11T13:29:34.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.537+0000 7f2c25d9b640 1 --1- >> v1:172.21.15.57:6790/0 conn(0x7f2c281095d0 0x7f2c281a39e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6790/0 says I am v1:172.21.15.57:48508/0 (socket says 172.21.15.57:48508) 2024-06-11T13:29:34.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.537+0000 7f2c2659c640 1 -- 172.21.15.57:0/3649977071 learned_addr learned my addr 172.21.15.57:0/3649977071 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:29:34.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.537+0000 7f2c177fe640 1 -- 172.21.15.57:0/3649977071 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1158783877 0 0) 0x7f2c281aae70 con 0x7f2c281095d0 2024-06-11T13:29:34.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.537+0000 7f2c177fe640 1 -- 172.21.15.57:0/3649977071 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2c10003160 con 0x7f2c281095d0 2024-06-11T13:29:34.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.537+0000 7f2c177fe640 1 -- 172.21.15.57:0/3649977071 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4151478202 0 0) 0x7f2c281a89f0 con 0x7f2c28106720 2024-06-11T13:29:34.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.537+0000 7f2c177fe640 1 -- 172.21.15.57:0/3649977071 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2c281aae70 con 0x7f2c28106720 2024-06-11T13:29:34.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.537+0000 7f2c177fe640 1 -- 172.21.15.57:0/3649977071 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1303449873 0 0) 0x7f2c281a9c30 con 0x7f2c2810d170 2024-06-11T13:29:34.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.537+0000 7f2c177fe640 1 -- 172.21.15.57:0/3649977071 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2c281a89f0 con 0x7f2c2810d170 2024-06-11T13:29:34.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.538+0000 7f2c177fe640 1 -- 172.21.15.57:0/3649977071 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2455720911 0 0) 0x7f2c10003160 con 0x7f2c281095d0 2024-06-11T13:29:34.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.538+0000 7f2c177fe640 1 -- 172.21.15.57:0/3649977071 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2c281a9c30 con 0x7f2c281095d0 2024-06-11T13:29:34.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.538+0000 7f2c177fe640 1 -- 172.21.15.57:0/3649977071 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f2c1c001130 con 0x7f2c281095d0 2024-06-11T13:29:34.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.538+0000 7f2c177fe640 1 -- 172.21.15.57:0/3649977071 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 973832296 0 0) 0x7f2c281aae70 con 0x7f2c28106720 2024-06-11T13:29:34.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.538+0000 7f2c177fe640 1 -- 172.21.15.57:0/3649977071 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2c10003160 con 0x7f2c28106720 2024-06-11T13:29:34.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.538+0000 7f2c177fe640 1 -- 172.21.15.57:0/3649977071 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f2c18001860 con 0x7f2c28106720 2024-06-11T13:29:34.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.538+0000 7f2c177fe640 1 -- 172.21.15.57:0/3649977071 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4127511940 0 0) 0x7f2c281a89f0 con 0x7f2c2810d170 2024-06-11T13:29:34.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.538+0000 7f2c177fe640 1 -- 172.21.15.57:0/3649977071 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2c281aae70 con 0x7f2c2810d170 2024-06-11T13:29:34.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.538+0000 7f2c177fe640 1 -- 172.21.15.57:0/3649977071 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f2c200013d0 con 0x7f2c2810d170 2024-06-11T13:29:34.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.538+0000 7f2c177fe640 1 -- 172.21.15.57:0/3649977071 <== mon.2 v1:172.21.15.57:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3035322567 0 0) 0x7f2c281a9c30 con 0x7f2c281095d0 2024-06-11T13:29:34.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.538+0000 7f2c177fe640 1 -- 172.21.15.57:0/3649977071 >> v1:172.21.15.107:6789/0 conn(0x7f2c2810d170 legacy=0x7f2c281a7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:34.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.538+0000 7f2c177fe640 1 -- 172.21.15.57:0/3649977071 >> v1:172.21.15.57:6789/0 conn(0x7f2c28106720 legacy=0x7f2c2810c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:34.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.538+0000 7f2c177fe640 1 -- 172.21.15.57:0/3649977071 --> v1:172.21.15.57:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c281ac0b0 con 0x7f2c281095d0 2024-06-11T13:29:34.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.538+0000 7f2c2cbf0640 1 -- 172.21.15.57:0/3649977071 --> v1:172.21.15.57:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f2c281a8c20 con 0x7f2c281095d0 2024-06-11T13:29:34.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.538+0000 7f2c2cbf0640 1 -- 172.21.15.57:0/3649977071 --> v1:172.21.15.57:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f2c281a90e0 con 0x7f2c281095d0 2024-06-11T13:29:34.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.538+0000 7f2c177fe640 1 -- 172.21.15.57:0/3649977071 <== mon.2 v1:172.21.15.57:6790/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f2c1c003a80 con 0x7f2c281095d0 2024-06-11T13:29:34.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.538+0000 7f2c177fe640 1 -- 172.21.15.57:0/3649977071 <== mon.2 v1:172.21.15.57:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f2c1c005bc0 con 0x7f2c281095d0 2024-06-11T13:29:34.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.539+0000 7f2c2cbf0640 1 -- 172.21.15.57:0/3649977071 --> v1:172.21.15.57:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c08005080 con 0x7f2c281095d0 2024-06-11T13:29:34.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.540+0000 7f2c177fe640 1 -- 172.21.15.57:0/3649977071 <== mon.2 v1:172.21.15.57:6790/0 7 ==== mgrmap(e 22) ==== 92138+0+0 (unknown 311317106 0 0) 0x7f2c1c0035f0 con 0x7f2c281095d0 2024-06-11T13:29:34.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.540+0000 7f2c177fe640 1 -- 172.21.15.57:0/3649977071 <== mon.2 v1:172.21.15.57:6790/0 8 ==== osd_map(70..70 src has 1..70) ==== 6172+0+0 (unknown 3813449920 0 0) 0x7f2c1c089200 con 0x7f2c281095d0 2024-06-11T13:29:34.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.544+0000 7f2c177fe640 1 -- 172.21.15.57:0/3649977071 <== mon.2 v1:172.21.15.57:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f2c1c056aa0 con 0x7f2c281095d0 2024-06-11T13:29:34.653 INFO:teuthology.orchestra.run.smithi057.stdout:107374182440 2024-06-11T13:29:34.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.732+0000 7f2c2cbf0640 1 -- 172.21.15.57:0/3649977071 --> v1:172.21.15.57:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f2c08005370 con 0x7f2c281095d0 2024-06-11T13:29:34.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.733+0000 7f2c177fe640 1 -- 172.21.15.57:0/3649977071 <== mon.2 v1:172.21.15.57:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (unknown 383520633 0 1087398483) 0x7f2c1c05a750 con 0x7f2c281095d0 2024-06-11T13:29:34.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.735+0000 7f2c2cbf0640 1 -- 172.21.15.57:0/3649977071 >> v1:172.21.15.57:6800/2077416415 conn(0x7f2c1006eef0 legacy=0x7f2c100713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:34.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.735+0000 7f2c2cbf0640 1 -- 172.21.15.57:0/3649977071 >> v1:172.21.15.57:6790/0 conn(0x7f2c281095d0 legacy=0x7f2c281a39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:34.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.735+0000 7f2c2cbf0640 1 -- 172.21.15.57:0/3649977071 shutdown_connections 2024-06-11T13:29:34.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.735+0000 7f2c2cbf0640 1 -- 172.21.15.57:0/3649977071 >> 172.21.15.57:0/3649977071 conn(0x7f2c280fc230 msgr2=0x7f2c2806a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:29:34.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.735+0000 7f2c2cbf0640 1 -- 172.21.15.57:0/3649977071 shutdown_connections 2024-06-11T13:29:34.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:33.735+0000 7f2c2cbf0640 1 -- 172.21.15.57:0/3649977071 wait complete. 2024-06-11T13:29:35.228 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182435 got 107374182440 for osd.3 2024-06-11T13:29:35.229 DEBUG:teuthology.parallel:result is None 2024-06-11T13:29:35.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:35 smithi057 ceph-mon[29805]: pgmap v25: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:29:35.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:35 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:35.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:35 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:29:35.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:35 smithi057 ceph-mon[40588]: pgmap v25: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:29:35.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:35 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:35.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:35 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:29:35.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:35 smithi107 ceph-mon[33178]: pgmap v25: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:29:35.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:35 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:35.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:35 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:29:35.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:35.801+0000 7fddfc3d7640 1 -- 172.21.15.57:0/1214138885 >> v1:172.21.15.57:6790/0 conn(0x7fddf4107840 legacy=0x7fddf4107c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:35.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:35.801+0000 7fddfc3d7640 1 -- 172.21.15.57:0/1214138885 shutdown_connections 2024-06-11T13:29:35.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:35.801+0000 7fddfc3d7640 1 -- 172.21.15.57:0/1214138885 >> 172.21.15.57:0/1214138885 conn(0x7fddf40fd350 msgr2=0x7fddf40ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:29:35.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:35.801+0000 7fddfc3d7640 1 -- 172.21.15.57:0/1214138885 shutdown_connections 2024-06-11T13:29:35.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:35.801+0000 7fddfc3d7640 1 -- 172.21.15.57:0/1214138885 wait complete. 2024-06-11T13:29:35.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:35.802+0000 7fddfc3d7640 1 Processor -- start 2024-06-11T13:29:35.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:35.802+0000 7fddfc3d7640 1 -- start start 2024-06-11T13:29:35.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:35.802+0000 7fddfc3d7640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fddf41a8ad0 con 0x7fddf410a6f0 2024-06-11T13:29:35.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:35.802+0000 7fddfc3d7640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fddf41a9cf0 con 0x7fddf4107840 2024-06-11T13:29:35.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:35.802+0000 7fddfc3d7640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fddf41aaf30 con 0x7fddf410e290 2024-06-11T13:29:35.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:35.802+0000 7fddf9972640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fddf410a6f0 0x7fddf41a3af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:35360/0 (socket says 172.21.15.57:35360) 2024-06-11T13:29:35.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:35.802+0000 7fddfa974640 1 --1- >> v1:172.21.15.57:6790/0 conn(0x7fddf410e290 0x7fddf41a7220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6790/0 says I am v1:172.21.15.57:48538/0 (socket says 172.21.15.57:48538) 2024-06-11T13:29:35.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:35.802+0000 7fddf9972640 1 -- 172.21.15.57:0/1838914864 learned_addr learned my addr 172.21.15.57:0/1838914864 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:29:35.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:35.802+0000 7fddeb7fe640 1 -- 172.21.15.57:0/1838914864 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 134949708 0 0) 0x7fddf41a8ad0 con 0x7fddf410a6f0 2024-06-11T13:29:35.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:35.802+0000 7fddeb7fe640 1 -- 172.21.15.57:0/1838914864 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdde0003520 con 0x7fddf410a6f0 2024-06-11T13:29:35.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:35.802+0000 7fddeb7fe640 1 -- 172.21.15.57:0/1838914864 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 332128117 0 0) 0x7fddf41aaf30 con 0x7fddf410e290 2024-06-11T13:29:35.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:35.802+0000 7fddeb7fe640 1 -- 172.21.15.57:0/1838914864 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fddf41a8ad0 con 0x7fddf410e290 2024-06-11T13:29:35.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:35.802+0000 7fddeb7fe640 1 -- 172.21.15.57:0/1838914864 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4109824863 0 0) 0x7fddf41a9cf0 con 0x7fddf4107840 2024-06-11T13:29:35.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:35.802+0000 7fddeb7fe640 1 -- 172.21.15.57:0/1838914864 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fddf41aaf30 con 0x7fddf4107840 2024-06-11T13:29:35.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:35.803+0000 7fddeb7fe640 1 -- 172.21.15.57:0/1838914864 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 825414394 0 0) 0x7fdde0003520 con 0x7fddf410a6f0 2024-06-11T13:29:35.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:35.803+0000 7fddeb7fe640 1 -- 172.21.15.57:0/1838914864 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fddf41a9cf0 con 0x7fddf410a6f0 2024-06-11T13:29:35.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:35.803+0000 7fddeb7fe640 1 -- 172.21.15.57:0/1838914864 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fdde4001930 con 0x7fddf410a6f0 2024-06-11T13:29:35.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:35.803+0000 7fddeb7fe640 1 -- 172.21.15.57:0/1838914864 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2667140333 0 0) 0x7fddf41a8ad0 con 0x7fddf410e290 2024-06-11T13:29:35.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:35.803+0000 7fddeb7fe640 1 -- 172.21.15.57:0/1838914864 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fdde0003520 con 0x7fddf410e290 2024-06-11T13:29:35.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:35.803+0000 7fddeb7fe640 1 -- 172.21.15.57:0/1838914864 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3711480101 0 0) 0x7fddf41aaf30 con 0x7fddf4107840 2024-06-11T13:29:35.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:35.803+0000 7fddeb7fe640 1 -- 172.21.15.57:0/1838914864 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fddf41a8ad0 con 0x7fddf4107840 2024-06-11T13:29:35.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:35.803+0000 7fddeb7fe640 1 -- 172.21.15.57:0/1838914864 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fddec001b20 con 0x7fddf410e290 2024-06-11T13:29:35.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:35.803+0000 7fddeb7fe640 1 -- 172.21.15.57:0/1838914864 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fddf0000fe0 con 0x7fddf4107840 2024-06-11T13:29:35.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:35.803+0000 7fddeb7fe640 1 -- 172.21.15.57:0/1838914864 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2876393508 0 0) 0x7fddf41a9cf0 con 0x7fddf410a6f0 2024-06-11T13:29:35.810 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:35.803+0000 7fddeb7fe640 1 -- 172.21.15.57:0/1838914864 >> v1:172.21.15.57:6790/0 conn(0x7fddf410e290 legacy=0x7fddf41a7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:35.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:35.803+0000 7fddeb7fe640 1 -- 172.21.15.57:0/1838914864 >> v1:172.21.15.107:6789/0 conn(0x7fddf4107840 legacy=0x7fddf410db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:35.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:35.803+0000 7fddeb7fe640 1 -- 172.21.15.57:0/1838914864 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddf41ac170 con 0x7fddf410a6f0 2024-06-11T13:29:35.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:35.803+0000 7fddfc3d7640 1 -- 172.21.15.57:0/1838914864 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fddf41a9f20 con 0x7fddf410a6f0 2024-06-11T13:29:35.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:35.803+0000 7fddfc3d7640 1 -- 172.21.15.57:0/1838914864 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fddf41aa3e0 con 0x7fddf410a6f0 2024-06-11T13:29:35.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:35.803+0000 7fddeb7fe640 1 -- 172.21.15.57:0/1838914864 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7fdde4003aa0 con 0x7fddf410a6f0 2024-06-11T13:29:35.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:35.803+0000 7fddeb7fe640 1 -- 172.21.15.57:0/1838914864 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fdde4005c10 con 0x7fddf410a6f0 2024-06-11T13:29:35.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:35.804+0000 7fddfc3d7640 1 -- 172.21.15.57:0/1838914864 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddd8005080 con 0x7fddf410a6f0 2024-06-11T13:29:35.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:35.805+0000 7fddeb7fe640 1 -- 172.21.15.57:0/1838914864 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 22) ==== 92138+0+0 (unknown 311317106 0 0) 0x7fdde4003610 con 0x7fddf410a6f0 2024-06-11T13:29:35.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:35.806+0000 7fddeb7fe640 1 -- 172.21.15.57:0/1838914864 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(70..70 src has 1..70) ==== 6172+0+0 (unknown 3813449920 0 0) 0x7fdde40892f0 con 0x7fddf410a6f0 2024-06-11T13:29:35.811 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:35.810+0000 7fddeb7fe640 1 -- 172.21.15.57:0/1838914864 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7fdde4056a90 con 0x7fddf410a6f0 2024-06-11T13:29:36.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:36.004+0000 7fddfc3d7640 1 -- 172.21.15.57:0/1838914864 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fddd8005370 con 0x7fddf410a6f0 2024-06-11T13:29:36.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:36.005+0000 7fddeb7fe640 1 -- 172.21.15.57:0/1838914864 <== mon.0 v1:172.21.15.57:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (unknown 1823589948 0 770812263) 0x7fdde405a740 con 0x7fddf410a6f0 2024-06-11T13:29:36.001 INFO:teuthology.orchestra.run.smithi057.stdout:128849018915 2024-06-11T13:29:36.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:36.007+0000 7fddfc3d7640 1 -- 172.21.15.57:0/1838914864 >> v1:172.21.15.57:6800/2077416415 conn(0x7fdde006eea0 legacy=0x7fdde0071360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:36.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:36.007+0000 7fddfc3d7640 1 -- 172.21.15.57:0/1838914864 >> v1:172.21.15.57:6789/0 conn(0x7fddf410a6f0 legacy=0x7fddf41a3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:36.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:36.007+0000 7fddfc3d7640 1 -- 172.21.15.57:0/1838914864 shutdown_connections 2024-06-11T13:29:36.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:36.007+0000 7fddfc3d7640 1 -- 172.21.15.57:0/1838914864 >> 172.21.15.57:0/1838914864 conn(0x7fddf40fd350 msgr2=0x7fddf40fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:29:36.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:36.007+0000 7fddfc3d7640 1 -- 172.21.15.57:0/1838914864 shutdown_connections 2024-06-11T13:29:36.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:36.007+0000 7fddfc3d7640 1 -- 172.21.15.57:0/1838914864 wait complete. 2024-06-11T13:29:36.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:36 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1838914864' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-11T13:29:36.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:36 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1838914864' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-11T13:29:36.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:36 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1838914864' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-11T13:29:36.832 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018911 got 128849018915 for osd.4 2024-06-11T13:29:36.833 DEBUG:teuthology.parallel:result is None 2024-06-11T13:29:37.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:37.048+0000 7f8712252640 1 -- 172.21.15.57:0/2756944722 >> v1:172.21.15.57:6790/0 conn(0x7f870c1029b0 legacy=0x7f870c102d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:37.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:37.048+0000 7f8712252640 1 -- 172.21.15.57:0/2756944722 shutdown_connections 2024-06-11T13:29:37.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:37.048+0000 7f8712252640 1 -- 172.21.15.57:0/2756944722 >> 172.21.15.57:0/2756944722 conn(0x7f870c0fc220 msgr2=0x7f870c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:29:37.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:37.048+0000 7f8712252640 1 -- 172.21.15.57:0/2756944722 shutdown_connections 2024-06-11T13:29:37.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:37.048+0000 7f8712252640 1 -- 172.21.15.57:0/2756944722 wait complete. 2024-06-11T13:29:37.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:37.048+0000 7f8712252640 1 Processor -- start 2024-06-11T13:29:37.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:37.049+0000 7f8712252640 1 -- start start 2024-06-11T13:29:37.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:37.049+0000 7f8712252640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f870c1a8ad0 con 0x7f870c10e2d0 2024-06-11T13:29:37.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:37.049+0000 7f8712252640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f870c1a9d10 con 0x7f870c1029b0 2024-06-11T13:29:37.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:37.049+0000 7f8712252640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f870c1aaf50 con 0x7f870c10a7c0 2024-06-11T13:29:37.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:37.049+0000 7f870affd640 1 --1- >> v1:172.21.15.57:6790/0 conn(0x7f870c10a7c0 0x7f870c1a3aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6790/0 says I am v1:172.21.15.57:48564/0 (socket says 172.21.15.57:48564) 2024-06-11T13:29:37.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:37.049+0000 7f870affd640 1 -- 172.21.15.57:0/1350869806 learned_addr learned my addr 172.21.15.57:0/1350869806 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:29:37.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:37.049+0000 7f8708ff9640 1 -- 172.21.15.57:0/1350869806 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2528837271 0 0) 0x7f870c1a9d10 con 0x7f870c1029b0 2024-06-11T13:29:37.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:37.049+0000 7f8708ff9640 1 -- 172.21.15.57:0/1350869806 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f86f4003520 con 0x7f870c1029b0 2024-06-11T13:29:37.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:37.049+0000 7f8708ff9640 1 -- 172.21.15.57:0/1350869806 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2702911990 0 0) 0x7f870c1aaf50 con 0x7f870c10a7c0 2024-06-11T13:29:37.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:37.049+0000 7f8708ff9640 1 -- 172.21.15.57:0/1350869806 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f870c1a9d10 con 0x7f870c10a7c0 2024-06-11T13:29:37.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:37.049+0000 7f8708ff9640 1 -- 172.21.15.57:0/1350869806 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1610650855 0 0) 0x7f870c1a8ad0 con 0x7f870c10e2d0 2024-06-11T13:29:37.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:37.049+0000 7f8708ff9640 1 -- 172.21.15.57:0/1350869806 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f870c1aaf50 con 0x7f870c10e2d0 2024-06-11T13:29:37.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:37.050+0000 7f8708ff9640 1 -- 172.21.15.57:0/1350869806 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2779052268 0 0) 0x7f86f4003520 con 0x7f870c1029b0 2024-06-11T13:29:37.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:37.050+0000 7f8708ff9640 1 -- 172.21.15.57:0/1350869806 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f870c1a8ad0 con 0x7f870c1029b0 2024-06-11T13:29:37.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:37.050+0000 7f8708ff9640 1 -- 172.21.15.57:0/1350869806 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f86fc001910 con 0x7f870c1029b0 2024-06-11T13:29:37.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:37.050+0000 7f8708ff9640 1 -- 172.21.15.57:0/1350869806 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2532489394 0 0) 0x7f870c1a9d10 con 0x7f870c10a7c0 2024-06-11T13:29:37.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:37.050+0000 7f8708ff9640 1 -- 172.21.15.57:0/1350869806 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f86f4003520 con 0x7f870c10a7c0 2024-06-11T13:29:37.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:37.050+0000 7f8708ff9640 1 -- 172.21.15.57:0/1350869806 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1862927634 0 0) 0x7f870c1aaf50 con 0x7f870c10e2d0 2024-06-11T13:29:37.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:37.050+0000 7f8708ff9640 1 -- 172.21.15.57:0/1350869806 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f870c1a9d10 con 0x7f870c10e2d0 2024-06-11T13:29:37.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:37.050+0000 7f8708ff9640 1 -- 172.21.15.57:0/1350869806 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f8700001870 con 0x7f870c10a7c0 2024-06-11T13:29:37.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:37.050+0000 7f8708ff9640 1 -- 172.21.15.57:0/1350869806 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f8704001b70 con 0x7f870c10e2d0 2024-06-11T13:29:37.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:37.050+0000 7f8708ff9640 1 -- 172.21.15.57:0/1350869806 <== mon.1 v1:172.21.15.107:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3238541526 0 0) 0x7f870c1a8ad0 con 0x7f870c1029b0 2024-06-11T13:29:37.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:37.050+0000 7f8708ff9640 1 -- 172.21.15.57:0/1350869806 >> v1:172.21.15.57:6790/0 conn(0x7f870c10a7c0 legacy=0x7f870c1a3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:37.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:37.050+0000 7f8708ff9640 1 -- 172.21.15.57:0/1350869806 >> v1:172.21.15.57:6789/0 conn(0x7f870c10e2d0 legacy=0x7f870c1a71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:37.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:37.050+0000 7f8708ff9640 1 -- 172.21.15.57:0/1350869806 --> v1:172.21.15.107:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f870c1ac190 con 0x7f870c1029b0 2024-06-11T13:29:37.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:37.050+0000 7f8712252640 1 -- 172.21.15.57:0/1350869806 --> v1:172.21.15.107:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f870c1ab180 con 0x7f870c1029b0 2024-06-11T13:29:37.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:37.050+0000 7f8712252640 1 -- 172.21.15.57:0/1350869806 --> v1:172.21.15.107:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f870c1ab640 con 0x7f870c1029b0 2024-06-11T13:29:37.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:37.050+0000 7f8708ff9640 1 -- 172.21.15.57:0/1350869806 <== mon.1 v1:172.21.15.107:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f86fc001c00 con 0x7f870c1029b0 2024-06-11T13:29:37.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:37.050+0000 7f8708ff9640 1 -- 172.21.15.57:0/1350869806 <== mon.1 v1:172.21.15.107:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f86fc004bb0 con 0x7f870c1029b0 2024-06-11T13:29:37.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:37.051+0000 7f8712252640 1 -- 172.21.15.57:0/1350869806 --> v1:172.21.15.107:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86f8005080 con 0x7f870c1029b0 2024-06-11T13:29:37.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:37.052+0000 7f8708ff9640 1 -- 172.21.15.57:0/1350869806 <== mon.1 v1:172.21.15.107:6789/0 7 ==== mgrmap(e 22) ==== 92138+0+0 (unknown 311317106 0 0) 0x7f86fc0035f0 con 0x7f870c1029b0 2024-06-11T13:29:37.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:37.052+0000 7f8708ff9640 1 -- 172.21.15.57:0/1350869806 <== mon.1 v1:172.21.15.107:6789/0 8 ==== osd_map(70..70 src has 1..70) ==== 6172+0+0 (unknown 3813449920 0 0) 0x7f86fc088260 con 0x7f870c1029b0 2024-06-11T13:29:37.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:37.057+0000 7f8708ff9640 1 -- 172.21.15.57:0/1350869806 <== mon.1 v1:172.21.15.107:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f86fc0559c0 con 0x7f870c1029b0 2024-06-11T13:29:37.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:37.243+0000 7f8712252640 1 -- 172.21.15.57:0/1350869806 --> v1:172.21.15.107:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f86f8005370 con 0x7f870c1029b0 2024-06-11T13:29:37.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:37.243+0000 7f8708ff9640 1 -- 172.21.15.57:0/1350869806 <== mon.1 v1:172.21.15.107:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (unknown 1482059429 0 3987907149) 0x7f86fc059670 con 0x7f870c1029b0 2024-06-11T13:29:37.240 INFO:teuthology.orchestra.run.smithi057.stdout:236223201303 2024-06-11T13:29:37.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:37.245+0000 7f8712252640 1 -- 172.21.15.57:0/1350869806 >> v1:172.21.15.57:6800/2077416415 conn(0x7f86f406eff0 legacy=0x7f86f40714b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:37.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:37.245+0000 7f8712252640 1 -- 172.21.15.57:0/1350869806 >> v1:172.21.15.107:6789/0 conn(0x7f870c1029b0 legacy=0x7f870c10a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:37.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:37.245+0000 7f8712252640 1 -- 172.21.15.57:0/1350869806 shutdown_connections 2024-06-11T13:29:37.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:37.245+0000 7f8712252640 1 -- 172.21.15.57:0/1350869806 >> 172.21.15.57:0/1350869806 conn(0x7f870c0fc220 msgr2=0x7f870c110700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:29:37.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:37.245+0000 7f8712252640 1 -- 172.21.15.57:0/1350869806 shutdown_connections 2024-06-11T13:29:37.242 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:37.245+0000 7f8712252640 1 -- 172.21.15.57:0/1350869806 wait complete. 2024-06-11T13:29:37.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:37 smithi057 ceph-mon[29805]: pgmap v26: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:29:37.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:37 smithi057 ceph-mon[40588]: pgmap v26: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:29:37.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:37 smithi107 ceph-mon[33178]: pgmap v26: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:29:37.937 INFO:tasks.cephadm.ceph_manager.ceph:need seq 236223201299 got 236223201303 for osd.7 2024-06-11T13:29:37.938 DEBUG:teuthology.parallel:result is None 2024-06-11T13:29:37.938 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-11T13:29:37.938 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-11T13:29:38.193 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.a/config 2024-06-11T13:29:38.462 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:38 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1350869806' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-11T13:29:38.500 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:38 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1350869806' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-11T13:29:38.500 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:38 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1350869806' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-11T13:29:39.449 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:39 smithi107 ceph-mon[33178]: pgmap v27: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:29:39.494 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:39 smithi057 ceph-mon[29805]: pgmap v27: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:29:39.494 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:39 smithi057 ceph-mon[40588]: pgmap v27: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:29:39.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:39.881+0000 7f918473c640 1 -- 172.21.15.57:0/1050821756 >> v1:172.21.15.57:6790/0 conn(0x7f917c1095a0 legacy=0x7f917c10b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:39.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:39.881+0000 7f918473c640 1 -- 172.21.15.57:0/1050821756 shutdown_connections 2024-06-11T13:29:39.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:39.881+0000 7f918473c640 1 -- 172.21.15.57:0/1050821756 >> 172.21.15.57:0/1050821756 conn(0x7f917c0fc220 msgr2=0x7f917c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:29:39.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:39.881+0000 7f918473c640 1 -- 172.21.15.57:0/1050821756 shutdown_connections 2024-06-11T13:29:39.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:39.881+0000 7f918473c640 1 -- 172.21.15.57:0/1050821756 wait complete. 2024-06-11T13:29:39.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:39.882+0000 7f918473c640 1 Processor -- start 2024-06-11T13:29:39.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:39.882+0000 7f918473c640 1 -- start start 2024-06-11T13:29:39.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:39.882+0000 7f918473c640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f917c10cd40 con 0x7f917c1095a0 2024-06-11T13:29:39.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:39.882+0000 7f918473c640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f917c10cf70 con 0x7f917c1066f0 2024-06-11T13:29:39.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:39.882+0000 7f918473c640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f917c1a8d90 con 0x7f917c10d140 2024-06-11T13:29:39.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:39.882+0000 7f9182cd9640 1 --1- >> v1:172.21.15.57:6790/0 conn(0x7f917c10d140 0x7f917c1a55f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6790/0 says I am v1:172.21.15.57:41652/0 (socket says 172.21.15.57:41652) 2024-06-11T13:29:39.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:39.882+0000 7f9182cd9640 1 -- 172.21.15.57:0/2052731032 learned_addr learned my addr 172.21.15.57:0/2052731032 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:29:39.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:39.882+0000 7f9181cd7640 1 --1- 172.21.15.57:0/2052731032 >> v1:172.21.15.57:6789/0 conn(0x7f917c1095a0 0x7f917c10b280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:38368/0 (socket says 172.21.15.57:38368) 2024-06-11T13:29:39.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:39.882+0000 7f91737fe640 1 -- 172.21.15.57:0/2052731032 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1788876554 0 0) 0x7f917c1a8d90 con 0x7f917c10d140 2024-06-11T13:29:39.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:39.882+0000 7f91737fe640 1 -- 172.21.15.57:0/2052731032 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9168003520 con 0x7f917c10d140 2024-06-11T13:29:39.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:39.882+0000 7f91737fe640 1 -- 172.21.15.57:0/2052731032 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1490720115 0 0) 0x7f917c10cd40 con 0x7f917c1095a0 2024-06-11T13:29:39.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:39.882+0000 7f91737fe640 1 -- 172.21.15.57:0/2052731032 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f917c1a8d90 con 0x7f917c1095a0 2024-06-11T13:29:39.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:39.882+0000 7f91737fe640 1 -- 172.21.15.57:0/2052731032 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2995349859 0 0) 0x7f917c10cf70 con 0x7f917c1066f0 2024-06-11T13:29:39.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:39.882+0000 7f91737fe640 1 -- 172.21.15.57:0/2052731032 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f917c10cd40 con 0x7f917c1066f0 2024-06-11T13:29:39.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:39.883+0000 7f91737fe640 1 -- 172.21.15.57:0/2052731032 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3514748624 0 0) 0x7f9168003520 con 0x7f917c10d140 2024-06-11T13:29:39.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:39.883+0000 7f91737fe640 1 -- 172.21.15.57:0/2052731032 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f917c10cf70 con 0x7f917c10d140 2024-06-11T13:29:39.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:39.883+0000 7f91737fe640 1 -- 172.21.15.57:0/2052731032 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f9174001c70 con 0x7f917c10d140 2024-06-11T13:29:39.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:39.883+0000 7f91737fe640 1 -- 172.21.15.57:0/2052731032 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3650930002 0 0) 0x7f917c1a8d90 con 0x7f917c1095a0 2024-06-11T13:29:39.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:39.883+0000 7f91737fe640 1 -- 172.21.15.57:0/2052731032 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9168003520 con 0x7f917c1095a0 2024-06-11T13:29:39.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:39.883+0000 7f91737fe640 1 -- 172.21.15.57:0/2052731032 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f916c004410 con 0x7f917c1095a0 2024-06-11T13:29:39.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:39.883+0000 7f91737fe640 1 -- 172.21.15.57:0/2052731032 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 647303609 0 0) 0x7f917c10cd40 con 0x7f917c1066f0 2024-06-11T13:29:39.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:39.883+0000 7f91737fe640 1 -- 172.21.15.57:0/2052731032 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f917c1a8d90 con 0x7f917c1066f0 2024-06-11T13:29:39.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:39.883+0000 7f91737fe640 1 -- 172.21.15.57:0/2052731032 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f9178001890 con 0x7f917c1066f0 2024-06-11T13:29:39.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:39.883+0000 7f91737fe640 1 -- 172.21.15.57:0/2052731032 <== mon.2 v1:172.21.15.57:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1693344574 0 0) 0x7f917c10cf70 con 0x7f917c10d140 2024-06-11T13:29:39.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:39.883+0000 7f91737fe640 1 -- 172.21.15.57:0/2052731032 >> v1:172.21.15.107:6789/0 conn(0x7f917c1066f0 legacy=0x7f917c10ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:39.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:39.883+0000 7f91737fe640 1 -- 172.21.15.57:0/2052731032 >> v1:172.21.15.57:6789/0 conn(0x7f917c1095a0 legacy=0x7f917c10b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:39.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:39.883+0000 7f91737fe640 1 -- 172.21.15.57:0/2052731032 --> v1:172.21.15.57:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f917c1a9fd0 con 0x7f917c10d140 2024-06-11T13:29:39.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:39.883+0000 7f918473c640 1 -- 172.21.15.57:0/2052731032 --> v1:172.21.15.57:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f917c1a8fc0 con 0x7f917c10d140 2024-06-11T13:29:39.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:39.883+0000 7f918473c640 1 -- 172.21.15.57:0/2052731032 --> v1:172.21.15.57:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f917c1a9480 con 0x7f917c10d140 2024-06-11T13:29:39.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:39.883+0000 7f91737fe640 1 -- 172.21.15.57:0/2052731032 <== mon.2 v1:172.21.15.57:6790/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f9174003e20 con 0x7f917c10d140 2024-06-11T13:29:39.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:39.883+0000 7f91737fe640 1 -- 172.21.15.57:0/2052731032 <== mon.2 v1:172.21.15.57:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f9174005fb0 con 0x7f917c10d140 2024-06-11T13:29:39.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:39.884+0000 7f918473c640 1 -- 172.21.15.57:0/2052731032 --> v1:172.21.15.57:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9160005080 con 0x7f917c10d140 2024-06-11T13:29:39.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:39.885+0000 7f91737fe640 1 -- 172.21.15.57:0/2052731032 <== mon.2 v1:172.21.15.57:6790/0 7 ==== mgrmap(e 22) ==== 92138+0+0 (unknown 311317106 0 0) 0x7f9174003990 con 0x7f917c10d140 2024-06-11T13:29:39.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:39.885+0000 7f91737fe640 1 -- 172.21.15.57:0/2052731032 <== mon.2 v1:172.21.15.57:6790/0 8 ==== osd_map(70..70 src has 1..70) ==== 6172+0+0 (unknown 3813449920 0 0) 0x7f91740898e0 con 0x7f917c10d140 2024-06-11T13:29:39.896 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:39.890+0000 7f91737fe640 1 -- 172.21.15.57:0/2052731032 <== mon.2 v1:172.21.15.57:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f91740570a0 con 0x7f917c10d140 2024-06-11T13:29:40.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:40.073+0000 7f918473c640 1 -- 172.21.15.57:0/2052731032 --> v1:172.21.15.57:6800/2077416415 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9160000b90 con 0x7f916806ee90 2024-06-11T13:29:40.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:40.082+0000 7f91737fe640 1 -- 172.21.15.57:0/2052731032 <== mgr.14604 v1:172.21.15.57:6800/2077416415 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+346459 (unknown 2965378022 0 2259130426) 0x7f9160000b90 con 0x7f916806ee90 2024-06-11T13:29:40.082 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:29:40.084 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2024-06-11T13:29:40.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:40.085+0000 7f918473c640 1 -- 172.21.15.57:0/2052731032 >> v1:172.21.15.57:6800/2077416415 conn(0x7f916806ee90 legacy=0x7f9168071350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:40.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:40.085+0000 7f918473c640 1 -- 172.21.15.57:0/2052731032 >> v1:172.21.15.57:6790/0 conn(0x7f917c10d140 legacy=0x7f917c1a55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:40.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:40.085+0000 7f918473c640 1 -- 172.21.15.57:0/2052731032 shutdown_connections 2024-06-11T13:29:40.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:40.085+0000 7f918473c640 1 -- 172.21.15.57:0/2052731032 >> 172.21.15.57:0/2052731032 conn(0x7f917c0fc220 msgr2=0x7f917c1006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:29:40.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:40.085+0000 7f918473c640 1 -- 172.21.15.57:0/2052731032 shutdown_connections 2024-06-11T13:29:40.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:40.085+0000 7f918473c640 1 -- 172.21.15.57:0/2052731032 wait complete. 2024-06-11T13:29:40.876 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-06-11T13:29:39.965715+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":198,"num_object_clones":0,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":198,"num_whiteouts":0,"num_read":1011,"num_read_kb":857,"num_write":615,"num_write_kb":2461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":600,"ondisk_log_size":600,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":248276,"kb_used_data":7164,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":749712940,"statfs":{"total":767960285184,"available":767706050560,"internally_reserved":0,"allocated":7335936,"data_stored":3919001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003030"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.031788+0000","last_change":"2024-06-11T13:28:19.550716+0000","last_active":"2024-06-11T13:28:50.031788+0000","last_peered":"2024-06-11T13:28:50.031788+0000","last_clean":"2024-06-11T13:28:50.031788+0000","last_became_active":"2024-06-11T13:28:19.550286+0000","last_became_peered":"2024-06-11T13:28:19.550286+0000","last_unstale":"2024-06-11T13:28:50.031788+0000","last_undegraded":"2024-06-11T13:28:50.031788+0000","last_fullsized":"2024-06-11T13:28:50.031788+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T21:15:28.536619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.943500+0000","last_change":"2024-06-11T13:28:13.182552+0000","last_active":"2024-06-11T13:28:49.943500+0000","last_peered":"2024-06-11T13:28:49.943500+0000","last_clean":"2024-06-11T13:28:49.943500+0000","last_became_active":"2024-06-11T13:28:13.181741+0000","last_became_peered":"2024-06-11T13:28:13.181741+0000","last_unstale":"2024-06-11T13:28:49.943500+0000","last_undegraded":"2024-06-11T13:28:49.943500+0000","last_fullsized":"2024-06-11T13:28:49.943500+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T18:31:32.801830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"68'15","reported_seq":50,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.031667+0000","last_change":"2024-06-11T13:28:15.428556+0000","last_active":"2024-06-11T13:28:50.031667+0000","last_peered":"2024-06-11T13:28:50.031667+0000","last_clean":"2024-06-11T13:28:50.031667+0000","last_became_active":"2024-06-11T13:28:15.428175+0000","last_became_peered":"2024-06-11T13:28:15.428175+0000","last_unstale":"2024-06-11T13:28:50.031667+0000","last_undegraded":"2024-06-11T13:28:50.031667+0000","last_fullsized":"2024-06-11T13:28:50.031667+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T18:09:37.101758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.454339+0000","last_change":"2024-06-11T13:28:17.455777+0000","last_active":"2024-06-11T13:28:51.454339+0000","last_peered":"2024-06-11T13:28:51.454339+0000","last_clean":"2024-06-11T13:28:51.454339+0000","last_became_active":"2024-06-11T13:28:17.454883+0000","last_became_peered":"2024-06-11T13:28:17.454883+0000","last_unstale":"2024-06-11T13:28:51.454339+0000","last_undegraded":"2024-06-11T13:28:51.454339+0000","last_fullsized":"2024-06-11T13:28:51.454339+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T15:26:14.098490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.033292+0000","last_change":"2024-06-11T13:28:13.185465+0000","last_active":"2024-06-11T13:28:50.033292+0000","last_peered":"2024-06-11T13:28:50.033292+0000","last_clean":"2024-06-11T13:28:50.033292+0000","last_became_active":"2024-06-11T13:28:13.184658+0000","last_became_peered":"2024-06-11T13:28:13.184658+0000","last_unstale":"2024-06-11T13:28:50.033292+0000","last_undegraded":"2024-06-11T13:28:50.033292+0000","last_fullsized":"2024-06-11T13:28:50.033292+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T14:30:05.952881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"68'9","reported_seq":41,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.454358+0000","last_change":"2024-06-11T13:28:15.420834+0000","last_active":"2024-06-11T13:28:51.454358+0000","last_peered":"2024-06-11T13:28:51.454358+0000","last_clean":"2024-06-11T13:28:51.454358+0000","last_became_active":"2024-06-11T13:28:15.419923+0000","last_became_peered":"2024-06-11T13:28:15.419923+0000","last_unstale":"2024-06-11T13:28:51.454358+0000","last_undegraded":"2024-06-11T13:28:51.454358+0000","last_fullsized":"2024-06-11T13:28:51.454358+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-13T00:22:41.587194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.039245+0000","last_change":"2024-06-11T13:28:17.452840+0000","last_active":"2024-06-11T13:28:50.039245+0000","last_peered":"2024-06-11T13:28:50.039245+0000","last_clean":"2024-06-11T13:28:50.039245+0000","last_became_active":"2024-06-11T13:28:17.452503+0000","last_became_peered":"2024-06-11T13:28:17.452503+0000","last_unstale":"2024-06-11T13:28:50.039245+0000","last_undegraded":"2024-06-11T13:28:50.039245+0000","last_fullsized":"2024-06-11T13:28:50.039245+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T21:49:46.680378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.454423+0000","last_change":"2024-06-11T13:28:19.457125+0000","last_active":"2024-06-11T13:28:51.454423+0000","last_peered":"2024-06-11T13:28:51.454423+0000","last_clean":"2024-06-11T13:28:51.454423+0000","last_became_active":"2024-06-11T13:28:19.456666+0000","last_became_peered":"2024-06-11T13:28:19.456666+0000","last_unstale":"2024-06-11T13:28:51.454423+0000","last_undegraded":"2024-06-11T13:28:51.454423+0000","last_fullsized":"2024-06-11T13:28:51.454423+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T19:58:57.614042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.945313+0000","last_change":"2024-06-11T13:28:13.187109+0000","last_active":"2024-06-11T13:28:49.945313+0000","last_peered":"2024-06-11T13:28:49.945313+0000","last_clean":"2024-06-11T13:28:49.945313+0000","last_became_active":"2024-06-11T13:28:13.186748+0000","last_became_peered":"2024-06-11T13:28:13.186748+0000","last_unstale":"2024-06-11T13:28:49.945313+0000","last_undegraded":"2024-06-11T13:28:49.945313+0000","last_fullsized":"2024-06-11T13:28:49.945313+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T17:55:41.469322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"68'5","reported_seq":35,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.454439+0000","last_change":"2024-06-11T13:28:15.423406+0000","last_active":"2024-06-11T13:28:51.454439+0000","last_peered":"2024-06-11T13:28:51.454439+0000","last_clean":"2024-06-11T13:28:51.454439+0000","last_became_active":"2024-06-11T13:28:15.422942+0000","last_became_peered":"2024-06-11T13:28:15.422942+0000","last_unstale":"2024-06-11T13:28:51.454439+0000","last_undegraded":"2024-06-11T13:28:51.454439+0000","last_fullsized":"2024-06-11T13:28:51.454439+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T22:04:17.940417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.944831+0000","last_change":"2024-06-11T13:28:17.451344+0000","last_active":"2024-06-11T13:28:49.944831+0000","last_peered":"2024-06-11T13:28:49.944831+0000","last_clean":"2024-06-11T13:28:49.944831+0000","last_became_active":"2024-06-11T13:28:17.451043+0000","last_became_peered":"2024-06-11T13:28:17.451043+0000","last_unstale":"2024-06-11T13:28:49.944831+0000","last_undegraded":"2024-06-11T13:28:49.944831+0000","last_fullsized":"2024-06-11T13:28:49.944831+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-13T00:25:28.323738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.050420+0000","last_change":"2024-06-11T13:28:19.462380+0000","last_active":"2024-06-11T13:28:51.050420+0000","last_peered":"2024-06-11T13:28:51.050420+0000","last_clean":"2024-06-11T13:28:51.050420+0000","last_became_active":"2024-06-11T13:28:19.461999+0000","last_became_peered":"2024-06-11T13:28:19.461999+0000","last_unstale":"2024-06-11T13:28:51.050420+0000","last_undegraded":"2024-06-11T13:28:51.050420+0000","last_fullsized":"2024-06-11T13:28:51.050420+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T15:46:02.464320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.945250+0000","last_change":"2024-06-11T13:28:13.183952+0000","last_active":"2024-06-11T13:28:49.945250+0000","last_peered":"2024-06-11T13:28:49.945250+0000","last_clean":"2024-06-11T13:28:49.945250+0000","last_became_active":"2024-06-11T13:28:13.183647+0000","last_became_peered":"2024-06-11T13:28:13.183647+0000","last_unstale":"2024-06-11T13:28:49.945250+0000","last_undegraded":"2024-06-11T13:28:49.945250+0000","last_fullsized":"2024-06-11T13:28:49.945250+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T22:32:55.509603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"68'9","reported_seq":41,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.454746+0000","last_change":"2024-06-11T13:28:15.423848+0000","last_active":"2024-06-11T13:28:51.454746+0000","last_peered":"2024-06-11T13:28:51.454746+0000","last_clean":"2024-06-11T13:28:51.454746+0000","last_became_active":"2024-06-11T13:28:15.423005+0000","last_became_peered":"2024-06-11T13:28:15.423005+0000","last_unstale":"2024-06-11T13:28:51.454746+0000","last_undegraded":"2024-06-11T13:28:51.454746+0000","last_fullsized":"2024-06-11T13:28:51.454746+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T17:10:25.210743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.050466+0000","last_change":"2024-06-11T13:28:17.457031+0000","last_active":"2024-06-11T13:28:51.050466+0000","last_peered":"2024-06-11T13:28:51.050466+0000","last_clean":"2024-06-11T13:28:51.050466+0000","last_became_active":"2024-06-11T13:28:17.456516+0000","last_became_peered":"2024-06-11T13:28:17.456516+0000","last_unstale":"2024-06-11T13:28:51.050466+0000","last_undegraded":"2024-06-11T13:28:51.050466+0000","last_fullsized":"2024-06-11T13:28:51.050466+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T15:29:41.820196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"68'1","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.942777+0000","last_change":"2024-06-11T13:28:19.463547+0000","last_active":"2024-06-11T13:28:49.942777+0000","last_peered":"2024-06-11T13:28:49.942777+0000","last_clean":"2024-06-11T13:28:49.942777+0000","last_became_active":"2024-06-11T13:28:19.462163+0000","last_became_peered":"2024-06-11T13:28:19.462163+0000","last_unstale":"2024-06-11T13:28:49.942777+0000","last_undegraded":"2024-06-11T13:28:49.942777+0000","last_fullsized":"2024-06-11T13:28:49.942777+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T14:30:39.875492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"68'17","reported_seq":53,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.454772+0000","last_change":"2024-06-11T13:28:15.423569+0000","last_active":"2024-06-11T13:28:51.454772+0000","last_peered":"2024-06-11T13:28:51.454772+0000","last_clean":"2024-06-11T13:28:51.454772+0000","last_became_active":"2024-06-11T13:28:15.423206+0000","last_became_peered":"2024-06-11T13:28:15.423206+0000","last_unstale":"2024-06-11T13:28:51.454772+0000","last_undegraded":"2024-06-11T13:28:51.454772+0000","last_fullsized":"2024-06-11T13:28:51.454772+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T19:42:24.824046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.945055+0000","last_change":"2024-06-11T13:28:13.180360+0000","last_active":"2024-06-11T13:28:49.945055+0000","last_peered":"2024-06-11T13:28:49.945055+0000","last_clean":"2024-06-11T13:28:49.945055+0000","last_became_active":"2024-06-11T13:28:13.179463+0000","last_became_peered":"2024-06-11T13:28:13.179463+0000","last_unstale":"2024-06-11T13:28:49.945055+0000","last_undegraded":"2024-06-11T13:28:49.945055+0000","last_fullsized":"2024-06-11T13:28:49.945055+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-13T00:06:50.134505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.039815+0000","last_change":"2024-06-11T13:28:17.456131+0000","last_active":"2024-06-11T13:28:50.039815+0000","last_peered":"2024-06-11T13:28:50.039815+0000","last_clean":"2024-06-11T13:28:50.039815+0000","last_became_active":"2024-06-11T13:28:17.455158+0000","last_became_peered":"2024-06-11T13:28:17.455158+0000","last_unstale":"2024-06-11T13:28:50.039815+0000","last_undegraded":"2024-06-11T13:28:50.039815+0000","last_fullsized":"2024-06-11T13:28:50.039815+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T19:41:29.645580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.946388+0000","last_change":"2024-06-11T13:28:19.457113+0000","last_active":"2024-06-11T13:28:49.946388+0000","last_peered":"2024-06-11T13:28:49.946388+0000","last_clean":"2024-06-11T13:28:49.946388+0000","last_became_active":"2024-06-11T13:28:19.456761+0000","last_became_peered":"2024-06-11T13:28:19.456761+0000","last_unstale":"2024-06-11T13:28:49.946388+0000","last_undegraded":"2024-06-11T13:28:49.946388+0000","last_fullsized":"2024-06-11T13:28:49.946388+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T21:53:57.692109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"68'10","reported_seq":40,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.454737+0000","last_change":"2024-06-11T13:28:15.424745+0000","last_active":"2024-06-11T13:28:51.454737+0000","last_peered":"2024-06-11T13:28:51.454737+0000","last_clean":"2024-06-11T13:28:51.454737+0000","last_became_active":"2024-06-11T13:28:15.424384+0000","last_became_peered":"2024-06-11T13:28:15.424384+0000","last_unstale":"2024-06-11T13:28:51.454737+0000","last_undegraded":"2024-06-11T13:28:51.454737+0000","last_fullsized":"2024-06-11T13:28:51.454737+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T20:28:30.196736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.040117+0000","last_change":"2024-06-11T13:28:13.188135+0000","last_active":"2024-06-11T13:28:50.040117+0000","last_peered":"2024-06-11T13:28:50.040117+0000","last_clean":"2024-06-11T13:28:50.040117+0000","last_became_active":"2024-06-11T13:28:13.187824+0000","last_became_peered":"2024-06-11T13:28:13.187824+0000","last_unstale":"2024-06-11T13:28:50.040117+0000","last_undegraded":"2024-06-11T13:28:50.040117+0000","last_fullsized":"2024-06-11T13:28:50.040117+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T16:54:37.369842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"68'8","reported_seq":46,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:29:28.243370+0000","last_change":"2024-06-11T13:28:17.452020+0000","last_active":"2024-06-11T13:29:28.243370+0000","last_peered":"2024-06-11T13:29:28.243370+0000","last_clean":"2024-06-11T13:29:28.243370+0000","last_became_active":"2024-06-11T13:28:17.451706+0000","last_became_peered":"2024-06-11T13:28:17.451706+0000","last_unstale":"2024-06-11T13:29:28.243370+0000","last_undegraded":"2024-06-11T13:29:28.243370+0000","last_fullsized":"2024-06-11T13:29:28.243370+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T20:44:28.252807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.454705+0000","last_change":"2024-06-11T13:28:19.457854+0000","last_active":"2024-06-11T13:28:51.454705+0000","last_peered":"2024-06-11T13:28:51.454705+0000","last_clean":"2024-06-11T13:28:51.454705+0000","last_became_active":"2024-06-11T13:28:19.457565+0000","last_became_peered":"2024-06-11T13:28:19.457565+0000","last_unstale":"2024-06-11T13:28:51.454705+0000","last_undegraded":"2024-06-11T13:28:51.454705+0000","last_fullsized":"2024-06-11T13:28:51.454705+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-13T00:55:14.966569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"68'15","reported_seq":50,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.040856+0000","last_change":"2024-06-11T13:28:15.426892+0000","last_active":"2024-06-11T13:28:50.040856+0000","last_peered":"2024-06-11T13:28:50.040856+0000","last_clean":"2024-06-11T13:28:50.040856+0000","last_became_active":"2024-06-11T13:28:15.426545+0000","last_became_peered":"2024-06-11T13:28:15.426545+0000","last_unstale":"2024-06-11T13:28:50.040856+0000","last_undegraded":"2024-06-11T13:28:50.040856+0000","last_fullsized":"2024-06-11T13:28:50.040856+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T19:08:25.374757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.039399+0000","last_change":"2024-06-11T13:28:13.189327+0000","last_active":"2024-06-11T13:28:50.039399+0000","last_peered":"2024-06-11T13:28:50.039399+0000","last_clean":"2024-06-11T13:28:50.039399+0000","last_became_active":"2024-06-11T13:28:13.188960+0000","last_became_peered":"2024-06-11T13:28:13.188960+0000","last_unstale":"2024-06-11T13:28:50.039399+0000","last_undegraded":"2024-06-11T13:28:50.039399+0000","last_fullsized":"2024-06-11T13:28:50.039399+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T15:02:47.835582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"68'8","reported_seq":46,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:29:28.243558+0000","last_change":"2024-06-11T13:28:17.454595+0000","last_active":"2024-06-11T13:29:28.243558+0000","last_peered":"2024-06-11T13:29:28.243558+0000","last_clean":"2024-06-11T13:29:28.243558+0000","last_became_active":"2024-06-11T13:28:17.454275+0000","last_became_peered":"2024-06-11T13:28:17.454275+0000","last_unstale":"2024-06-11T13:29:28.243558+0000","last_undegraded":"2024-06-11T13:29:28.243558+0000","last_fullsized":"2024-06-11T13:29:28.243558+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-13T00:13:07.986474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.050689+0000","last_change":"2024-06-11T13:28:19.456803+0000","last_active":"2024-06-11T13:28:51.050689+0000","last_peered":"2024-06-11T13:28:51.050689+0000","last_clean":"2024-06-11T13:28:51.050689+0000","last_became_active":"2024-06-11T13:28:19.456193+0000","last_became_peered":"2024-06-11T13:28:19.456193+0000","last_unstale":"2024-06-11T13:28:51.050689+0000","last_undegraded":"2024-06-11T13:28:51.050689+0000","last_fullsized":"2024-06-11T13:28:51.050689+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T19:16:32.736555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"68'11","reported_seq":44,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.454911+0000","last_change":"2024-06-11T13:28:15.420549+0000","last_active":"2024-06-11T13:28:51.454911+0000","last_peered":"2024-06-11T13:28:51.454911+0000","last_clean":"2024-06-11T13:28:51.454911+0000","last_became_active":"2024-06-11T13:28:15.419738+0000","last_became_peered":"2024-06-11T13:28:15.419738+0000","last_unstale":"2024-06-11T13:28:51.454911+0000","last_undegraded":"2024-06-11T13:28:51.454911+0000","last_fullsized":"2024-06-11T13:28:51.454911+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T18:45:27.891582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.944617+0000","last_change":"2024-06-11T13:28:13.186258+0000","last_active":"2024-06-11T13:28:49.944617+0000","last_peered":"2024-06-11T13:28:49.944617+0000","last_clean":"2024-06-11T13:28:49.944617+0000","last_became_active":"2024-06-11T13:28:13.185858+0000","last_became_peered":"2024-06-11T13:28:13.185858+0000","last_unstale":"2024-06-11T13:28:49.944617+0000","last_undegraded":"2024-06-11T13:28:49.944617+0000","last_fullsized":"2024-06-11T13:28:49.944617+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T18:11:30.855241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.050931+0000","last_change":"2024-06-11T13:28:17.457708+0000","last_active":"2024-06-11T13:28:51.050931+0000","last_peered":"2024-06-11T13:28:51.050931+0000","last_clean":"2024-06-11T13:28:51.050931+0000","last_became_active":"2024-06-11T13:28:17.457371+0000","last_became_peered":"2024-06-11T13:28:17.457371+0000","last_unstale":"2024-06-11T13:28:51.050931+0000","last_undegraded":"2024-06-11T13:28:51.050931+0000","last_fullsized":"2024-06-11T13:28:51.050931+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T23:19:32.764805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.031863+0000","last_change":"2024-06-11T13:28:19.548749+0000","last_active":"2024-06-11T13:28:50.031863+0000","last_peered":"2024-06-11T13:28:50.031863+0000","last_clean":"2024-06-11T13:28:50.031863+0000","last_became_active":"2024-06-11T13:28:19.548339+0000","last_became_peered":"2024-06-11T13:28:19.548339+0000","last_unstale":"2024-06-11T13:28:50.031863+0000","last_undegraded":"2024-06-11T13:28:50.031863+0000","last_fullsized":"2024-06-11T13:28:50.031863+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T15:33:09.658474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"68'14","reported_seq":46,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.454502+0000","last_change":"2024-06-11T13:28:15.419993+0000","last_active":"2024-06-11T13:28:51.454502+0000","last_peered":"2024-06-11T13:28:51.454502+0000","last_clean":"2024-06-11T13:28:51.454502+0000","last_became_active":"2024-06-11T13:28:15.419625+0000","last_became_peered":"2024-06-11T13:28:15.419625+0000","last_unstale":"2024-06-11T13:28:51.454502+0000","last_undegraded":"2024-06-11T13:28:51.454502+0000","last_fullsized":"2024-06-11T13:28:51.454502+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T13:31:14.387479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.549206+0000","last_change":"2024-06-11T13:28:13.184600+0000","last_active":"2024-06-11T13:28:50.549206+0000","last_peered":"2024-06-11T13:28:50.549206+0000","last_clean":"2024-06-11T13:28:50.549206+0000","last_became_active":"2024-06-11T13:28:13.183623+0000","last_became_peered":"2024-06-11T13:28:13.183623+0000","last_unstale":"2024-06-11T13:28:50.549206+0000","last_undegraded":"2024-06-11T13:28:50.549206+0000","last_fullsized":"2024-06-11T13:28:50.549206+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T21:38:39.160094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"68'8","reported_seq":46,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:29:28.243425+0000","last_change":"2024-06-11T13:28:17.455639+0000","last_active":"2024-06-11T13:29:28.243425+0000","last_peered":"2024-06-11T13:29:28.243425+0000","last_clean":"2024-06-11T13:29:28.243425+0000","last_became_active":"2024-06-11T13:28:17.455202+0000","last_became_peered":"2024-06-11T13:28:17.455202+0000","last_unstale":"2024-06-11T13:29:28.243425+0000","last_undegraded":"2024-06-11T13:29:28.243425+0000","last_fullsized":"2024-06-11T13:29:28.243425+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T13:44:30.136745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.944854+0000","last_change":"2024-06-11T13:28:19.549166+0000","last_active":"2024-06-11T13:28:49.944854+0000","last_peered":"2024-06-11T13:28:49.944854+0000","last_clean":"2024-06-11T13:28:49.944854+0000","last_became_active":"2024-06-11T13:28:19.548881+0000","last_became_peered":"2024-06-11T13:28:19.548881+0000","last_unstale":"2024-06-11T13:28:49.944854+0000","last_undegraded":"2024-06-11T13:28:49.944854+0000","last_fullsized":"2024-06-11T13:28:49.944854+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T21:02:42.466664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"68'18","reported_seq":57,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.031497+0000","last_change":"2024-06-11T13:28:15.599521+0000","last_active":"2024-06-11T13:28:50.031497+0000","last_peered":"2024-06-11T13:28:50.031497+0000","last_clean":"2024-06-11T13:28:50.031497+0000","last_became_active":"2024-06-11T13:28:15.599158+0000","last_became_peered":"2024-06-11T13:28:15.599158+0000","last_unstale":"2024-06-11T13:28:50.031497+0000","last_undegraded":"2024-06-11T13:28:50.031497+0000","last_fullsized":"2024-06-11T13:28:50.031497+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T13:51:02.836977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"61'1","reported_seq":30,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.040798+0000","last_change":"2024-06-11T13:28:13.188861+0000","last_active":"2024-06-11T13:28:50.040798+0000","last_peered":"2024-06-11T13:28:50.040798+0000","last_clean":"2024-06-11T13:28:50.040798+0000","last_became_active":"2024-06-11T13:28:13.188576+0000","last_became_peered":"2024-06-11T13:28:13.188576+0000","last_unstale":"2024-06-11T13:28:50.040798+0000","last_undegraded":"2024-06-11T13:28:50.040798+0000","last_fullsized":"2024-06-11T13:28:50.040798+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T21:25:57.212708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.454341+0000","last_change":"2024-06-11T13:28:17.447311+0000","last_active":"2024-06-11T13:28:51.454341+0000","last_peered":"2024-06-11T13:28:51.454341+0000","last_clean":"2024-06-11T13:28:51.454341+0000","last_became_active":"2024-06-11T13:28:17.446472+0000","last_became_peered":"2024-06-11T13:28:17.446472+0000","last_unstale":"2024-06-11T13:28:51.454341+0000","last_undegraded":"2024-06-11T13:28:51.454341+0000","last_fullsized":"2024-06-11T13:28:51.454341+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T16:04:40.986019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.454642+0000","last_change":"2024-06-11T13:28:19.461310+0000","last_active":"2024-06-11T13:28:51.454642+0000","last_peered":"2024-06-11T13:28:51.454642+0000","last_clean":"2024-06-11T13:28:51.454642+0000","last_became_active":"2024-06-11T13:28:19.460964+0000","last_became_peered":"2024-06-11T13:28:19.460964+0000","last_unstale":"2024-06-11T13:28:51.454642+0000","last_undegraded":"2024-06-11T13:28:51.454642+0000","last_fullsized":"2024-06-11T13:28:51.454642+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T20:52:14.875740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"68'19","reported_seq":61,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.943249+0000","last_change":"2024-06-11T13:28:15.597334+0000","last_active":"2024-06-11T13:28:49.943249+0000","last_peered":"2024-06-11T13:28:49.943249+0000","last_clean":"2024-06-11T13:28:49.943249+0000","last_became_active":"2024-06-11T13:28:15.596997+0000","last_became_peered":"2024-06-11T13:28:15.596997+0000","last_unstale":"2024-06-11T13:28:49.943249+0000","last_undegraded":"2024-06-11T13:28:49.943249+0000","last_fullsized":"2024-06-11T13:28:49.943249+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T18:54:34.631686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.945329+0000","last_change":"2024-06-11T13:28:13.187039+0000","last_active":"2024-06-11T13:28:49.945329+0000","last_peered":"2024-06-11T13:28:49.945329+0000","last_clean":"2024-06-11T13:28:49.945329+0000","last_became_active":"2024-06-11T13:28:13.186689+0000","last_became_peered":"2024-06-11T13:28:13.186689+0000","last_unstale":"2024-06-11T13:28:49.945329+0000","last_undegraded":"2024-06-11T13:28:49.945329+0000","last_fullsized":"2024-06-11T13:28:49.945329+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T13:30:37.646586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.548624+0000","last_change":"2024-06-11T13:28:17.453365+0000","last_active":"2024-06-11T13:28:50.548624+0000","last_peered":"2024-06-11T13:28:50.548624+0000","last_clean":"2024-06-11T13:28:50.548624+0000","last_became_active":"2024-06-11T13:28:17.452631+0000","last_became_peered":"2024-06-11T13:28:17.452631+0000","last_unstale":"2024-06-11T13:28:50.548624+0000","last_undegraded":"2024-06-11T13:28:50.548624+0000","last_fullsized":"2024-06-11T13:28:50.548624+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T22:33:41.316200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.039157+0000","last_change":"2024-06-11T13:28:19.548659+0000","last_active":"2024-06-11T13:28:50.039157+0000","last_peered":"2024-06-11T13:28:50.039157+0000","last_clean":"2024-06-11T13:28:50.039157+0000","last_became_active":"2024-06-11T13:28:19.548215+0000","last_became_peered":"2024-06-11T13:28:19.548215+0000","last_unstale":"2024-06-11T13:28:50.039157+0000","last_undegraded":"2024-06-11T13:28:50.039157+0000","last_fullsized":"2024-06-11T13:28:50.039157+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-13T00:24:40.719668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"68'10","reported_seq":40,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.040000+0000","last_change":"2024-06-11T13:28:15.424259+0000","last_active":"2024-06-11T13:28:50.040000+0000","last_peered":"2024-06-11T13:28:50.040000+0000","last_clean":"2024-06-11T13:28:50.040000+0000","last_became_active":"2024-06-11T13:28:15.423884+0000","last_became_peered":"2024-06-11T13:28:15.423884+0000","last_unstale":"2024-06-11T13:28:50.040000+0000","last_undegraded":"2024-06-11T13:28:50.040000+0000","last_fullsized":"2024-06-11T13:28:50.040000+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T23:34:08.611441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"61'1","reported_seq":35,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.040237+0000","last_change":"2024-06-11T13:28:13.189208+0000","last_active":"2024-06-11T13:28:50.040237+0000","last_peered":"2024-06-11T13:28:50.040237+0000","last_clean":"2024-06-11T13:28:50.040237+0000","last_became_active":"2024-06-11T13:28:13.187456+0000","last_became_peered":"2024-06-11T13:28:13.187456+0000","last_unstale":"2024-06-11T13:28:50.040237+0000","last_undegraded":"2024-06-11T13:28:50.040237+0000","last_fullsized":"2024-06-11T13:28:50.040237+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T17:28:59.916170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"68'8","reported_seq":49,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:29:28.243370+0000","last_change":"2024-06-11T13:28:17.454331+0000","last_active":"2024-06-11T13:29:28.243370+0000","last_peered":"2024-06-11T13:29:28.243370+0000","last_clean":"2024-06-11T13:29:28.243370+0000","last_became_active":"2024-06-11T13:28:17.454043+0000","last_became_peered":"2024-06-11T13:28:17.454043+0000","last_unstale":"2024-06-11T13:29:28.243370+0000","last_undegraded":"2024-06-11T13:29:28.243370+0000","last_fullsized":"2024-06-11T13:29:28.243370+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T13:51:30.785359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.942612+0000","last_change":"2024-06-11T13:28:19.463318+0000","last_active":"2024-06-11T13:28:49.942612+0000","last_peered":"2024-06-11T13:28:49.942612+0000","last_clean":"2024-06-11T13:28:49.942612+0000","last_became_active":"2024-06-11T13:28:19.462695+0000","last_became_peered":"2024-06-11T13:28:19.462695+0000","last_unstale":"2024-06-11T13:28:49.942612+0000","last_undegraded":"2024-06-11T13:28:49.942612+0000","last_fullsized":"2024-06-11T13:28:49.942612+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T22:45:18.631444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"68'28","reported_seq":90,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:29:28.243370+0000","last_change":"2024-06-11T13:28:15.428569+0000","last_active":"2024-06-11T13:29:28.243370+0000","last_peered":"2024-06-11T13:29:28.243370+0000","last_clean":"2024-06-11T13:29:28.243370+0000","last_became_active":"2024-06-11T13:28:15.428267+0000","last_became_peered":"2024-06-11T13:28:15.428267+0000","last_unstale":"2024-06-11T13:29:28.243370+0000","last_undegraded":"2024-06-11T13:29:28.243370+0000","last_fullsized":"2024-06-11T13:29:28.243370+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T18:39:04.384388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.050675+0000","last_change":"2024-06-11T13:28:13.182915+0000","last_active":"2024-06-11T13:28:51.050675+0000","last_peered":"2024-06-11T13:28:51.050675+0000","last_clean":"2024-06-11T13:28:51.050675+0000","last_became_active":"2024-06-11T13:28:13.182608+0000","last_became_peered":"2024-06-11T13:28:13.182608+0000","last_unstale":"2024-06-11T13:28:51.050675+0000","last_undegraded":"2024-06-11T13:28:51.050675+0000","last_fullsized":"2024-06-11T13:28:51.050675+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T23:24:44.892880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.943622+0000","last_change":"2024-06-11T13:28:17.460256+0000","last_active":"2024-06-11T13:28:49.943622+0000","last_peered":"2024-06-11T13:28:49.943622+0000","last_clean":"2024-06-11T13:28:49.943622+0000","last_became_active":"2024-06-11T13:28:17.459765+0000","last_became_peered":"2024-06-11T13:28:17.459765+0000","last_unstale":"2024-06-11T13:28:49.943622+0000","last_undegraded":"2024-06-11T13:28:49.943622+0000","last_fullsized":"2024-06-11T13:28:49.943622+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T22:36:22.716109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.033165+0000","last_change":"2024-06-11T13:28:19.459741+0000","last_active":"2024-06-11T13:28:50.033165+0000","last_peered":"2024-06-11T13:28:50.033165+0000","last_clean":"2024-06-11T13:28:50.033165+0000","last_became_active":"2024-06-11T13:28:19.459289+0000","last_became_peered":"2024-06-11T13:28:19.459289+0000","last_unstale":"2024-06-11T13:28:50.033165+0000","last_undegraded":"2024-06-11T13:28:50.033165+0000","last_fullsized":"2024-06-11T13:28:50.033165+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T22:08:28.682276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"68'13","reported_seq":52,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.040991+0000","last_change":"2024-06-11T13:28:15.432290+0000","last_active":"2024-06-11T13:28:50.040991+0000","last_peered":"2024-06-11T13:28:50.040991+0000","last_clean":"2024-06-11T13:28:50.040991+0000","last_became_active":"2024-06-11T13:28:15.431483+0000","last_became_peered":"2024-06-11T13:28:15.431483+0000","last_unstale":"2024-06-11T13:28:50.040991+0000","last_undegraded":"2024-06-11T13:28:50.040991+0000","last_fullsized":"2024-06-11T13:28:50.040991+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T16:52:06.965341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"69'5","reported_seq":116,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:29:34.805631+0000","last_change":"2024-06-11T13:28:17.451284+0000","last_active":"2024-06-11T13:29:34.805631+0000","last_peered":"2024-06-11T13:29:34.805631+0000","last_clean":"2024-06-11T13:29:34.805631+0000","last_became_active":"2024-06-11T13:28:13.188087+0000","last_became_peered":"2024-06-11T13:28:13.188087+0000","last_unstale":"2024-06-11T13:29:34.805631+0000","last_undegraded":"2024-06-11T13:29:34.805631+0000","last_fullsized":"2024-06-11T13:29:34.805631+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T22:58:03.955354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000481383,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":77,"num_read_kb":72,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.944927+0000","last_change":"2024-06-11T13:28:13.176414+0000","last_active":"2024-06-11T13:28:49.944927+0000","last_peered":"2024-06-11T13:28:49.944927+0000","last_clean":"2024-06-11T13:28:49.944927+0000","last_became_active":"2024-06-11T13:28:13.176108+0000","last_became_peered":"2024-06-11T13:28:13.176108+0000","last_unstale":"2024-06-11T13:28:49.944927+0000","last_undegraded":"2024-06-11T13:28:49.944927+0000","last_fullsized":"2024-06-11T13:28:49.944927+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-13T01:08:57.498698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.945323+0000","last_change":"2024-06-11T13:28:17.450721+0000","last_active":"2024-06-11T13:28:49.945323+0000","last_peered":"2024-06-11T13:28:49.945323+0000","last_clean":"2024-06-11T13:28:49.945323+0000","last_became_active":"2024-06-11T13:28:17.450357+0000","last_became_peered":"2024-06-11T13:28:17.450357+0000","last_unstale":"2024-06-11T13:28:49.945323+0000","last_undegraded":"2024-06-11T13:28:49.945323+0000","last_fullsized":"2024-06-11T13:28:49.945323+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T22:36:05.731389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.945158+0000","last_change":"2024-06-11T13:28:19.548117+0000","last_active":"2024-06-11T13:28:49.945158+0000","last_peered":"2024-06-11T13:28:49.945158+0000","last_clean":"2024-06-11T13:28:49.945158+0000","last_became_active":"2024-06-11T13:28:19.547782+0000","last_became_peered":"2024-06-11T13:28:19.547782+0000","last_unstale":"2024-06-11T13:28:49.945158+0000","last_undegraded":"2024-06-11T13:28:49.945158+0000","last_fullsized":"2024-06-11T13:28:49.945158+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T21:25:59.597610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"68'12","reported_seq":43,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.944052+0000","last_change":"2024-06-11T13:28:15.432010+0000","last_active":"2024-06-11T13:28:49.944052+0000","last_peered":"2024-06-11T13:28:49.944052+0000","last_clean":"2024-06-11T13:28:49.944052+0000","last_became_active":"2024-06-11T13:28:15.431515+0000","last_became_peered":"2024-06-11T13:28:15.431515+0000","last_unstale":"2024-06-11T13:28:49.944052+0000","last_undegraded":"2024-06-11T13:28:49.944052+0000","last_fullsized":"2024-06-11T13:28:49.944052+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T13:40:06.929108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":36,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.943001+0000","last_change":"2024-06-11T13:28:17.451445+0000","last_active":"2024-06-11T13:28:49.943001+0000","last_peered":"2024-06-11T13:28:49.943001+0000","last_clean":"2024-06-11T13:28:49.943001+0000","last_became_active":"2024-06-11T13:28:13.177645+0000","last_became_peered":"2024-06-11T13:28:13.177645+0000","last_unstale":"2024-06-11T13:28:49.943001+0000","last_undegraded":"2024-06-11T13:28:49.943001+0000","last_fullsized":"2024-06-11T13:28:49.943001+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T19:31:18.415958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030853080000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.944658+0000","last_change":"2024-06-11T13:28:13.185313+0000","last_active":"2024-06-11T13:28:49.944658+0000","last_peered":"2024-06-11T13:28:49.944658+0000","last_clean":"2024-06-11T13:28:49.944658+0000","last_became_active":"2024-06-11T13:28:13.185045+0000","last_became_peered":"2024-06-11T13:28:13.185045+0000","last_unstale":"2024-06-11T13:28:49.944658+0000","last_undegraded":"2024-06-11T13:28:49.944658+0000","last_fullsized":"2024-06-11T13:28:49.944658+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T18:40:25.221292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.050934+0000","last_change":"2024-06-11T13:28:17.455912+0000","last_active":"2024-06-11T13:28:51.050934+0000","last_peered":"2024-06-11T13:28:51.050934+0000","last_clean":"2024-06-11T13:28:51.050934+0000","last_became_active":"2024-06-11T13:28:17.455445+0000","last_became_peered":"2024-06-11T13:28:17.455445+0000","last_unstale":"2024-06-11T13:28:51.050934+0000","last_undegraded":"2024-06-11T13:28:51.050934+0000","last_fullsized":"2024-06-11T13:28:51.050934+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T21:56:37.105430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.039704+0000","last_change":"2024-06-11T13:28:19.461348+0000","last_active":"2024-06-11T13:28:50.039704+0000","last_peered":"2024-06-11T13:28:50.039704+0000","last_clean":"2024-06-11T13:28:50.039704+0000","last_became_active":"2024-06-11T13:28:19.460988+0000","last_became_peered":"2024-06-11T13:28:19.460988+0000","last_unstale":"2024-06-11T13:28:50.039704+0000","last_undegraded":"2024-06-11T13:28:50.039704+0000","last_fullsized":"2024-06-11T13:28:50.039704+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T23:37:04.637331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"68'16","reported_seq":64,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:29:28.243598+0000","last_change":"2024-06-11T13:28:15.599233+0000","last_active":"2024-06-11T13:29:28.243598+0000","last_peered":"2024-06-11T13:29:28.243598+0000","last_clean":"2024-06-11T13:29:28.243598+0000","last_became_active":"2024-06-11T13:28:15.598906+0000","last_became_peered":"2024-06-11T13:28:15.598906+0000","last_unstale":"2024-06-11T13:29:28.243598+0000","last_undegraded":"2024-06-11T13:29:28.243598+0000","last_fullsized":"2024-06-11T13:29:28.243598+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T17:58:57.982815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"65'2","reported_seq":38,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.031880+0000","last_change":"2024-06-11T13:28:17.440514+0000","last_active":"2024-06-11T13:28:50.031880+0000","last_peered":"2024-06-11T13:28:50.031880+0000","last_clean":"2024-06-11T13:28:50.031880+0000","last_became_active":"2024-06-11T13:28:13.184678+0000","last_became_peered":"2024-06-11T13:28:13.184678+0000","last_unstale":"2024-06-11T13:28:50.031880+0000","last_undegraded":"2024-06-11T13:28:50.031880+0000","last_fullsized":"2024-06-11T13:28:50.031880+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T22:44:49.176865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065293199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.050629+0000","last_change":"2024-06-11T13:28:13.180355+0000","last_active":"2024-06-11T13:28:51.050629+0000","last_peered":"2024-06-11T13:28:51.050629+0000","last_clean":"2024-06-11T13:28:51.050629+0000","last_became_active":"2024-06-11T13:28:13.179885+0000","last_became_peered":"2024-06-11T13:28:13.179885+0000","last_unstale":"2024-06-11T13:28:51.050629+0000","last_undegraded":"2024-06-11T13:28:51.050629+0000","last_fullsized":"2024-06-11T13:28:51.050629+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T19:29:49.164602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"70'128","reported_seq":273,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:52.020828+0000","last_change":"2024-06-11T13:28:00.364232+0000","last_active":"2024-06-11T13:28:52.020828+0000","last_peered":"2024-06-11T13:28:52.020828+0000","last_clean":"2024-06-11T13:28:52.020828+0000","last_became_active":"2024-06-11T13:28:00.363797+0000","last_became_peered":"2024-06-11T13:28:00.363797+0000","last_unstale":"2024-06-11T13:28:52.020828+0000","last_undegraded":"2024-06-11T13:28:52.020828+0000","last_fullsized":"2024-06-11T13:28:52.020828+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:25:59.379125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:25:59.379125+0000","last_clean_scrub_stamp":"2024-06-11T13:25:59.379125+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T18:23:25.764067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.945016+0000","last_change":"2024-06-11T13:28:17.453607+0000","last_active":"2024-06-11T13:28:49.945016+0000","last_peered":"2024-06-11T13:28:49.945016+0000","last_clean":"2024-06-11T13:28:49.945016+0000","last_became_active":"2024-06-11T13:28:17.452331+0000","last_became_peered":"2024-06-11T13:28:17.452331+0000","last_unstale":"2024-06-11T13:28:49.945016+0000","last_undegraded":"2024-06-11T13:28:49.945016+0000","last_fullsized":"2024-06-11T13:28:49.945016+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T17:05:44.373332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.051034+0000","last_change":"2024-06-11T13:28:19.457503+0000","last_active":"2024-06-11T13:28:51.051034+0000","last_peered":"2024-06-11T13:28:51.051034+0000","last_clean":"2024-06-11T13:28:51.051034+0000","last_became_active":"2024-06-11T13:28:19.457112+0000","last_became_peered":"2024-06-11T13:28:19.457112+0000","last_unstale":"2024-06-11T13:28:51.051034+0000","last_undegraded":"2024-06-11T13:28:51.051034+0000","last_fullsized":"2024-06-11T13:28:51.051034+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T19:12:49.788901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"68'19","reported_seq":56,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.549128+0000","last_change":"2024-06-11T13:28:15.597859+0000","last_active":"2024-06-11T13:28:50.549128+0000","last_peered":"2024-06-11T13:28:50.549128+0000","last_clean":"2024-06-11T13:28:50.549128+0000","last_became_active":"2024-06-11T13:28:15.597494+0000","last_became_peered":"2024-06-11T13:28:15.597494+0000","last_unstale":"2024-06-11T13:28:50.549128+0000","last_undegraded":"2024-06-11T13:28:50.549128+0000","last_fullsized":"2024-06-11T13:28:50.549128+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T22:07:14.319927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.947015+0000","last_change":"2024-06-11T13:28:13.175188+0000","last_active":"2024-06-11T13:28:49.947015+0000","last_peered":"2024-06-11T13:28:49.947015+0000","last_clean":"2024-06-11T13:28:49.947015+0000","last_became_active":"2024-06-11T13:28:13.174844+0000","last_became_peered":"2024-06-11T13:28:13.174844+0000","last_unstale":"2024-06-11T13:28:49.947015+0000","last_undegraded":"2024-06-11T13:28:49.947015+0000","last_fullsized":"2024-06-11T13:28:49.947015+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T15:02:06.405290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.946221+0000","last_change":"2024-06-11T13:28:17.454294+0000","last_active":"2024-06-11T13:28:49.946221+0000","last_peered":"2024-06-11T13:28:49.946221+0000","last_clean":"2024-06-11T13:28:49.946221+0000","last_became_active":"2024-06-11T13:28:17.453974+0000","last_became_peered":"2024-06-11T13:28:17.453974+0000","last_unstale":"2024-06-11T13:28:49.946221+0000","last_undegraded":"2024-06-11T13:28:49.946221+0000","last_fullsized":"2024-06-11T13:28:49.946221+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T19:18:14.396961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.944699+0000","last_change":"2024-06-11T13:28:19.461481+0000","last_active":"2024-06-11T13:28:49.944699+0000","last_peered":"2024-06-11T13:28:49.944699+0000","last_clean":"2024-06-11T13:28:49.944699+0000","last_became_active":"2024-06-11T13:28:19.461198+0000","last_became_peered":"2024-06-11T13:28:19.461198+0000","last_unstale":"2024-06-11T13:28:49.944699+0000","last_undegraded":"2024-06-11T13:28:49.944699+0000","last_fullsized":"2024-06-11T13:28:49.944699+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T13:33:08.490127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"68'9","reported_seq":41,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.943210+0000","last_change":"2024-06-11T13:28:15.430307+0000","last_active":"2024-06-11T13:28:49.943210+0000","last_peered":"2024-06-11T13:28:49.943210+0000","last_clean":"2024-06-11T13:28:49.943210+0000","last_became_active":"2024-06-11T13:28:15.429052+0000","last_became_peered":"2024-06-11T13:28:15.429052+0000","last_unstale":"2024-06-11T13:28:49.943210+0000","last_undegraded":"2024-06-11T13:28:49.943210+0000","last_fullsized":"2024-06-11T13:28:49.943210+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T22:16:55.706916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.040105+0000","last_change":"2024-06-11T13:28:13.189710+0000","last_active":"2024-06-11T13:28:50.040105+0000","last_peered":"2024-06-11T13:28:50.040105+0000","last_clean":"2024-06-11T13:28:50.040105+0000","last_became_active":"2024-06-11T13:28:13.189153+0000","last_became_peered":"2024-06-11T13:28:13.189153+0000","last_unstale":"2024-06-11T13:28:50.040105+0000","last_undegraded":"2024-06-11T13:28:50.040105+0000","last_fullsized":"2024-06-11T13:28:50.040105+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T13:43:52.827970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.945047+0000","last_change":"2024-06-11T13:28:17.443853+0000","last_active":"2024-06-11T13:28:49.945047+0000","last_peered":"2024-06-11T13:28:49.945047+0000","last_clean":"2024-06-11T13:28:49.945047+0000","last_became_active":"2024-06-11T13:28:17.443520+0000","last_became_peered":"2024-06-11T13:28:17.443520+0000","last_unstale":"2024-06-11T13:28:49.945047+0000","last_undegraded":"2024-06-11T13:28:49.945047+0000","last_fullsized":"2024-06-11T13:28:49.945047+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T17:44:22.792663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.942888+0000","last_change":"2024-06-11T13:28:19.461977+0000","last_active":"2024-06-11T13:28:49.942888+0000","last_peered":"2024-06-11T13:28:49.942888+0000","last_clean":"2024-06-11T13:28:49.942888+0000","last_became_active":"2024-06-11T13:28:19.460370+0000","last_became_peered":"2024-06-11T13:28:19.460370+0000","last_unstale":"2024-06-11T13:28:49.942888+0000","last_undegraded":"2024-06-11T13:28:49.942888+0000","last_fullsized":"2024-06-11T13:28:49.942888+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T18:19:29.644278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"68'15","reported_seq":50,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.050290+0000","last_change":"2024-06-11T13:28:15.599585+0000","last_active":"2024-06-11T13:28:51.050290+0000","last_peered":"2024-06-11T13:28:51.050290+0000","last_clean":"2024-06-11T13:28:51.050290+0000","last_became_active":"2024-06-11T13:28:15.599247+0000","last_became_peered":"2024-06-11T13:28:15.599247+0000","last_unstale":"2024-06-11T13:28:51.050290+0000","last_undegraded":"2024-06-11T13:28:51.050290+0000","last_fullsized":"2024-06-11T13:28:51.050290+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T16:10:15.007390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.946012+0000","last_change":"2024-06-11T13:28:13.181623+0000","last_active":"2024-06-11T13:28:49.946012+0000","last_peered":"2024-06-11T13:28:49.946012+0000","last_clean":"2024-06-11T13:28:49.946012+0000","last_became_active":"2024-06-11T13:28:13.180442+0000","last_became_peered":"2024-06-11T13:28:13.180442+0000","last_unstale":"2024-06-11T13:28:49.946012+0000","last_undegraded":"2024-06-11T13:28:49.946012+0000","last_fullsized":"2024-06-11T13:28:49.946012+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T21:19:42.863903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"68'8","reported_seq":46,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:29:28.243594+0000","last_change":"2024-06-11T13:28:17.454160+0000","last_active":"2024-06-11T13:29:28.243594+0000","last_peered":"2024-06-11T13:29:28.243594+0000","last_clean":"2024-06-11T13:29:28.243594+0000","last_became_active":"2024-06-11T13:28:17.453879+0000","last_became_peered":"2024-06-11T13:28:17.453879+0000","last_unstale":"2024-06-11T13:29:28.243594+0000","last_undegraded":"2024-06-11T13:29:28.243594+0000","last_fullsized":"2024-06-11T13:29:28.243594+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T22:37:47.435002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.050392+0000","last_change":"2024-06-11T13:28:19.548196+0000","last_active":"2024-06-11T13:28:51.050392+0000","last_peered":"2024-06-11T13:28:51.050392+0000","last_clean":"2024-06-11T13:28:51.050392+0000","last_became_active":"2024-06-11T13:28:19.547915+0000","last_became_peered":"2024-06-11T13:28:19.547915+0000","last_unstale":"2024-06-11T13:28:51.050392+0000","last_undegraded":"2024-06-11T13:28:51.050392+0000","last_fullsized":"2024-06-11T13:28:51.050392+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T17:18:55.545802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"68'12","reported_seq":48,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.454357+0000","last_change":"2024-06-11T13:28:15.423664+0000","last_active":"2024-06-11T13:28:51.454357+0000","last_peered":"2024-06-11T13:28:51.454357+0000","last_clean":"2024-06-11T13:28:51.454357+0000","last_became_active":"2024-06-11T13:28:15.423353+0000","last_became_peered":"2024-06-11T13:28:15.423353+0000","last_unstale":"2024-06-11T13:28:51.454357+0000","last_undegraded":"2024-06-11T13:28:51.454357+0000","last_fullsized":"2024-06-11T13:28:51.454357+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T19:12:37.550194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"61'2","reported_seq":41,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.455075+0000","last_change":"2024-06-11T13:28:13.181088+0000","last_active":"2024-06-11T13:28:51.455075+0000","last_peered":"2024-06-11T13:28:51.455075+0000","last_clean":"2024-06-11T13:28:51.455075+0000","last_became_active":"2024-06-11T13:28:13.179671+0000","last_became_peered":"2024-06-11T13:28:13.179671+0000","last_unstale":"2024-06-11T13:28:51.455075+0000","last_undegraded":"2024-06-11T13:28:51.455075+0000","last_fullsized":"2024-06-11T13:28:51.455075+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T19:33:59.988970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.946208+0000","last_change":"2024-06-11T13:28:17.451962+0000","last_active":"2024-06-11T13:28:49.946208+0000","last_peered":"2024-06-11T13:28:49.946208+0000","last_clean":"2024-06-11T13:28:49.946208+0000","last_became_active":"2024-06-11T13:28:17.451641+0000","last_became_peered":"2024-06-11T13:28:17.451641+0000","last_unstale":"2024-06-11T13:28:49.946208+0000","last_undegraded":"2024-06-11T13:28:49.946208+0000","last_fullsized":"2024-06-11T13:28:49.946208+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T21:30:42.621687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.031943+0000","last_change":"2024-06-11T13:28:19.465914+0000","last_active":"2024-06-11T13:28:50.031943+0000","last_peered":"2024-06-11T13:28:50.031943+0000","last_clean":"2024-06-11T13:28:50.031943+0000","last_became_active":"2024-06-11T13:28:19.464999+0000","last_became_peered":"2024-06-11T13:28:19.464999+0000","last_unstale":"2024-06-11T13:28:50.031943+0000","last_undegraded":"2024-06-11T13:28:50.031943+0000","last_fullsized":"2024-06-11T13:28:50.031943+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T23:36:56.227801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"68'9","reported_seq":41,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.944227+0000","last_change":"2024-06-11T13:28:15.599340+0000","last_active":"2024-06-11T13:28:49.944227+0000","last_peered":"2024-06-11T13:28:49.944227+0000","last_clean":"2024-06-11T13:28:49.944227+0000","last_became_active":"2024-06-11T13:28:15.599052+0000","last_became_peered":"2024-06-11T13:28:15.599052+0000","last_unstale":"2024-06-11T13:28:49.944227+0000","last_undegraded":"2024-06-11T13:28:49.944227+0000","last_fullsized":"2024-06-11T13:28:49.944227+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T21:12:32.237748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.946288+0000","last_change":"2024-06-11T13:28:13.182839+0000","last_active":"2024-06-11T13:28:49.946288+0000","last_peered":"2024-06-11T13:28:49.946288+0000","last_clean":"2024-06-11T13:28:49.946288+0000","last_became_active":"2024-06-11T13:28:13.182011+0000","last_became_peered":"2024-06-11T13:28:13.182011+0000","last_unstale":"2024-06-11T13:28:49.946288+0000","last_undegraded":"2024-06-11T13:28:49.946288+0000","last_fullsized":"2024-06-11T13:28:49.946288+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T20:58:31.725113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.033448+0000","last_change":"2024-06-11T13:28:17.453737+0000","last_active":"2024-06-11T13:28:50.033448+0000","last_peered":"2024-06-11T13:28:50.033448+0000","last_clean":"2024-06-11T13:28:50.033448+0000","last_became_active":"2024-06-11T13:28:17.453353+0000","last_became_peered":"2024-06-11T13:28:17.453353+0000","last_unstale":"2024-06-11T13:28:50.033448+0000","last_undegraded":"2024-06-11T13:28:50.033448+0000","last_fullsized":"2024-06-11T13:28:50.033448+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T16:57:34.452332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.946436+0000","last_change":"2024-06-11T13:28:19.457532+0000","last_active":"2024-06-11T13:28:49.946436+0000","last_peered":"2024-06-11T13:28:49.946436+0000","last_clean":"2024-06-11T13:28:49.946436+0000","last_became_active":"2024-06-11T13:28:19.457210+0000","last_became_peered":"2024-06-11T13:28:19.457210+0000","last_unstale":"2024-06-11T13:28:49.946436+0000","last_undegraded":"2024-06-11T13:28:49.946436+0000","last_fullsized":"2024-06-11T13:28:49.946436+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T23:03:22.695531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.548955+0000","last_change":"2024-06-11T13:28:13.185332+0000","last_active":"2024-06-11T13:28:50.548955+0000","last_peered":"2024-06-11T13:28:50.548955+0000","last_clean":"2024-06-11T13:28:50.548955+0000","last_became_active":"2024-06-11T13:28:13.183098+0000","last_became_peered":"2024-06-11T13:28:13.183098+0000","last_unstale":"2024-06-11T13:28:50.548955+0000","last_undegraded":"2024-06-11T13:28:50.548955+0000","last_fullsized":"2024-06-11T13:28:50.548955+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T19:07:47.442319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"68'6","reported_seq":34,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.033548+0000","last_change":"2024-06-11T13:28:15.598731+0000","last_active":"2024-06-11T13:28:50.033548+0000","last_peered":"2024-06-11T13:28:50.033548+0000","last_clean":"2024-06-11T13:28:50.033548+0000","last_became_active":"2024-06-11T13:28:15.598410+0000","last_became_peered":"2024-06-11T13:28:15.598410+0000","last_unstale":"2024-06-11T13:28:50.033548+0000","last_undegraded":"2024-06-11T13:28:50.033548+0000","last_fullsized":"2024-06-11T13:28:50.033548+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T18:13:52.552294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.050832+0000","last_change":"2024-06-11T13:28:17.456431+0000","last_active":"2024-06-11T13:28:51.050832+0000","last_peered":"2024-06-11T13:28:51.050832+0000","last_clean":"2024-06-11T13:28:51.050832+0000","last_became_active":"2024-06-11T13:28:17.456112+0000","last_became_peered":"2024-06-11T13:28:17.456112+0000","last_unstale":"2024-06-11T13:28:51.050832+0000","last_undegraded":"2024-06-11T13:28:51.050832+0000","last_fullsized":"2024-06-11T13:28:51.050832+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T18:54:14.165022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.945120+0000","last_change":"2024-06-11T13:28:19.548544+0000","last_active":"2024-06-11T13:28:49.945120+0000","last_peered":"2024-06-11T13:28:49.945120+0000","last_clean":"2024-06-11T13:28:49.945120+0000","last_became_active":"2024-06-11T13:28:19.548259+0000","last_became_peered":"2024-06-11T13:28:19.548259+0000","last_unstale":"2024-06-11T13:28:49.945120+0000","last_undegraded":"2024-06-11T13:28:49.945120+0000","last_fullsized":"2024-06-11T13:28:49.945120+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T20:54:44.002109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"68'10","reported_seq":40,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.032738+0000","last_change":"2024-06-11T13:28:15.597642+0000","last_active":"2024-06-11T13:28:50.032738+0000","last_peered":"2024-06-11T13:28:50.032738+0000","last_clean":"2024-06-11T13:28:50.032738+0000","last_became_active":"2024-06-11T13:28:15.597175+0000","last_became_peered":"2024-06-11T13:28:15.597175+0000","last_unstale":"2024-06-11T13:28:50.032738+0000","last_undegraded":"2024-06-11T13:28:50.032738+0000","last_fullsized":"2024-06-11T13:28:50.032738+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T21:39:28.191888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.050623+0000","last_change":"2024-06-11T13:28:13.181755+0000","last_active":"2024-06-11T13:28:51.050623+0000","last_peered":"2024-06-11T13:28:51.050623+0000","last_clean":"2024-06-11T13:28:51.050623+0000","last_became_active":"2024-06-11T13:28:13.180056+0000","last_became_peered":"2024-06-11T13:28:13.180056+0000","last_unstale":"2024-06-11T13:28:51.050623+0000","last_undegraded":"2024-06-11T13:28:51.050623+0000","last_fullsized":"2024-06-11T13:28:51.050623+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T22:04:35.756359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"68'8","reported_seq":46,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:29:28.243558+0000","last_change":"2024-06-11T13:28:17.456221+0000","last_active":"2024-06-11T13:29:28.243558+0000","last_peered":"2024-06-11T13:29:28.243558+0000","last_clean":"2024-06-11T13:29:28.243558+0000","last_became_active":"2024-06-11T13:28:17.455856+0000","last_became_peered":"2024-06-11T13:28:17.455856+0000","last_unstale":"2024-06-11T13:29:28.243558+0000","last_undegraded":"2024-06-11T13:29:28.243558+0000","last_fullsized":"2024-06-11T13:29:28.243558+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T23:48:17.791639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.943805+0000","last_change":"2024-06-11T13:28:19.462632+0000","last_active":"2024-06-11T13:28:49.943805+0000","last_peered":"2024-06-11T13:28:49.943805+0000","last_clean":"2024-06-11T13:28:49.943805+0000","last_became_active":"2024-06-11T13:28:19.462089+0000","last_became_peered":"2024-06-11T13:28:19.462089+0000","last_unstale":"2024-06-11T13:28:49.943805+0000","last_undegraded":"2024-06-11T13:28:49.943805+0000","last_fullsized":"2024-06-11T13:28:49.943805+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-13T00:41:07.090299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"68'9","reported_seq":41,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.050842+0000","last_change":"2024-06-11T13:28:15.598208+0000","last_active":"2024-06-11T13:28:51.050842+0000","last_peered":"2024-06-11T13:28:51.050842+0000","last_clean":"2024-06-11T13:28:51.050842+0000","last_became_active":"2024-06-11T13:28:15.597896+0000","last_became_peered":"2024-06-11T13:28:15.597896+0000","last_unstale":"2024-06-11T13:28:51.050842+0000","last_undegraded":"2024-06-11T13:28:51.050842+0000","last_fullsized":"2024-06-11T13:28:51.050842+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T15:53:46.789334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"61'1","reported_seq":35,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.943443+0000","last_change":"2024-06-11T13:28:13.181673+0000","last_active":"2024-06-11T13:28:49.943443+0000","last_peered":"2024-06-11T13:28:49.943443+0000","last_clean":"2024-06-11T13:28:49.943443+0000","last_became_active":"2024-06-11T13:28:13.181384+0000","last_became_peered":"2024-06-11T13:28:13.181384+0000","last_unstale":"2024-06-11T13:28:49.943443+0000","last_undegraded":"2024-06-11T13:28:49.943443+0000","last_fullsized":"2024-06-11T13:28:49.943443+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T16:20:04.406915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"68'8","reported_seq":46,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:29:28.243520+0000","last_change":"2024-06-11T13:28:17.448787+0000","last_active":"2024-06-11T13:29:28.243520+0000","last_peered":"2024-06-11T13:29:28.243520+0000","last_clean":"2024-06-11T13:29:28.243520+0000","last_became_active":"2024-06-11T13:28:17.448475+0000","last_became_peered":"2024-06-11T13:28:17.448475+0000","last_unstale":"2024-06-11T13:29:28.243520+0000","last_undegraded":"2024-06-11T13:29:28.243520+0000","last_fullsized":"2024-06-11T13:29:28.243520+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T17:44:36.219808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.454963+0000","last_change":"2024-06-11T13:28:19.455787+0000","last_active":"2024-06-11T13:28:51.454963+0000","last_peered":"2024-06-11T13:28:51.454963+0000","last_clean":"2024-06-11T13:28:51.454963+0000","last_became_active":"2024-06-11T13:28:19.455507+0000","last_became_peered":"2024-06-11T13:28:19.455507+0000","last_unstale":"2024-06-11T13:28:51.454963+0000","last_undegraded":"2024-06-11T13:28:51.454963+0000","last_fullsized":"2024-06-11T13:28:51.454963+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T16:57:52.936758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"68'9","reported_seq":41,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.039870+0000","last_change":"2024-06-11T13:28:15.429400+0000","last_active":"2024-06-11T13:28:50.039870+0000","last_peered":"2024-06-11T13:28:50.039870+0000","last_clean":"2024-06-11T13:28:50.039870+0000","last_became_active":"2024-06-11T13:28:15.429093+0000","last_became_peered":"2024-06-11T13:28:15.429093+0000","last_unstale":"2024-06-11T13:28:50.039870+0000","last_undegraded":"2024-06-11T13:28:50.039870+0000","last_fullsized":"2024-06-11T13:28:50.039870+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T13:43:41.146514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.548957+0000","last_change":"2024-06-11T13:28:13.182165+0000","last_active":"2024-06-11T13:28:50.548957+0000","last_peered":"2024-06-11T13:28:50.548957+0000","last_clean":"2024-06-11T13:28:50.548957+0000","last_became_active":"2024-06-11T13:28:13.181895+0000","last_became_peered":"2024-06-11T13:28:13.181895+0000","last_unstale":"2024-06-11T13:28:50.548957+0000","last_undegraded":"2024-06-11T13:28:50.548957+0000","last_fullsized":"2024-06-11T13:28:50.548957+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T13:44:29.552937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.032032+0000","last_change":"2024-06-11T13:28:17.454531+0000","last_active":"2024-06-11T13:28:50.032032+0000","last_peered":"2024-06-11T13:28:50.032032+0000","last_clean":"2024-06-11T13:28:50.032032+0000","last_became_active":"2024-06-11T13:28:17.454160+0000","last_became_peered":"2024-06-11T13:28:17.454160+0000","last_unstale":"2024-06-11T13:28:50.032032+0000","last_undegraded":"2024-06-11T13:28:50.032032+0000","last_fullsized":"2024-06-11T13:28:50.032032+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T13:59:47.847881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.942610+0000","last_change":"2024-06-11T13:28:19.463034+0000","last_active":"2024-06-11T13:28:49.942610+0000","last_peered":"2024-06-11T13:28:49.942610+0000","last_clean":"2024-06-11T13:28:49.942610+0000","last_became_active":"2024-06-11T13:28:19.461138+0000","last_became_peered":"2024-06-11T13:28:19.461138+0000","last_unstale":"2024-06-11T13:28:49.942610+0000","last_undegraded":"2024-06-11T13:28:49.942610+0000","last_fullsized":"2024-06-11T13:28:49.942610+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-13T01:16:08.991874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"68'11","reported_seq":44,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.454463+0000","last_change":"2024-06-11T13:28:15.419950+0000","last_active":"2024-06-11T13:28:51.454463+0000","last_peered":"2024-06-11T13:28:51.454463+0000","last_clean":"2024-06-11T13:28:51.454463+0000","last_became_active":"2024-06-11T13:28:15.419594+0000","last_became_peered":"2024-06-11T13:28:15.419594+0000","last_unstale":"2024-06-11T13:28:51.454463+0000","last_undegraded":"2024-06-11T13:28:51.454463+0000","last_fullsized":"2024-06-11T13:28:51.454463+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T22:27:46.009525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.040019+0000","last_change":"2024-06-11T13:28:13.189433+0000","last_active":"2024-06-11T13:28:50.040019+0000","last_peered":"2024-06-11T13:28:50.040019+0000","last_clean":"2024-06-11T13:28:50.040019+0000","last_became_active":"2024-06-11T13:28:13.189104+0000","last_became_peered":"2024-06-11T13:28:13.189104+0000","last_unstale":"2024-06-11T13:28:50.040019+0000","last_undegraded":"2024-06-11T13:28:50.040019+0000","last_fullsized":"2024-06-11T13:28:50.040019+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T19:37:38.570887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.039713+0000","last_change":"2024-06-11T13:28:17.454941+0000","last_active":"2024-06-11T13:28:50.039713+0000","last_peered":"2024-06-11T13:28:50.039713+0000","last_clean":"2024-06-11T13:28:50.039713+0000","last_became_active":"2024-06-11T13:28:17.454491+0000","last_became_peered":"2024-06-11T13:28:17.454491+0000","last_unstale":"2024-06-11T13:28:50.039713+0000","last_undegraded":"2024-06-11T13:28:50.039713+0000","last_fullsized":"2024-06-11T13:28:50.039713+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T20:07:52.471274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.031368+0000","last_change":"2024-06-11T13:28:19.463446+0000","last_active":"2024-06-11T13:28:50.031368+0000","last_peered":"2024-06-11T13:28:50.031368+0000","last_clean":"2024-06-11T13:28:50.031368+0000","last_became_active":"2024-06-11T13:28:19.463098+0000","last_became_peered":"2024-06-11T13:28:19.463098+0000","last_unstale":"2024-06-11T13:28:50.031368+0000","last_undegraded":"2024-06-11T13:28:50.031368+0000","last_fullsized":"2024-06-11T13:28:50.031368+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T19:28:15.335320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"68'4","reported_seq":31,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.031392+0000","last_change":"2024-06-11T13:28:15.430394+0000","last_active":"2024-06-11T13:28:50.031392+0000","last_peered":"2024-06-11T13:28:50.031392+0000","last_clean":"2024-06-11T13:28:50.031392+0000","last_became_active":"2024-06-11T13:28:15.429952+0000","last_became_peered":"2024-06-11T13:28:15.429952+0000","last_unstale":"2024-06-11T13:28:50.031392+0000","last_undegraded":"2024-06-11T13:28:50.031392+0000","last_fullsized":"2024-06-11T13:28:50.031392+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T18:38:56.317846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.050948+0000","last_change":"2024-06-11T13:28:13.178773+0000","last_active":"2024-06-11T13:28:51.050948+0000","last_peered":"2024-06-11T13:28:51.050948+0000","last_clean":"2024-06-11T13:28:51.050948+0000","last_became_active":"2024-06-11T13:28:13.178476+0000","last_became_peered":"2024-06-11T13:28:13.178476+0000","last_unstale":"2024-06-11T13:28:51.050948+0000","last_undegraded":"2024-06-11T13:28:51.050948+0000","last_fullsized":"2024-06-11T13:28:51.050948+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-13T00:39:08.981141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.548633+0000","last_change":"2024-06-11T13:28:17.455687+0000","last_active":"2024-06-11T13:28:50.548633+0000","last_peered":"2024-06-11T13:28:50.548633+0000","last_clean":"2024-06-11T13:28:50.548633+0000","last_became_active":"2024-06-11T13:28:17.455407+0000","last_became_peered":"2024-06-11T13:28:17.455407+0000","last_unstale":"2024-06-11T13:28:50.548633+0000","last_undegraded":"2024-06-11T13:28:50.548633+0000","last_fullsized":"2024-06-11T13:28:50.548633+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T19:18:38.407996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.944755+0000","last_change":"2024-06-11T13:28:19.459144+0000","last_active":"2024-06-11T13:28:49.944755+0000","last_peered":"2024-06-11T13:28:49.944755+0000","last_clean":"2024-06-11T13:28:49.944755+0000","last_became_active":"2024-06-11T13:28:19.458871+0000","last_became_peered":"2024-06-11T13:28:19.458871+0000","last_unstale":"2024-06-11T13:28:49.944755+0000","last_undegraded":"2024-06-11T13:28:49.944755+0000","last_fullsized":"2024-06-11T13:28:49.944755+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T16:56:25.393539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"68'11","reported_seq":44,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.031489+0000","last_change":"2024-06-11T13:28:15.597576+0000","last_active":"2024-06-11T13:28:50.031489+0000","last_peered":"2024-06-11T13:28:50.031489+0000","last_clean":"2024-06-11T13:28:50.031489+0000","last_became_active":"2024-06-11T13:28:15.597267+0000","last_became_peered":"2024-06-11T13:28:15.597267+0000","last_unstale":"2024-06-11T13:28:50.031489+0000","last_undegraded":"2024-06-11T13:28:50.031489+0000","last_fullsized":"2024-06-11T13:28:50.031489+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T22:36:35.613372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.549140+0000","last_change":"2024-06-11T13:28:13.183716+0000","last_active":"2024-06-11T13:28:50.549140+0000","last_peered":"2024-06-11T13:28:50.549140+0000","last_clean":"2024-06-11T13:28:50.549140+0000","last_became_active":"2024-06-11T13:28:13.182944+0000","last_became_peered":"2024-06-11T13:28:13.182944+0000","last_unstale":"2024-06-11T13:28:50.549140+0000","last_undegraded":"2024-06-11T13:28:50.549140+0000","last_fullsized":"2024-06-11T13:28:50.549140+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T21:46:49.799256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.944452+0000","last_change":"2024-06-11T13:28:17.454563+0000","last_active":"2024-06-11T13:28:49.944452+0000","last_peered":"2024-06-11T13:28:49.944452+0000","last_clean":"2024-06-11T13:28:49.944452+0000","last_became_active":"2024-06-11T13:28:17.453614+0000","last_became_peered":"2024-06-11T13:28:17.453614+0000","last_unstale":"2024-06-11T13:28:49.944452+0000","last_undegraded":"2024-06-11T13:28:49.944452+0000","last_fullsized":"2024-06-11T13:28:49.944452+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T13:40:40.512080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.031895+0000","last_change":"2024-06-11T13:28:19.550177+0000","last_active":"2024-06-11T13:28:50.031895+0000","last_peered":"2024-06-11T13:28:50.031895+0000","last_clean":"2024-06-11T13:28:50.031895+0000","last_became_active":"2024-06-11T13:28:19.549829+0000","last_became_peered":"2024-06-11T13:28:19.549829+0000","last_unstale":"2024-06-11T13:28:50.031895+0000","last_undegraded":"2024-06-11T13:28:50.031895+0000","last_fullsized":"2024-06-11T13:28:50.031895+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T22:13:54.754536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"68'1","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.944917+0000","last_change":"2024-06-11T13:28:19.459347+0000","last_active":"2024-06-11T13:28:49.944917+0000","last_peered":"2024-06-11T13:28:49.944917+0000","last_clean":"2024-06-11T13:28:49.944917+0000","last_became_active":"2024-06-11T13:28:19.458967+0000","last_became_peered":"2024-06-11T13:28:19.458967+0000","last_unstale":"2024-06-11T13:28:49.944917+0000","last_undegraded":"2024-06-11T13:28:49.944917+0000","last_fullsized":"2024-06-11T13:28:49.944917+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T22:42:00.605019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.050322+0000","last_change":"2024-06-11T13:28:13.179416+0000","last_active":"2024-06-11T13:28:51.050322+0000","last_peered":"2024-06-11T13:28:51.050322+0000","last_clean":"2024-06-11T13:28:51.050322+0000","last_became_active":"2024-06-11T13:28:13.178793+0000","last_became_peered":"2024-06-11T13:28:13.178793+0000","last_unstale":"2024-06-11T13:28:51.050322+0000","last_undegraded":"2024-06-11T13:28:51.050322+0000","last_fullsized":"2024-06-11T13:28:51.050322+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T22:28:56.806331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"68'10","reported_seq":40,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.943929+0000","last_change":"2024-06-11T13:28:15.599580+0000","last_active":"2024-06-11T13:28:49.943929+0000","last_peered":"2024-06-11T13:28:49.943929+0000","last_clean":"2024-06-11T13:28:49.943929+0000","last_became_active":"2024-06-11T13:28:15.599285+0000","last_became_peered":"2024-06-11T13:28:15.599285+0000","last_unstale":"2024-06-11T13:28:49.943929+0000","last_undegraded":"2024-06-11T13:28:49.943929+0000","last_fullsized":"2024-06-11T13:28:49.943929+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T17:31:18.047138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"68'8","reported_seq":49,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:29:28.243594+0000","last_change":"2024-06-11T13:28:17.458379+0000","last_active":"2024-06-11T13:29:28.243594+0000","last_peered":"2024-06-11T13:29:28.243594+0000","last_clean":"2024-06-11T13:29:28.243594+0000","last_became_active":"2024-06-11T13:28:17.458051+0000","last_became_peered":"2024-06-11T13:28:17.458051+0000","last_unstale":"2024-06-11T13:29:28.243594+0000","last_undegraded":"2024-06-11T13:29:28.243594+0000","last_fullsized":"2024-06-11T13:29:28.243594+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T16:16:25.577731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.039540+0000","last_change":"2024-06-11T13:28:19.456367+0000","last_active":"2024-06-11T13:28:50.039540+0000","last_peered":"2024-06-11T13:28:50.039540+0000","last_clean":"2024-06-11T13:28:50.039540+0000","last_became_active":"2024-06-11T13:28:19.456042+0000","last_became_peered":"2024-06-11T13:28:19.456042+0000","last_unstale":"2024-06-11T13:28:50.039540+0000","last_undegraded":"2024-06-11T13:28:50.039540+0000","last_fullsized":"2024-06-11T13:28:50.039540+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T18:35:49.905789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"61'1","reported_seq":30,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.031626+0000","last_change":"2024-06-11T13:28:13.178724+0000","last_active":"2024-06-11T13:28:50.031626+0000","last_peered":"2024-06-11T13:28:50.031626+0000","last_clean":"2024-06-11T13:28:50.031626+0000","last_became_active":"2024-06-11T13:28:13.178441+0000","last_became_peered":"2024-06-11T13:28:13.178441+0000","last_unstale":"2024-06-11T13:28:50.031626+0000","last_undegraded":"2024-06-11T13:28:50.031626+0000","last_fullsized":"2024-06-11T13:28:50.031626+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T16:44:02.621263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"68'11","reported_seq":44,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.549090+0000","last_change":"2024-06-11T13:28:15.427408+0000","last_active":"2024-06-11T13:28:50.549090+0000","last_peered":"2024-06-11T13:28:50.549090+0000","last_clean":"2024-06-11T13:28:50.549090+0000","last_became_active":"2024-06-11T13:28:15.427112+0000","last_became_peered":"2024-06-11T13:28:15.427112+0000","last_unstale":"2024-06-11T13:28:50.549090+0000","last_undegraded":"2024-06-11T13:28:50.549090+0000","last_fullsized":"2024-06-11T13:28:50.549090+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T22:17:47.146167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.943834+0000","last_change":"2024-06-11T13:28:17.457781+0000","last_active":"2024-06-11T13:28:49.943834+0000","last_peered":"2024-06-11T13:28:49.943834+0000","last_clean":"2024-06-11T13:28:49.943834+0000","last_became_active":"2024-06-11T13:28:17.457361+0000","last_became_peered":"2024-06-11T13:28:17.457361+0000","last_unstale":"2024-06-11T13:28:49.943834+0000","last_undegraded":"2024-06-11T13:28:49.943834+0000","last_fullsized":"2024-06-11T13:28:49.943834+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-13T00:45:57.100880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.454644+0000","last_change":"2024-06-11T13:28:19.548337+0000","last_active":"2024-06-11T13:28:51.454644+0000","last_peered":"2024-06-11T13:28:51.454644+0000","last_clean":"2024-06-11T13:28:51.454644+0000","last_became_active":"2024-06-11T13:28:19.548053+0000","last_became_peered":"2024-06-11T13:28:19.548053+0000","last_unstale":"2024-06-11T13:28:51.454644+0000","last_undegraded":"2024-06-11T13:28:51.454644+0000","last_fullsized":"2024-06-11T13:28:51.454644+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-13T01:10:51.290117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.549224+0000","last_change":"2024-06-11T13:28:13.185703+0000","last_active":"2024-06-11T13:28:50.549224+0000","last_peered":"2024-06-11T13:28:50.549224+0000","last_clean":"2024-06-11T13:28:50.549224+0000","last_became_active":"2024-06-11T13:28:13.185430+0000","last_became_peered":"2024-06-11T13:28:13.185430+0000","last_unstale":"2024-06-11T13:28:50.549224+0000","last_undegraded":"2024-06-11T13:28:50.549224+0000","last_fullsized":"2024-06-11T13:28:50.549224+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T21:35:22.667919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"68'15","reported_seq":50,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.946717+0000","last_change":"2024-06-11T13:28:15.420442+0000","last_active":"2024-06-11T13:28:49.946717+0000","last_peered":"2024-06-11T13:28:49.946717+0000","last_clean":"2024-06-11T13:28:49.946717+0000","last_became_active":"2024-06-11T13:28:15.419791+0000","last_became_peered":"2024-06-11T13:28:15.419791+0000","last_unstale":"2024-06-11T13:28:49.946717+0000","last_undegraded":"2024-06-11T13:28:49.946717+0000","last_fullsized":"2024-06-11T13:28:49.946717+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-13T01:26:10.838592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.039796+0000","last_change":"2024-06-11T13:28:17.458660+0000","last_active":"2024-06-11T13:28:50.039796+0000","last_peered":"2024-06-11T13:28:50.039796+0000","last_clean":"2024-06-11T13:28:50.039796+0000","last_became_active":"2024-06-11T13:28:17.457745+0000","last_became_peered":"2024-06-11T13:28:17.457745+0000","last_unstale":"2024-06-11T13:28:50.039796+0000","last_undegraded":"2024-06-11T13:28:50.039796+0000","last_fullsized":"2024-06-11T13:28:50.039796+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T14:29:05.352069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.033338+0000","last_change":"2024-06-11T13:28:19.550139+0000","last_active":"2024-06-11T13:28:50.033338+0000","last_peered":"2024-06-11T13:28:50.033338+0000","last_clean":"2024-06-11T13:28:50.033338+0000","last_became_active":"2024-06-11T13:28:19.549738+0000","last_became_peered":"2024-06-11T13:28:19.549738+0000","last_unstale":"2024-06-11T13:28:50.033338+0000","last_undegraded":"2024-06-11T13:28:50.033338+0000","last_fullsized":"2024-06-11T13:28:50.033338+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-13T00:10:52.610842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.032229+0000","last_change":"2024-06-11T13:28:13.185050+0000","last_active":"2024-06-11T13:28:50.032229+0000","last_peered":"2024-06-11T13:28:50.032229+0000","last_clean":"2024-06-11T13:28:50.032229+0000","last_became_active":"2024-06-11T13:28:13.184692+0000","last_became_peered":"2024-06-11T13:28:13.184692+0000","last_unstale":"2024-06-11T13:28:50.032229+0000","last_undegraded":"2024-06-11T13:28:50.032229+0000","last_fullsized":"2024-06-11T13:28:50.032229+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T20:46:46.305342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"68'12","reported_seq":48,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.032996+0000","last_change":"2024-06-11T13:28:15.429779+0000","last_active":"2024-06-11T13:28:50.032996+0000","last_peered":"2024-06-11T13:28:50.032996+0000","last_clean":"2024-06-11T13:28:50.032996+0000","last_became_active":"2024-06-11T13:28:15.429361+0000","last_became_peered":"2024-06-11T13:28:15.429361+0000","last_unstale":"2024-06-11T13:28:50.032996+0000","last_undegraded":"2024-06-11T13:28:50.032996+0000","last_fullsized":"2024-06-11T13:28:50.032996+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T15:30:38.304227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.454594+0000","last_change":"2024-06-11T13:28:17.442983+0000","last_active":"2024-06-11T13:28:51.454594+0000","last_peered":"2024-06-11T13:28:51.454594+0000","last_clean":"2024-06-11T13:28:51.454594+0000","last_became_active":"2024-06-11T13:28:17.442719+0000","last_became_peered":"2024-06-11T13:28:17.442719+0000","last_unstale":"2024-06-11T13:28:51.454594+0000","last_undegraded":"2024-06-11T13:28:51.454594+0000","last_fullsized":"2024-06-11T13:28:51.454594+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T17:06:13.616524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":72,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":55,"seq":236223201303,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32432,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712720,"statfs":{"total":95995035648,"available":95961825280,"internally_reserved":0,"allocated":1286144,"data_stored":858751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462939,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32420,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712732,"statfs":{"total":95995035648,"available":95961837568,"internally_reserved":0,"allocated":1273856,"data_stored":858087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724575,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31836,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713316,"statfs":{"total":95995035648,"available":95962435584,"internally_reserved":0,"allocated":671744,"data_stored":267597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018916,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31860,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713292,"statfs":{"total":95995035648,"available":95962411008,"internally_reserved":0,"allocated":696320,"data_stored":268790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182441,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31876,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713276,"statfs":{"total":95995035648,"available":95962394624,"internally_reserved":0,"allocated":712704,"data_stored":269141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411373,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27672,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717480,"statfs":{"total":95995035648,"available":95966699520,"internally_reserved":0,"allocated":667648,"data_stored":267775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31896,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713256,"statfs":{"total":95995035648,"available":95962374144,"internally_reserved":0,"allocated":733184,"data_stored":269407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738424,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28284,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716868,"statfs":{"total":95995035648,"available":95966072832,"internally_reserved":0,"allocated":1294336,"data_stored":859453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-11T13:29:40.882 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-11T13:29:41.001 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:40 smithi107 podman[63384]: 2024-06-11 13:29:40.834114399 +0000 UTC m=+0.017383172 image pull 2bacad6d85d802cd176c146528adcc0e47a8a83e94bc4a8b7ba1a9b9ef2c65e7 quay.io/ceph/grafana:9.4.12 2024-06-11T13:29:41.133 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.a/config 2024-06-11T13:29:41.216 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 13:29:40 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-alertmanager-a[74565]: ts=2024-06-11T13:29:40.966Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.0021959s 2024-06-11T13:29:41.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-mon[33178]: pgmap v28: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:29:41.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-mon[33178]: from='client.34628 v1:172.21.15.57:0/2052731032' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-11T13:29:41.297 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:29:41.297 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 podman[63384]: 2024-06-11 13:29:41.146314551 +0000 UTC m=+0.329583310 container init c9d154c6ff5012639083fc205067f0c4948d6dbb87ee0a380578b88de3e52557 (image=quay.io/ceph/grafana:9.4.12, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a, maintainer=Grafana Labs ) 2024-06-11T13:29:41.297 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 podman[63384]: 2024-06-11 13:29:41.154731689 +0000 UTC m=+0.338000459 container start c9d154c6ff5012639083fc205067f0c4948d6dbb87ee0a380578b88de3e52557 (image=quay.io/ceph/grafana:9.4.12, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a, maintainer=Grafana Labs ) 2024-06-11T13:29:41.297 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 bash[63384]: c9d154c6ff5012639083fc205067f0c4948d6dbb87ee0a380578b88de3e52557 2024-06-11T13:29:41.297 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 systemd[1]: Started Ceph grafana.a for 98a6635e-27f5-11ef-bc9b-c7b262605968. 2024-06-11T13:29:41.502 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:41 smithi057 ceph-mon[40588]: pgmap v28: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:29:41.502 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:41 smithi057 ceph-mon[40588]: from='client.34628 v1:172.21.15.57:0/2052731032' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-11T13:29:41.502 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:41 smithi057 ceph-mon[29805]: pgmap v28: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:29:41.503 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:41 smithi057 ceph-mon[29805]: from='client.34628 v1:172.21.15.57:0/2052731032' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-11T13:29:41.548 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=settings t=2024-06-11T13:29:41.304940429Z level=info msg="Starting Grafana" version=9.4.12 commit=37ee25ba46 branch=HEAD compiled=2023-05-22T15:58:05Z 2024-06-11T13:29:41.548 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=settings t=2024-06-11T13:29:41.305115733Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-06-11T13:29:41.548 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=settings t=2024-06-11T13:29:41.305126797Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-06-11T13:29:41.548 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=settings t=2024-06-11T13:29:41.305130205Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-06-11T13:29:41.548 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=settings t=2024-06-11T13:29:41.305133134Z level=info msg="Config overridden from command line" arg="default.paths.data=/var/lib/grafana" 2024-06-11T13:29:41.548 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=settings t=2024-06-11T13:29:41.305135811Z level=info msg="Config overridden from command line" arg="default.paths.logs=/var/log/grafana" 2024-06-11T13:29:41.549 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=settings t=2024-06-11T13:29:41.305138556Z level=info msg="Config overridden from command line" arg="default.paths.plugins=/var/lib/grafana/plugins" 2024-06-11T13:29:41.549 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=settings t=2024-06-11T13:29:41.305141156Z level=info msg="Config overridden from command line" arg="default.paths.provisioning=/etc/grafana/provisioning" 2024-06-11T13:29:41.549 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=settings t=2024-06-11T13:29:41.305143657Z level=info msg="Config overridden from command line" arg="default.log.mode=console" 2024-06-11T13:29:41.549 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=settings t=2024-06-11T13:29:41.305146506Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-06-11T13:29:41.549 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=settings t=2024-06-11T13:29:41.305150963Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-06-11T13:29:41.549 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=settings t=2024-06-11T13:29:41.305153461Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-06-11T13:29:41.549 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=settings t=2024-06-11T13:29:41.305155925Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-06-11T13:29:41.549 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=settings t=2024-06-11T13:29:41.305158492Z level=info msg="Path Home" path=/usr/share/grafana 2024-06-11T13:29:41.549 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=settings t=2024-06-11T13:29:41.305161074Z level=info msg="Path Data" path=/var/lib/grafana 2024-06-11T13:29:41.549 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=settings t=2024-06-11T13:29:41.305163422Z level=info msg="Path Logs" path=/var/log/grafana 2024-06-11T13:29:41.549 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=settings t=2024-06-11T13:29:41.305165753Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-06-11T13:29:41.549 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=settings t=2024-06-11T13:29:41.305168233Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-06-11T13:29:41.550 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=settings t=2024-06-11T13:29:41.305170701Z level=info msg="App mode production" 2024-06-11T13:29:41.550 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=sqlstore t=2024-06-11T13:29:41.305211252Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-06-11T13:29:41.551 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=sqlstore t=2024-06-11T13:29:41.305224571Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-06-11T13:29:41.551 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.306031987Z level=info msg="Starting DB migrations" 2024-06-11T13:29:41.551 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.306091237Z level=info msg="Executing migration" id="create migration_log table" 2024-06-11T13:29:41.551 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.306588741Z level=info msg="Executing migration" id="create user table" 2024-06-11T13:29:41.551 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.307004574Z level=info msg="Executing migration" id="add unique index user.login" 2024-06-11T13:29:41.551 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.307483218Z level=info msg="Executing migration" id="add unique index user.email" 2024-06-11T13:29:41.551 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.308369295Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-06-11T13:29:41.551 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.309167222Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-06-11T13:29:41.551 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.310061081Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-06-11T13:29:41.551 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.31097015Z level=info msg="Executing migration" id="create user table v2" 2024-06-11T13:29:41.551 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.311820177Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-06-11T13:29:41.551 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.314607676Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-06-11T13:29:41.551 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.317422064Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-06-11T13:29:41.552 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.320444819Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-06-11T13:29:41.552 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.323114212Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-06-11T13:29:41.552 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.326099154Z level=info msg="Executing migration" id="Update user table charset" 2024-06-11T13:29:41.552 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.326403317Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-06-11T13:29:41.552 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.329303273Z level=info msg="Executing migration" id="Add missing user data" 2024-06-11T13:29:41.552 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.329670016Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-06-11T13:29:41.552 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.332358308Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-06-11T13:29:41.552 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.334936201Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-06-11T13:29:41.552 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.335362712Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-06-11T13:29:41.552 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.33837467Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-06-11T13:29:41.552 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.341248095Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-06-11T13:29:41.552 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.343979848Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-06-11T13:29:41.552 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.346589377Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-06-11T13:29:41.553 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.349467292Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-06-11T13:29:41.553 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.352304594Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-06-11T13:29:41.553 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.354890182Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-06-11T13:29:41.553 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.357512316Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-06-11T13:29:41.553 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.360419846Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-06-11T13:29:41.553 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.362988504Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-06-11T13:29:41.553 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.366001426Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-06-11T13:29:41.553 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.368524662Z level=info msg="Executing migration" id="create temp_user v2" 2024-06-11T13:29:41.553 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.371452078Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-06-11T13:29:41.553 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.371965618Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-06-11T13:29:41.553 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.372362472Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-06-11T13:29:41.553 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.372720905Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-06-11T13:29:41.554 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.373126889Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-06-11T13:29:41.554 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.373448215Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-06-11T13:29:41.554 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.373754968Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-06-11T13:29:41.554 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.374092113Z level=info msg="Executing migration" id="create star table" 2024-06-11T13:29:41.554 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.37445397Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-06-11T13:29:41.554 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.374876032Z level=info msg="Executing migration" id="create org table v1" 2024-06-11T13:29:41.554 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.3752623Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-06-11T13:29:41.554 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.375652022Z level=info msg="Executing migration" id="create org_user table v1" 2024-06-11T13:29:41.554 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.376016218Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-06-11T13:29:41.554 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.37640119Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-06-11T13:29:41.554 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.376773508Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-06-11T13:29:41.554 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.377152181Z level=info msg="Executing migration" id="Update org table charset" 2024-06-11T13:29:41.554 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.379910842Z level=info msg="Executing migration" id="Update org_user table charset" 2024-06-11T13:29:41.555 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.380203032Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-06-11T13:29:41.555 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.382854331Z level=info msg="Executing migration" id="create dashboard table" 2024-06-11T13:29:41.555 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.383399456Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-06-11T13:29:41.555 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.38561314Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-06-11T13:29:41.555 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.388248585Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-06-11T13:29:41.555 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.388635995Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-06-11T13:29:41.555 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.391493147Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-06-11T13:29:41.555 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.392312467Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-06-11T13:29:41.555 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.393561311Z level=info msg="Executing migration" id="create dashboard v2" 2024-06-11T13:29:41.555 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.393944226Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-06-11T13:29:41.555 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.394756445Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-06-11T13:29:41.555 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.395748899Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-06-11T13:29:41.555 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.39724326Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-06-11T13:29:41.556 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.397900371Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-06-11T13:29:41.556 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.39869952Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-06-11T13:29:41.556 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.399505706Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-06-11T13:29:41.556 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.400476738Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-06-11T13:29:41.556 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.401189332Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-06-11T13:29:41.556 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.401969752Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-06-11T13:29:41.556 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.40280661Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-06-11T13:29:41.556 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.403813719Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-06-11T13:29:41.556 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.404288377Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-06-11T13:29:41.556 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.405117172Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-06-11T13:29:41.556 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.405498106Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-06-11T13:29:41.556 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.406002961Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-06-11T13:29:41.556 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.406549971Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-06-11T13:29:41.557 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.407062546Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-06-11T13:29:41.557 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.407928899Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-06-11T13:29:41.557 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.408937788Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-06-11T13:29:41.557 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.409780959Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-06-11T13:29:41.557 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.410339977Z level=info msg="Executing migration" id="Update dashboard title length" 2024-06-11T13:29:41.557 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.411216161Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-06-11T13:29:41.557 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.411629122Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-06-11T13:29:41.557 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.412048156Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-06-11T13:29:41.557 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.413087106Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-06-11T13:29:41.557 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.413464349Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-06-11T13:29:41.557 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.414341185Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-06-11T13:29:41.557 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.415178935Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-06-11T13:29:41.557 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.416078296Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-06-11T13:29:41.557 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.41644178Z level=info msg="Executing migration" id="Add check_sum column" 2024-06-11T13:29:41.558 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.417265636Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-06-11T13:29:41.558 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.417697503Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-06-11T13:29:41.558 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.41813878Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-06-11T13:29:41.558 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.421264851Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-06-11T13:29:41.559 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.421677787Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-06-11T13:29:41.559 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.424662934Z level=info msg="Executing migration" id="create data_source table" 2024-06-11T13:29:41.559 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.427390638Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-06-11T13:29:41.559 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.427820427Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-06-11T13:29:41.559 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.430345179Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-06-11T13:29:41.559 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.4329719Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-06-11T13:29:41.559 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.435801621Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-06-11T13:29:41.559 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.43844396Z level=info msg="Executing migration" id="create data_source table v2" 2024-06-11T13:29:41.559 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.438872469Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-06-11T13:29:41.559 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.441679212Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-06-11T13:29:41.559 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.444425885Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-06-11T13:29:41.559 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.447259251Z level=info msg="Executing migration" id="Add column with_credentials" 2024-06-11T13:29:41.560 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.44788339Z level=info msg="Executing migration" id="Add secure json data column" 2024-06-11T13:29:41.560 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.450925802Z level=info msg="Executing migration" id="Update data_source table charset" 2024-06-11T13:29:41.560 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.453625325Z level=info msg="Executing migration" id="Update initial version to 1" 2024-06-11T13:29:41.560 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.456172979Z level=info msg="Executing migration" id="Add read_only data column" 2024-06-11T13:29:41.560 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.456772522Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-06-11T13:29:41.560 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.459645846Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-06-11T13:29:41.560 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.462379536Z level=info msg="Executing migration" id="Add uid column" 2024-06-11T13:29:41.560 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.465051515Z level=info msg="Executing migration" id="Update uid value" 2024-06-11T13:29:41.560 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.46542215Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-06-11T13:29:41.560 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.467806529Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-06-11T13:29:41.560 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.468245792Z level=info msg="Executing migration" id="create api_key table" 2024-06-11T13:29:41.560 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.470640323Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-06-11T13:29:41.560 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.473336284Z level=info msg="Executing migration" id="add index api_key.key" 2024-06-11T13:29:41.561 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.473873243Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-06-11T13:29:41.561 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.476676691Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-06-11T13:29:41.561 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.479074568Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-06-11T13:29:41.561 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.48177287Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-06-11T13:29:41.561 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.484601367Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-06-11T13:29:41.561 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.487079079Z level=info msg="Executing migration" id="create api_key table v2" 2024-06-11T13:29:41.561 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.489799126Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-06-11T13:29:41.561 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.492272759Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-06-11T13:29:41.561 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.495062403Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-06-11T13:29:41.561 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.497693718Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-06-11T13:29:41.561 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.498553578Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-06-11T13:29:41.561 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.499014891Z level=info msg="Executing migration" id="Update api_key table charset" 2024-06-11T13:29:41.561 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.499823612Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-06-11T13:29:41.561 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.500642569Z level=info msg="Executing migration" id="Add service account foreign key" 2024-06-11T13:29:41.562 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.50146166Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-06-11T13:29:41.562 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.502321805Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-06-11T13:29:41.562 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.503199867Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-06-11T13:29:41.562 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.503947586Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-06-11T13:29:41.562 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.504799483Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-06-11T13:29:41.562 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.505755927Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-06-11T13:29:41.562 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.506394672Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-06-11T13:29:41.562 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.507296988Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-06-11T13:29:41.562 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.507972911Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-06-11T13:29:41.562 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.508904327Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-06-11T13:29:41.562 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.509797193Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-06-11T13:29:41.562 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.510479371Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-06-11T13:29:41.562 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.51127899Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-06-11T13:29:41.562 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.512064494Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-06-11T13:29:41.563 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.512878865Z level=info msg="Executing migration" id="create quota table v1" 2024-06-11T13:29:41.563 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.51383064Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-06-11T13:29:41.563 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.514602482Z level=info msg="Executing migration" id="Update quota table charset" 2024-06-11T13:29:41.563 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.514852851Z level=info msg="Executing migration" id="create plugin_setting table" 2024-06-11T13:29:41.563 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.515494345Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-06-11T13:29:41.563 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.516348453Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-06-11T13:29:41.563 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.517236056Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-06-11T13:29:41.563 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.518027459Z level=info msg="Executing migration" id="create session table" 2024-06-11T13:29:41.563 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.518468003Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-06-11T13:29:41.563 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.518715598Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-06-11T13:29:41.563 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.519241902Z level=info msg="Executing migration" id="create playlist table v2" 2024-06-11T13:29:41.563 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.519653251Z level=info msg="Executing migration" id="create playlist item table v2" 2024-06-11T13:29:41.563 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.520608136Z level=info msg="Executing migration" id="Update playlist table charset" 2024-06-11T13:29:41.564 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.52099429Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-06-11T13:29:41.564 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.521285895Z level=info msg="Executing migration" id="drop preferences table v2" 2024-06-11T13:29:41.564 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.521532506Z level=info msg="Executing migration" id="drop preferences table v3" 2024-06-11T13:29:41.564 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.521779229Z level=info msg="Executing migration" id="create preferences table v3" 2024-06-11T13:29:41.564 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.522208406Z level=info msg="Executing migration" id="Update preferences table charset" 2024-06-11T13:29:41.564 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.523013656Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-06-11T13:29:41.564 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.523716365Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-06-11T13:29:41.564 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.524501074Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-06-11T13:29:41.564 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.525210587Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-06-11T13:29:41.564 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.526052756Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-06-11T13:29:41.564 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.528097107Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-06-11T13:29:41.564 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.530793961Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-06-11T13:29:41.564 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.533593019Z level=info msg="Executing migration" id="create alert table v1" 2024-06-11T13:29:41.564 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.536307924Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-06-11T13:29:41.565 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.539060534Z level=info msg="Executing migration" id="add index alert state" 2024-06-11T13:29:41.565 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.54172456Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-06-11T13:29:41.565 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.544440123Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-06-11T13:29:41.565 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.547157766Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-06-11T13:29:41.565 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.549760577Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-06-11T13:29:41.565 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.5524809Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-06-11T13:29:41.565 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.555148807Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-06-11T13:29:41.799 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.557808775Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-06-11T13:29:41.799 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.560576901Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-06-11T13:29:41.799 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.563347874Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-06-11T13:29:41.799 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.565917975Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-06-11T13:29:41.799 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.566711488Z level=info msg="Executing migration" id="Add column is_default" 2024-06-11T13:29:41.799 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.567541876Z level=info msg="Executing migration" id="Add column frequency" 2024-06-11T13:29:41.799 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.568427892Z level=info msg="Executing migration" id="Add column send_reminder" 2024-06-11T13:29:41.799 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.569281592Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-06-11T13:29:41.799 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.57006352Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-06-11T13:29:41.799 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.570845112Z level=info msg="Executing migration" id="Update alert table charset" 2024-06-11T13:29:41.800 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.571719379Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-06-11T13:29:41.800 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.572580016Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-06-11T13:29:41.800 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.573507046Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-06-11T13:29:41.800 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.574139399Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-06-11T13:29:41.800 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.574999177Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-06-11T13:29:41.800 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.575972873Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-06-11T13:29:41.800 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.5766624Z level=info msg="Executing migration" id="Add for to alert table" 2024-06-11T13:29:41.800 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.57748288Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-06-11T13:29:41.800 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.578267081Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-06-11T13:29:41.800 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.579227261Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-06-11T13:29:41.800 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.580088368Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-06-11T13:29:41.800 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.580814211Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-06-11T13:29:41.801 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.58169404Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-06-11T13:29:41.801 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.582545837Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-06-11T13:29:41.801 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.582965131Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-06-11T13:29:41.801 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.583432438Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-06-11T13:29:41.801 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.584351195Z level=info msg="Executing migration" id="create annotation table v5" 2024-06-11T13:29:41.801 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.585025351Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-06-11T13:29:41.801 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.585417209Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-06-11T13:29:41.801 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.586288537Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-06-11T13:29:41.801 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.587157956Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-06-11T13:29:41.801 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.587952785Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-06-11T13:29:41.801 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.58890255Z level=info msg="Executing migration" id="Update annotation table charset" 2024-06-11T13:29:41.801 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.589680625Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-06-11T13:29:41.802 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.590492297Z level=info msg="Executing migration" id="Drop category_id index" 2024-06-11T13:29:41.802 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.591356245Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-06-11T13:29:41.802 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.59223867Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-06-11T13:29:41.802 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.593278151Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-06-11T13:29:41.803 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.594004475Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-06-11T13:29:41.803 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.596736517Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-06-11T13:29:41.803 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.599626847Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-06-11T13:29:41.803 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.602646065Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-06-11T13:29:41.803 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.605477152Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-06-11T13:29:41.804 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.608392012Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-06-11T13:29:41.804 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.611002208Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-06-11T13:29:41.804 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.614002128Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-06-11T13:29:41.804 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.617287278Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-06-11T13:29:41.804 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.619575553Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-06-11T13:29:41.804 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.622274802Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-06-11T13:29:41.804 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.625102583Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-06-11T13:29:41.804 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.6279553Z level=info msg="Executing migration" id="Add epoch_end column" 2024-06-11T13:29:41.804 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.628924485Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-06-11T13:29:41.804 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.631694638Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-06-11T13:29:41.804 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.634566178Z level=info msg="Executing migration" id="Move region to single row" 2024-06-11T13:29:41.805 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.637429144Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-06-11T13:29:41.805 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.637908391Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-06-11T13:29:41.805 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.640896524Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-06-11T13:29:41.805 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.643448265Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-06-11T13:29:41.805 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.646347276Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-06-11T13:29:41.805 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.648999723Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-06-11T13:29:41.805 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.651667776Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-06-11T13:29:41.805 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.65404197Z level=info msg="Executing migration" id="create test_data table" 2024-06-11T13:29:41.805 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.65677047Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-06-11T13:29:41.805 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.657319742Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-06-11T13:29:41.805 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.660008576Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-06-11T13:29:41.805 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.662750237Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-06-11T13:29:41.806 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.665422298Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-06-11T13:29:41.806 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.665782911Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-06-11T13:29:41.806 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.668713012Z level=info msg="Executing migration" id="create team table" 2024-06-11T13:29:41.806 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.671241744Z level=info msg="Executing migration" id="add index team.org_id" 2024-06-11T13:29:41.806 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.674038312Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-06-11T13:29:41.806 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.674645801Z level=info msg="Executing migration" id="create team member table" 2024-06-11T13:29:41.806 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.677446302Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-06-11T13:29:41.806 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.680247956Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-06-11T13:29:41.806 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.680941086Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-06-11T13:29:41.806 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.681387547Z level=info msg="Executing migration" id="Add column email to team table" 2024-06-11T13:29:41.806 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.682264856Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-06-11T13:29:41.807 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.683106645Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-06-11T13:29:41.807 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.683960658Z level=info msg="Executing migration" id="create dashboard acl table" 2024-06-11T13:29:41.807 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.684385745Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-06-11T13:29:41.807 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.685309984Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-06-11T13:29:41.807 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.68606781Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-06-11T13:29:41.807 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.68687767Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-06-11T13:29:41.807 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.687350601Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-06-11T13:29:41.807 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.688167311Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-06-11T13:29:41.807 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.689108988Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-06-11T13:29:41.807 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.689777414Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-06-11T13:29:41.807 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.690728293Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-06-11T13:29:41.807 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.691375694Z level=info msg="Executing migration" id="create tag table" 2024-06-11T13:29:41.808 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.692418665Z level=info msg="Executing migration" id="add index tag.key_value" 2024-06-11T13:29:41.808 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.693103075Z level=info msg="Executing migration" id="create login attempt table" 2024-06-11T13:29:41.808 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.693951009Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-06-11T13:29:41.808 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.694880761Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-06-11T13:29:41.808 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.695688529Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-06-11T13:29:41.808 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.69801193Z level=info msg="Executing migration" id="create login_attempt v2" 2024-06-11T13:29:41.808 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.698925324Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-06-11T13:29:41.808 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.699685973Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-06-11T13:29:41.808 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.700547634Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-06-11T13:29:41.808 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.701029683Z level=info msg="Executing migration" id="create user auth table" 2024-06-11T13:29:41.808 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.701423457Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-06-11T13:29:41.808 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.702277723Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-06-11T13:29:41.809 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.703229121Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-06-11T13:29:41.809 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.70426572Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-06-11T13:29:41.809 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.705126275Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-06-11T13:29:41.809 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.70606412Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-06-11T13:29:41.809 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.706970894Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-06-11T13:29:41.809 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.707703908Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-06-11T13:29:41.809 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.708590654Z level=info msg="Executing migration" id="create server_lock table" 2024-06-11T13:29:41.809 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.709571972Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-06-11T13:29:41.809 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.712255042Z level=info msg="Executing migration" id="create user auth token table" 2024-06-11T13:29:41.809 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.715069286Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-06-11T13:29:41.809 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.718148064Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-06-11T13:29:41.809 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.720800928Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-06-11T13:29:41.810 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.723505291Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-06-11T13:29:41.810 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.726242324Z level=info msg="Executing migration" id="create cache_data table" 2024-06-11T13:29:41.810 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.729201503Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-06-11T13:29:41.810 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.731810341Z level=info msg="Executing migration" id="create short_url table v1" 2024-06-11T13:29:41.810 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.734710775Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-06-11T13:29:41.810 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.737569765Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-06-11T13:29:41.810 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.74022236Z level=info msg="Executing migration" id="delete alert_definition table" 2024-06-11T13:29:41.810 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.740551213Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-06-11T13:29:41.810 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.740997841Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-06-11T13:29:41.810 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.74373971Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-06-11T13:29:41.810 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.744189661Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-06-11T13:29:41.811 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.744502964Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-06-11T13:29:41.811 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.744906261Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-06-11T13:29:41.811 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.745302085Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-06-11T13:29:41.811 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.745684767Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-06-11T13:29:41.811 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.746102975Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-06-11T13:29:41.811 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.746962782Z level=info msg="Executing migration" id="drop alert_definition table" 2024-06-11T13:29:41.811 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.747368802Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-06-11T13:29:41.811 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.747623864Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-06-11T13:29:41.811 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.748059968Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-06-11T13:29:41.811 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.748470677Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-06-11T13:29:41.811 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.748918278Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-06-11T13:29:41.811 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.74921564Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-06-11T13:29:41.812 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.749634593Z level=info msg="Executing migration" id="create alert_instance table" 2024-06-11T13:29:41.812 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.750035213Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-06-11T13:29:41.812 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.750434302Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-06-11T13:29:41.812 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.750836222Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-06-11T13:29:41.812 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.751717431Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-06-11T13:29:41.812 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.752117507Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-06-11T13:29:41.812 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.752507742Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-06-11T13:29:41.812 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.756953133Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-06-11T13:29:41.812 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.761292029Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-06-11T13:29:41.812 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.76170877Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-06-11T13:29:41.812 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.762118803Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-06-11T13:29:41.812 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.763069977Z level=info msg="Executing migration" id="create alert_rule table" 2024-06-11T13:29:41.813 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.763469692Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-06-11T13:29:41.813 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.763884747Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-06-11T13:29:41.813 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.764294061Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-06-11T13:29:41.813 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.764697409Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-06-11T13:29:41.813 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.764944806Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-06-11T13:29:41.813 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.765856442Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-06-11T13:29:41.813 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.766754747Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-06-11T13:29:41.813 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.76765168Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-06-11T13:29:41.813 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.768068698Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-06-11T13:29:41.813 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.768483072Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-06-11T13:29:41.813 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.769367797Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-06-11T13:29:41.813 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.770278213Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-06-11T13:29:41.814 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.770699893Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-06-11T13:29:41.814 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.771622775Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-06-11T13:29:41.814 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.772545512Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-06-11T13:29:41.814 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.772844091Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-06-11T13:29:41.814 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.773279232Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-06-11T13:29:41.814 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.773682869Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-06-11T13:29:41.814 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.774098263Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-06-11T13:29:41.814 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.776211925Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-06-11T13:29:41.814 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.778919671Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-06-11T13:29:41.814 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.781582582Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-06-11T13:29:41.814 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.784237804Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-06-11T13:29:41.815 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.786861904Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-06-11T13:29:41.815 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.78973144Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-06-11T13:29:41.815 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.792350231Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-06-11T13:29:41.815 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.795007824Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-06-11T13:29:41.815 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.797611718Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-06-11T13:29:41.815 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.800226948Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-06-11T13:29:41.815 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.802784136Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-06-11T13:29:41.815 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.805445417Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-06-11T13:29:42.054 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.808011578Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-06-11T13:29:42.054 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.810687013Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-06-11T13:29:42.054 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.813557041Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-06-11T13:29:42.054 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.81612465Z level=info msg="Executing migration" id="create provenance_type table" 2024-06-11T13:29:42.054 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.818944688Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-06-11T13:29:42.054 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.821268875Z level=info msg="Executing migration" id="create alert_image table" 2024-06-11T13:29:42.054 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.822088811Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-06-11T13:29:42.054 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.822879331Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-06-11T13:29:42.055 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.823681074Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-06-11T13:29:42.055 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.824462594Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-06-11T13:29:42.055 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.8248946Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-06-11T13:29:42.055 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.824967685Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-06-11T13:29:42.055 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.825819664Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-06-11T13:29:42.055 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.826577284Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-06-11T13:29:42.055 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.827426467Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-06-11T13:29:42.055 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.827612495Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-06-11T13:29:42.055 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.827707558Z level=info msg="alerts found to migrate" alerts=0 2024-06-11T13:29:42.055 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.828223366Z level=info msg="Executing migration" id="create library_element table v1" 2024-06-11T13:29:42.056 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.829168058Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-06-11T13:29:42.056 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.830070848Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-06-11T13:29:42.056 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.830692869Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-06-11T13:29:42.056 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.831489768Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-06-11T13:29:42.056 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.832307296Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-06-11T13:29:42.056 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.833267263Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-06-11T13:29:42.056 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.833966126Z level=info msg="Executing migration" id="create data_keys table" 2024-06-11T13:29:42.057 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.834768619Z level=info msg="Executing migration" id="create secrets table" 2024-06-11T13:29:42.057 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.835680454Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-06-11T13:29:42.057 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.841418848Z level=info msg="Executing migration" id="add name column into data_keys" 2024-06-11T13:29:42.057 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.842505594Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-06-11T13:29:42.057 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.843459274Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-06-11T13:29:42.057 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.849124917Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-06-11T13:29:42.057 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.854763635Z level=info msg="Executing migration" id="create kv_store table v1" 2024-06-11T13:29:42.057 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.855730578Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-06-11T13:29:42.057 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.856574516Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-06-11T13:29:42.057 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.857305876Z level=info msg="Executing migration" id="create permission table" 2024-06-11T13:29:42.057 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.8583046Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-06-11T13:29:42.058 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.859239539Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-06-11T13:29:42.058 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.859851583Z level=info msg="Executing migration" id="create role table" 2024-06-11T13:29:42.058 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.860710559Z level=info msg="Executing migration" id="add column display_name" 2024-06-11T13:29:42.058 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.861807827Z level=info msg="Executing migration" id="add column group_name" 2024-06-11T13:29:42.058 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.86286198Z level=info msg="Executing migration" id="add index role.org_id" 2024-06-11T13:29:42.058 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.863797165Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-06-11T13:29:42.058 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.864765676Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-06-11T13:29:42.058 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.865504943Z level=info msg="Executing migration" id="create team role table" 2024-06-11T13:29:42.058 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.866340209Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-06-11T13:29:42.058 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.869407072Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-06-11T13:29:42.058 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.872214863Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-06-11T13:29:42.058 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.87506026Z level=info msg="Executing migration" id="create user role table" 2024-06-11T13:29:42.059 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.877765782Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-06-11T13:29:42.059 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.880737455Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-06-11T13:29:42.059 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.883383644Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-06-11T13:29:42.059 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.88622491Z level=info msg="Executing migration" id="create builtin role table" 2024-06-11T13:29:42.059 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.888897186Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-06-11T13:29:42.059 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.891694753Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-06-11T13:29:42.059 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.894554816Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-06-11T13:29:42.059 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.897106731Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-06-11T13:29:42.059 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.897611951Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-06-11T13:29:42.059 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.898064293Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-06-11T13:29:42.059 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.898550846Z level=info msg="Executing migration" id="add unique index role.uid" 2024-06-11T13:29:42.060 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.898975684Z level=info msg="Executing migration" id="create seed assignment table" 2024-06-11T13:29:42.060 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.899385504Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-06-11T13:29:42.060 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.899853543Z level=info msg="Executing migration" id="add column hidden to role table" 2024-06-11T13:29:42.060 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.900967269Z level=info msg="Executing migration" id="create query_history table v1" 2024-06-11T13:29:42.060 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.903695955Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-06-11T13:29:42.060 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.904429841Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-06-11T13:29:42.060 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.904744954Z level=info msg="Executing migration" id="teams permissions migration" 2024-06-11T13:29:42.060 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.905167197Z level=info msg="Executing migration" id="dashboard permissions" 2024-06-11T13:29:42.060 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.90564028Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-06-11T13:29:42.060 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.906026385Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-06-11T13:29:42.060 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.906353841Z level=info msg="Executing migration" id="alerting notification permissions" 2024-06-11T13:29:42.060 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.906693397Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-06-11T13:29:42.061 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.90709679Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-06-11T13:29:42.061 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.907572541Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-06-11T13:29:42.061 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.90871681Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-06-11T13:29:42.061 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.909032261Z level=info msg="Executing migration" id="create correlation table v1" 2024-06-11T13:29:42.061 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.909477766Z level=info msg="Executing migration" id="add index correlations.uid" 2024-06-11T13:29:42.062 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.90995159Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-06-11T13:29:42.062 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.910441622Z level=info msg="Executing migration" id="add correlation config column" 2024-06-11T13:29:42.062 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.911590362Z level=info msg="Executing migration" id="create entity_events table" 2024-06-11T13:29:42.062 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.913032795Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-06-11T13:29:42.062 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.915724026Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-06-11T13:29:42.062 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.915902523Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-06-11T13:29:42.062 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.918589572Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-06-11T13:29:42.063 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.918687201Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-06-11T13:29:42.063 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.921379068Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-06-11T13:29:42.063 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.923979169Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-06-11T13:29:42.063 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.927049121Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-06-11T13:29:42.063 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.92982131Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-06-11T13:29:42.063 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.932520051Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-06-11T13:29:42.063 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.935225588Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-06-11T13:29:42.063 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.938167058Z level=info msg="Executing migration" id="Drop public config table" 2024-06-11T13:29:42.063 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.940981373Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-06-11T13:29:42.063 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.943527561Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-06-11T13:29:42.063 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.946205188Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-06-11T13:29:42.063 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.948840823Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-06-11T13:29:42.064 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.951693582Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-06-11T13:29:42.064 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.955486076Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-06-11T13:29:42.064 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.958188356Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-06-11T13:29:42.064 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.961061106Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-06-11T13:29:42.064 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.963798928Z level=info msg="Executing migration" id="add share column" 2024-06-11T13:29:42.064 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.966553967Z level=info msg="Executing migration" id="create default alerting folders" 2024-06-11T13:29:42.064 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.969241621Z level=info msg="Executing migration" id="create file table" 2024-06-11T13:29:42.064 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.972077642Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-06-11T13:29:42.064 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.974753057Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-06-11T13:29:42.064 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.977241646Z level=info msg="Executing migration" id="create file_meta table" 2024-06-11T13:29:42.064 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.980119472Z level=info msg="Executing migration" id="file table idx: path key" 2024-06-11T13:29:42.065 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.982677316Z level=info msg="Executing migration" id="set path collation in file table" 2024-06-11T13:29:42.065 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.985285557Z level=info msg="Executing migration" id="managed permissions migration" 2024-06-11T13:29:42.065 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.988099543Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-06-11T13:29:42.065 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.990918038Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-06-11T13:29:42.065 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.992170724Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-06-11T13:29:42.065 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.993671009Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-06-11T13:29:42.065 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.994693514Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-06-11T13:29:42.065 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.995366817Z level=info msg="Executing migration" id="update group index for alert rules" 2024-06-11T13:29:42.065 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.99635463Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-06-11T13:29:42.065 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.997000871Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-06-11T13:29:42.065 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.997858847Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-06-11T13:29:42.065 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:41.999031502Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-06-11T13:29:42.066 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:42.00021295Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-06-11T13:29:42.066 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:42.001150631Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-06-11T13:29:42.066 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:42.014146167Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-06-11T13:29:42.066 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:42.015036891Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-06-11T13:29:42.066 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:42.015836447Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-06-11T13:29:42.066 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:42.020109076Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-06-11T13:29:42.066 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:42.021015995Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-06-11T13:29:42.066 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:42.021779856Z level=info msg="Executing migration" id="create folder table" 2024-06-11T13:29:42.066 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:42.022594305Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-06-11T13:29:42.066 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:42.023451597Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-06-11T13:29:42.066 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:42.024265397Z level=info msg="Executing migration" id="Update folder title length" 2024-06-11T13:29:42.066 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:42.02512048Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-06-11T13:29:42.067 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=migrator t=2024-06-11T13:29:42.026089843Z level=info msg="migrations completed" performed=485 skipped=0 duration=720.013993ms 2024-06-11T13:29:42.067 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=sqlstore t=2024-06-11T13:29:42.026573308Z level=info msg="Created default organization" 2024-06-11T13:29:42.067 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=plugin.finder t=2024-06-11T13:29:42.05765248Z level=warn msg="Skipping finding plugins as directory does not exist" path=/usr/share/grafana/plugins-bundled 2024-06-11T13:29:42.067 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=secrets t=2024-06-11T13:29:42.057733736Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-06-11T13:29:42.067 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=query_data t=2024-06-11T13:29:42.061183549Z level=info msg="Query Service initialization" 2024-06-11T13:29:42.308 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=live.push_http t=2024-06-11T13:29:42.065042941Z level=info msg="Live Push Gateway initialization" 2024-06-11T13:29:42.309 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:42 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:29:42.309 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:42 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:42.309 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:42 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:42.309 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:42 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:42.309 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:42 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:42.309 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:42 smithi107 ceph-mon[33178]: Deploying daemon node-exporter.a on smithi057 2024-06-11T13:29:42.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:42 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:29:42.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:42 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:42.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:42 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:42.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:42 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:42.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:42 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:42.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:42 smithi057 ceph-mon[29805]: Deploying daemon node-exporter.a on smithi057 2024-06-11T13:29:42.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:42 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:29:42.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:42 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:42.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:42 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:42.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:42 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:42.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:42 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:42.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:42 smithi057 ceph-mon[40588]: Deploying daemon node-exporter.a on smithi057 2024-06-11T13:29:42.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:42.867+0000 7fe0e6fb2640 1 -- 172.21.15.57:0/630099135 >> v1:172.21.15.57:6789/0 conn(0x7fe0e01095a0 legacy=0x7fe0e010b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:42.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:42.867+0000 7fe0e6fb2640 1 -- 172.21.15.57:0/630099135 shutdown_connections 2024-06-11T13:29:42.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:42.867+0000 7fe0e6fb2640 1 -- 172.21.15.57:0/630099135 >> 172.21.15.57:0/630099135 conn(0x7fe0e00fc220 msgr2=0x7fe0e00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:29:42.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:42.867+0000 7fe0e6fb2640 1 -- 172.21.15.57:0/630099135 shutdown_connections 2024-06-11T13:29:42.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:42.867+0000 7fe0e6fb2640 1 -- 172.21.15.57:0/630099135 wait complete. 2024-06-11T13:29:42.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:42.868+0000 7fe0e6fb2640 1 Processor -- start 2024-06-11T13:29:42.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:42.868+0000 7fe0e6fb2640 1 -- start start 2024-06-11T13:29:42.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:42.868+0000 7fe0e6fb2640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fe0e01a8b10 con 0x7fe0e01095a0 2024-06-11T13:29:42.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:42.868+0000 7fe0e6fb2640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fe0e01a9d10 con 0x7fe0e01066f0 2024-06-11T13:29:42.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:42.868+0000 7fe0e6fb2640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fe0e01aaf50 con 0x7fe0e010d140 2024-06-11T13:29:42.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:42.868+0000 7fe0e4d4e640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7fe0e01066f0 0x7fe0e010c9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.57:48562/0 (socket says 172.21.15.57:48562) 2024-06-11T13:29:42.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:42.868+0000 7fe0e4d4e640 1 -- 172.21.15.57:0/2348525814 learned_addr learned my addr 172.21.15.57:0/2348525814 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:29:42.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:42.869+0000 7fe0ddffb640 1 -- 172.21.15.57:0/2348525814 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1902981342 0 0) 0x7fe0e01a8b10 con 0x7fe0e01095a0 2024-06-11T13:29:42.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:42.869+0000 7fe0ddffb640 1 -- 172.21.15.57:0/2348525814 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe0cc003520 con 0x7fe0e01095a0 2024-06-11T13:29:42.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:42.869+0000 7fe0ddffb640 1 -- 172.21.15.57:0/2348525814 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2388094724 0 0) 0x7fe0e01a9d10 con 0x7fe0e01066f0 2024-06-11T13:29:42.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:42.869+0000 7fe0ddffb640 1 -- 172.21.15.57:0/2348525814 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe0e01a8b10 con 0x7fe0e01066f0 2024-06-11T13:29:42.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:42.869+0000 7fe0ddffb640 1 -- 172.21.15.57:0/2348525814 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3955597348 0 0) 0x7fe0e01aaf50 con 0x7fe0e010d140 2024-06-11T13:29:42.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:42.869+0000 7fe0ddffb640 1 -- 172.21.15.57:0/2348525814 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe0e01a9d10 con 0x7fe0e010d140 2024-06-11T13:29:42.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:42.869+0000 7fe0ddffb640 1 -- 172.21.15.57:0/2348525814 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2037156111 0 0) 0x7fe0cc003520 con 0x7fe0e01095a0 2024-06-11T13:29:42.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:42.869+0000 7fe0ddffb640 1 -- 172.21.15.57:0/2348525814 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe0e01aaf50 con 0x7fe0e01095a0 2024-06-11T13:29:42.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:42.869+0000 7fe0ddffb640 1 -- 172.21.15.57:0/2348525814 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1001682241 0 0) 0x7fe0e01a8b10 con 0x7fe0e01066f0 2024-06-11T13:29:42.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:42.869+0000 7fe0ddffb640 1 -- 172.21.15.57:0/2348525814 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe0cc003520 con 0x7fe0e01066f0 2024-06-11T13:29:42.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:42.869+0000 7fe0ddffb640 1 -- 172.21.15.57:0/2348525814 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fe0c8001840 con 0x7fe0e01095a0 2024-06-11T13:29:42.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:42.869+0000 7fe0ddffb640 1 -- 172.21.15.57:0/2348525814 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3116554921 0 0) 0x7fe0e01a9d10 con 0x7fe0e010d140 2024-06-11T13:29:42.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:42.869+0000 7fe0ddffb640 1 -- 172.21.15.57:0/2348525814 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe0e01a8b10 con 0x7fe0e010d140 2024-06-11T13:29:42.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:42.869+0000 7fe0ddffb640 1 -- 172.21.15.57:0/2348525814 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fe0d8001940 con 0x7fe0e01066f0 2024-06-11T13:29:42.881 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:42.869+0000 7fe0ddffb640 1 -- 172.21.15.57:0/2348525814 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fe0d0001bb0 con 0x7fe0e010d140 2024-06-11T13:29:42.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:42.869+0000 7fe0ddffb640 1 -- 172.21.15.57:0/2348525814 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3037140167 0 0) 0x7fe0e01aaf50 con 0x7fe0e01095a0 2024-06-11T13:29:42.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:42.869+0000 7fe0ddffb640 1 -- 172.21.15.57:0/2348525814 >> v1:172.21.15.57:6790/0 conn(0x7fe0e010d140 legacy=0x7fe0e01a71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:42.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:42.869+0000 7fe0ddffb640 1 -- 172.21.15.57:0/2348525814 >> v1:172.21.15.107:6789/0 conn(0x7fe0e01066f0 legacy=0x7fe0e010c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:42.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:42.869+0000 7fe0ddffb640 1 -- 172.21.15.57:0/2348525814 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0e01ac190 con 0x7fe0e01095a0 2024-06-11T13:29:42.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:42.869+0000 7fe0e6fb2640 1 -- 172.21.15.57:0/2348525814 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fe0e01a9f40 con 0x7fe0e01095a0 2024-06-11T13:29:42.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:42.870+0000 7fe0ddffb640 1 -- 172.21.15.57:0/2348525814 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7fe0c8001ac0 con 0x7fe0e01095a0 2024-06-11T13:29:42.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:42.870+0000 7fe0ddffb640 1 -- 172.21.15.57:0/2348525814 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fe0c8004d30 con 0x7fe0e01095a0 2024-06-11T13:29:42.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:42.870+0000 7fe0e6fb2640 1 -- 172.21.15.57:0/2348525814 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fe0e01aa400 con 0x7fe0e01095a0 2024-06-11T13:29:42.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:42.871+0000 7fe0e6fb2640 1 -- 172.21.15.57:0/2348525814 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0e0101e30 con 0x7fe0e01095a0 2024-06-11T13:29:42.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:42.871+0000 7fe0ddffb640 1 -- 172.21.15.57:0/2348525814 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 22) ==== 92138+0+0 (unknown 311317106 0 0) 0x7fe0c801b820 con 0x7fe0e01095a0 2024-06-11T13:29:42.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:42.872+0000 7fe0ddffb640 1 -- 172.21.15.57:0/2348525814 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(70..70 src has 1..70) ==== 6172+0+0 (unknown 3813449920 0 0) 0x7fe0c8088460 con 0x7fe0e01095a0 2024-06-11T13:29:42.882 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:42.876+0000 7fe0ddffb640 1 -- 172.21.15.57:0/2348525814 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7fe0c8055c20 con 0x7fe0e01095a0 2024-06-11T13:29:43.047 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=infra.usagestats.collector t=2024-06-11T13:29:42.689506815Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-06-11T13:29:43.047 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=provisioning.datasources t=2024-06-11T13:29:42.690940671Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-06-11T13:29:43.047 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=provisioning.datasources t=2024-06-11T13:29:42.702094388Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-06-11T13:29:43.047 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=provisioning.alerting t=2024-06-11T13:29:42.712731378Z level=info msg="starting to provision alerting" 2024-06-11T13:29:43.047 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=provisioning.alerting t=2024-06-11T13:29:42.712744612Z level=info msg="finished to provision alerting" 2024-06-11T13:29:43.047 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=ngalert.state.manager t=2024-06-11T13:29:42.713051804Z level=info msg="Warming state cache for startup" 2024-06-11T13:29:43.047 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=grafanaStorageLogger t=2024-06-11T13:29:42.713391701Z level=info msg="storage starting" 2024-06-11T13:29:43.048 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=http.server t=2024-06-11T13:29:42.715387249Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-06-11T13:29:43.048 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=ngalert.state.manager t=2024-06-11T13:29:42.717913259Z level=info msg="State cache has been initialized" states=0 duration=4.861403ms 2024-06-11T13:29:43.048 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=ngalert.multiorg.alertmanager t=2024-06-11T13:29:42.717975891Z level=info msg="starting MultiOrg Alertmanager" 2024-06-11T13:29:43.048 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=ticker t=2024-06-11T13:29:42.71800616Z level=info msg=starting first_tick=2024-06-11T13:29:50Z 2024-06-11T13:29:43.048 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=sqlstore.transactions t=2024-06-11T13:29:42.723925532Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-06-11T13:29:43.048 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=sqlstore.transactions t=2024-06-11T13:29:42.729052018Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-06-11T13:29:43.048 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=sqlstore.transactions t=2024-06-11T13:29:42.745548952Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=1 code="database is locked" 2024-06-11T13:29:43.048 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=sqlstore.transactions t=2024-06-11T13:29:42.755876536Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=2 code="database is locked" 2024-06-11T13:29:43.048 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:29:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=sqlstore.transactions t=2024-06-11T13:29:42.767781044Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=3 code="database is locked" 2024-06-11T13:29:43.073 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:43.075+0000 7fe0e6fb2640 1 -- 172.21.15.57:0/2348525814 --> v1:172.21.15.57:6800/2077416415 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe0e0104710 con 0x7fe0cc06ef70 2024-06-11T13:29:43.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:43.085+0000 7fe0ddffb640 1 -- 172.21.15.57:0/2348525814 <== mgr.14604 v1:172.21.15.57:6800/2077416415 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+346459 (unknown 2965378022 0 2255884469) 0x7fe0e0104710 con 0x7fe0cc06ef70 2024-06-11T13:29:43.086 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:29:43.088 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2024-06-11T13:29:43.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:43.088+0000 7fe0e6fb2640 1 -- 172.21.15.57:0/2348525814 >> v1:172.21.15.57:6800/2077416415 conn(0x7fe0cc06ef70 legacy=0x7fe0cc071430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:43.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:43.089+0000 7fe0e6fb2640 1 -- 172.21.15.57:0/2348525814 >> v1:172.21.15.57:6789/0 conn(0x7fe0e01095a0 legacy=0x7fe0e01a39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:43.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:43.089+0000 7fe0e6fb2640 1 -- 172.21.15.57:0/2348525814 shutdown_connections 2024-06-11T13:29:43.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:43.089+0000 7fe0e6fb2640 1 -- 172.21.15.57:0/2348525814 >> 172.21.15.57:0/2348525814 conn(0x7fe0e00fc220 msgr2=0x7fe0e00fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:29:43.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:43.089+0000 7fe0e6fb2640 1 -- 172.21.15.57:0/2348525814 shutdown_connections 2024-06-11T13:29:43.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:43.089+0000 7fe0e6fb2640 1 -- 172.21.15.57:0/2348525814 wait complete. 2024-06-11T13:29:43.398 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:43 smithi057 ceph-mon[29805]: pgmap v29: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:29:43.399 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:43 smithi057 ceph-mon[40588]: pgmap v29: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:29:43.731 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:43 smithi057 bash[75408]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-06-11T13:29:43.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:43 smithi107 ceph-mon[33178]: pgmap v29: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:29:43.981 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2024-06-11T13:29:41.966085+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":198,"num_object_clones":0,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":198,"num_whiteouts":0,"num_read":1016,"num_read_kb":862,"num_write":615,"num_write_kb":2461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":600,"ondisk_log_size":600,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":248276,"kb_used_data":7164,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":749712940,"statfs":{"total":767960285184,"available":767706050560,"internally_reserved":0,"allocated":7335936,"data_stored":3919001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002706"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.031788+0000","last_change":"2024-06-11T13:28:19.550716+0000","last_active":"2024-06-11T13:28:50.031788+0000","last_peered":"2024-06-11T13:28:50.031788+0000","last_clean":"2024-06-11T13:28:50.031788+0000","last_became_active":"2024-06-11T13:28:19.550286+0000","last_became_peered":"2024-06-11T13:28:19.550286+0000","last_unstale":"2024-06-11T13:28:50.031788+0000","last_undegraded":"2024-06-11T13:28:50.031788+0000","last_fullsized":"2024-06-11T13:28:50.031788+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T21:15:28.536619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.943500+0000","last_change":"2024-06-11T13:28:13.182552+0000","last_active":"2024-06-11T13:28:49.943500+0000","last_peered":"2024-06-11T13:28:49.943500+0000","last_clean":"2024-06-11T13:28:49.943500+0000","last_became_active":"2024-06-11T13:28:13.181741+0000","last_became_peered":"2024-06-11T13:28:13.181741+0000","last_unstale":"2024-06-11T13:28:49.943500+0000","last_undegraded":"2024-06-11T13:28:49.943500+0000","last_fullsized":"2024-06-11T13:28:49.943500+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T18:31:32.801830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"68'15","reported_seq":50,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.031667+0000","last_change":"2024-06-11T13:28:15.428556+0000","last_active":"2024-06-11T13:28:50.031667+0000","last_peered":"2024-06-11T13:28:50.031667+0000","last_clean":"2024-06-11T13:28:50.031667+0000","last_became_active":"2024-06-11T13:28:15.428175+0000","last_became_peered":"2024-06-11T13:28:15.428175+0000","last_unstale":"2024-06-11T13:28:50.031667+0000","last_undegraded":"2024-06-11T13:28:50.031667+0000","last_fullsized":"2024-06-11T13:28:50.031667+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T18:09:37.101758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.454339+0000","last_change":"2024-06-11T13:28:17.455777+0000","last_active":"2024-06-11T13:28:51.454339+0000","last_peered":"2024-06-11T13:28:51.454339+0000","last_clean":"2024-06-11T13:28:51.454339+0000","last_became_active":"2024-06-11T13:28:17.454883+0000","last_became_peered":"2024-06-11T13:28:17.454883+0000","last_unstale":"2024-06-11T13:28:51.454339+0000","last_undegraded":"2024-06-11T13:28:51.454339+0000","last_fullsized":"2024-06-11T13:28:51.454339+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T15:26:14.098490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.033292+0000","last_change":"2024-06-11T13:28:13.185465+0000","last_active":"2024-06-11T13:28:50.033292+0000","last_peered":"2024-06-11T13:28:50.033292+0000","last_clean":"2024-06-11T13:28:50.033292+0000","last_became_active":"2024-06-11T13:28:13.184658+0000","last_became_peered":"2024-06-11T13:28:13.184658+0000","last_unstale":"2024-06-11T13:28:50.033292+0000","last_undegraded":"2024-06-11T13:28:50.033292+0000","last_fullsized":"2024-06-11T13:28:50.033292+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T14:30:05.952881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"68'9","reported_seq":41,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.454358+0000","last_change":"2024-06-11T13:28:15.420834+0000","last_active":"2024-06-11T13:28:51.454358+0000","last_peered":"2024-06-11T13:28:51.454358+0000","last_clean":"2024-06-11T13:28:51.454358+0000","last_became_active":"2024-06-11T13:28:15.419923+0000","last_became_peered":"2024-06-11T13:28:15.419923+0000","last_unstale":"2024-06-11T13:28:51.454358+0000","last_undegraded":"2024-06-11T13:28:51.454358+0000","last_fullsized":"2024-06-11T13:28:51.454358+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-13T00:22:41.587194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.039245+0000","last_change":"2024-06-11T13:28:17.452840+0000","last_active":"2024-06-11T13:28:50.039245+0000","last_peered":"2024-06-11T13:28:50.039245+0000","last_clean":"2024-06-11T13:28:50.039245+0000","last_became_active":"2024-06-11T13:28:17.452503+0000","last_became_peered":"2024-06-11T13:28:17.452503+0000","last_unstale":"2024-06-11T13:28:50.039245+0000","last_undegraded":"2024-06-11T13:28:50.039245+0000","last_fullsized":"2024-06-11T13:28:50.039245+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T21:49:46.680378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.454423+0000","last_change":"2024-06-11T13:28:19.457125+0000","last_active":"2024-06-11T13:28:51.454423+0000","last_peered":"2024-06-11T13:28:51.454423+0000","last_clean":"2024-06-11T13:28:51.454423+0000","last_became_active":"2024-06-11T13:28:19.456666+0000","last_became_peered":"2024-06-11T13:28:19.456666+0000","last_unstale":"2024-06-11T13:28:51.454423+0000","last_undegraded":"2024-06-11T13:28:51.454423+0000","last_fullsized":"2024-06-11T13:28:51.454423+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T19:58:57.614042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.945313+0000","last_change":"2024-06-11T13:28:13.187109+0000","last_active":"2024-06-11T13:28:49.945313+0000","last_peered":"2024-06-11T13:28:49.945313+0000","last_clean":"2024-06-11T13:28:49.945313+0000","last_became_active":"2024-06-11T13:28:13.186748+0000","last_became_peered":"2024-06-11T13:28:13.186748+0000","last_unstale":"2024-06-11T13:28:49.945313+0000","last_undegraded":"2024-06-11T13:28:49.945313+0000","last_fullsized":"2024-06-11T13:28:49.945313+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T17:55:41.469322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"68'5","reported_seq":35,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.454439+0000","last_change":"2024-06-11T13:28:15.423406+0000","last_active":"2024-06-11T13:28:51.454439+0000","last_peered":"2024-06-11T13:28:51.454439+0000","last_clean":"2024-06-11T13:28:51.454439+0000","last_became_active":"2024-06-11T13:28:15.422942+0000","last_became_peered":"2024-06-11T13:28:15.422942+0000","last_unstale":"2024-06-11T13:28:51.454439+0000","last_undegraded":"2024-06-11T13:28:51.454439+0000","last_fullsized":"2024-06-11T13:28:51.454439+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T22:04:17.940417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.944831+0000","last_change":"2024-06-11T13:28:17.451344+0000","last_active":"2024-06-11T13:28:49.944831+0000","last_peered":"2024-06-11T13:28:49.944831+0000","last_clean":"2024-06-11T13:28:49.944831+0000","last_became_active":"2024-06-11T13:28:17.451043+0000","last_became_peered":"2024-06-11T13:28:17.451043+0000","last_unstale":"2024-06-11T13:28:49.944831+0000","last_undegraded":"2024-06-11T13:28:49.944831+0000","last_fullsized":"2024-06-11T13:28:49.944831+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-13T00:25:28.323738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.050420+0000","last_change":"2024-06-11T13:28:19.462380+0000","last_active":"2024-06-11T13:28:51.050420+0000","last_peered":"2024-06-11T13:28:51.050420+0000","last_clean":"2024-06-11T13:28:51.050420+0000","last_became_active":"2024-06-11T13:28:19.461999+0000","last_became_peered":"2024-06-11T13:28:19.461999+0000","last_unstale":"2024-06-11T13:28:51.050420+0000","last_undegraded":"2024-06-11T13:28:51.050420+0000","last_fullsized":"2024-06-11T13:28:51.050420+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T15:46:02.464320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.945250+0000","last_change":"2024-06-11T13:28:13.183952+0000","last_active":"2024-06-11T13:28:49.945250+0000","last_peered":"2024-06-11T13:28:49.945250+0000","last_clean":"2024-06-11T13:28:49.945250+0000","last_became_active":"2024-06-11T13:28:13.183647+0000","last_became_peered":"2024-06-11T13:28:13.183647+0000","last_unstale":"2024-06-11T13:28:49.945250+0000","last_undegraded":"2024-06-11T13:28:49.945250+0000","last_fullsized":"2024-06-11T13:28:49.945250+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T22:32:55.509603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"68'9","reported_seq":41,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.454746+0000","last_change":"2024-06-11T13:28:15.423848+0000","last_active":"2024-06-11T13:28:51.454746+0000","last_peered":"2024-06-11T13:28:51.454746+0000","last_clean":"2024-06-11T13:28:51.454746+0000","last_became_active":"2024-06-11T13:28:15.423005+0000","last_became_peered":"2024-06-11T13:28:15.423005+0000","last_unstale":"2024-06-11T13:28:51.454746+0000","last_undegraded":"2024-06-11T13:28:51.454746+0000","last_fullsized":"2024-06-11T13:28:51.454746+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T17:10:25.210743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.050466+0000","last_change":"2024-06-11T13:28:17.457031+0000","last_active":"2024-06-11T13:28:51.050466+0000","last_peered":"2024-06-11T13:28:51.050466+0000","last_clean":"2024-06-11T13:28:51.050466+0000","last_became_active":"2024-06-11T13:28:17.456516+0000","last_became_peered":"2024-06-11T13:28:17.456516+0000","last_unstale":"2024-06-11T13:28:51.050466+0000","last_undegraded":"2024-06-11T13:28:51.050466+0000","last_fullsized":"2024-06-11T13:28:51.050466+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T15:29:41.820196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"68'1","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.942777+0000","last_change":"2024-06-11T13:28:19.463547+0000","last_active":"2024-06-11T13:28:49.942777+0000","last_peered":"2024-06-11T13:28:49.942777+0000","last_clean":"2024-06-11T13:28:49.942777+0000","last_became_active":"2024-06-11T13:28:19.462163+0000","last_became_peered":"2024-06-11T13:28:19.462163+0000","last_unstale":"2024-06-11T13:28:49.942777+0000","last_undegraded":"2024-06-11T13:28:49.942777+0000","last_fullsized":"2024-06-11T13:28:49.942777+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T14:30:39.875492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"68'17","reported_seq":53,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.454772+0000","last_change":"2024-06-11T13:28:15.423569+0000","last_active":"2024-06-11T13:28:51.454772+0000","last_peered":"2024-06-11T13:28:51.454772+0000","last_clean":"2024-06-11T13:28:51.454772+0000","last_became_active":"2024-06-11T13:28:15.423206+0000","last_became_peered":"2024-06-11T13:28:15.423206+0000","last_unstale":"2024-06-11T13:28:51.454772+0000","last_undegraded":"2024-06-11T13:28:51.454772+0000","last_fullsized":"2024-06-11T13:28:51.454772+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T19:42:24.824046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.945055+0000","last_change":"2024-06-11T13:28:13.180360+0000","last_active":"2024-06-11T13:28:49.945055+0000","last_peered":"2024-06-11T13:28:49.945055+0000","last_clean":"2024-06-11T13:28:49.945055+0000","last_became_active":"2024-06-11T13:28:13.179463+0000","last_became_peered":"2024-06-11T13:28:13.179463+0000","last_unstale":"2024-06-11T13:28:49.945055+0000","last_undegraded":"2024-06-11T13:28:49.945055+0000","last_fullsized":"2024-06-11T13:28:49.945055+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-13T00:06:50.134505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.039815+0000","last_change":"2024-06-11T13:28:17.456131+0000","last_active":"2024-06-11T13:28:50.039815+0000","last_peered":"2024-06-11T13:28:50.039815+0000","last_clean":"2024-06-11T13:28:50.039815+0000","last_became_active":"2024-06-11T13:28:17.455158+0000","last_became_peered":"2024-06-11T13:28:17.455158+0000","last_unstale":"2024-06-11T13:28:50.039815+0000","last_undegraded":"2024-06-11T13:28:50.039815+0000","last_fullsized":"2024-06-11T13:28:50.039815+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T19:41:29.645580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.946388+0000","last_change":"2024-06-11T13:28:19.457113+0000","last_active":"2024-06-11T13:28:49.946388+0000","last_peered":"2024-06-11T13:28:49.946388+0000","last_clean":"2024-06-11T13:28:49.946388+0000","last_became_active":"2024-06-11T13:28:19.456761+0000","last_became_peered":"2024-06-11T13:28:19.456761+0000","last_unstale":"2024-06-11T13:28:49.946388+0000","last_undegraded":"2024-06-11T13:28:49.946388+0000","last_fullsized":"2024-06-11T13:28:49.946388+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T21:53:57.692109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"68'10","reported_seq":40,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.454737+0000","last_change":"2024-06-11T13:28:15.424745+0000","last_active":"2024-06-11T13:28:51.454737+0000","last_peered":"2024-06-11T13:28:51.454737+0000","last_clean":"2024-06-11T13:28:51.454737+0000","last_became_active":"2024-06-11T13:28:15.424384+0000","last_became_peered":"2024-06-11T13:28:15.424384+0000","last_unstale":"2024-06-11T13:28:51.454737+0000","last_undegraded":"2024-06-11T13:28:51.454737+0000","last_fullsized":"2024-06-11T13:28:51.454737+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T20:28:30.196736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.040117+0000","last_change":"2024-06-11T13:28:13.188135+0000","last_active":"2024-06-11T13:28:50.040117+0000","last_peered":"2024-06-11T13:28:50.040117+0000","last_clean":"2024-06-11T13:28:50.040117+0000","last_became_active":"2024-06-11T13:28:13.187824+0000","last_became_peered":"2024-06-11T13:28:13.187824+0000","last_unstale":"2024-06-11T13:28:50.040117+0000","last_undegraded":"2024-06-11T13:28:50.040117+0000","last_fullsized":"2024-06-11T13:28:50.040117+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T16:54:37.369842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"68'8","reported_seq":47,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:29:33.244516+0000","last_change":"2024-06-11T13:28:17.452020+0000","last_active":"2024-06-11T13:29:33.244516+0000","last_peered":"2024-06-11T13:29:33.244516+0000","last_clean":"2024-06-11T13:29:33.244516+0000","last_became_active":"2024-06-11T13:28:17.451706+0000","last_became_peered":"2024-06-11T13:28:17.451706+0000","last_unstale":"2024-06-11T13:29:33.244516+0000","last_undegraded":"2024-06-11T13:29:33.244516+0000","last_fullsized":"2024-06-11T13:29:33.244516+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T20:44:28.252807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.454705+0000","last_change":"2024-06-11T13:28:19.457854+0000","last_active":"2024-06-11T13:28:51.454705+0000","last_peered":"2024-06-11T13:28:51.454705+0000","last_clean":"2024-06-11T13:28:51.454705+0000","last_became_active":"2024-06-11T13:28:19.457565+0000","last_became_peered":"2024-06-11T13:28:19.457565+0000","last_unstale":"2024-06-11T13:28:51.454705+0000","last_undegraded":"2024-06-11T13:28:51.454705+0000","last_fullsized":"2024-06-11T13:28:51.454705+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-13T00:55:14.966569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"68'15","reported_seq":50,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.040856+0000","last_change":"2024-06-11T13:28:15.426892+0000","last_active":"2024-06-11T13:28:50.040856+0000","last_peered":"2024-06-11T13:28:50.040856+0000","last_clean":"2024-06-11T13:28:50.040856+0000","last_became_active":"2024-06-11T13:28:15.426545+0000","last_became_peered":"2024-06-11T13:28:15.426545+0000","last_unstale":"2024-06-11T13:28:50.040856+0000","last_undegraded":"2024-06-11T13:28:50.040856+0000","last_fullsized":"2024-06-11T13:28:50.040856+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T19:08:25.374757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.039399+0000","last_change":"2024-06-11T13:28:13.189327+0000","last_active":"2024-06-11T13:28:50.039399+0000","last_peered":"2024-06-11T13:28:50.039399+0000","last_clean":"2024-06-11T13:28:50.039399+0000","last_became_active":"2024-06-11T13:28:13.188960+0000","last_became_peered":"2024-06-11T13:28:13.188960+0000","last_unstale":"2024-06-11T13:28:50.039399+0000","last_undegraded":"2024-06-11T13:28:50.039399+0000","last_fullsized":"2024-06-11T13:28:50.039399+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T15:02:47.835582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"68'8","reported_seq":47,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:29:33.244597+0000","last_change":"2024-06-11T13:28:17.454595+0000","last_active":"2024-06-11T13:29:33.244597+0000","last_peered":"2024-06-11T13:29:33.244597+0000","last_clean":"2024-06-11T13:29:33.244597+0000","last_became_active":"2024-06-11T13:28:17.454275+0000","last_became_peered":"2024-06-11T13:28:17.454275+0000","last_unstale":"2024-06-11T13:29:33.244597+0000","last_undegraded":"2024-06-11T13:29:33.244597+0000","last_fullsized":"2024-06-11T13:29:33.244597+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-13T00:13:07.986474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.050689+0000","last_change":"2024-06-11T13:28:19.456803+0000","last_active":"2024-06-11T13:28:51.050689+0000","last_peered":"2024-06-11T13:28:51.050689+0000","last_clean":"2024-06-11T13:28:51.050689+0000","last_became_active":"2024-06-11T13:28:19.456193+0000","last_became_peered":"2024-06-11T13:28:19.456193+0000","last_unstale":"2024-06-11T13:28:51.050689+0000","last_undegraded":"2024-06-11T13:28:51.050689+0000","last_fullsized":"2024-06-11T13:28:51.050689+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T19:16:32.736555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"68'11","reported_seq":44,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.454911+0000","last_change":"2024-06-11T13:28:15.420549+0000","last_active":"2024-06-11T13:28:51.454911+0000","last_peered":"2024-06-11T13:28:51.454911+0000","last_clean":"2024-06-11T13:28:51.454911+0000","last_became_active":"2024-06-11T13:28:15.419738+0000","last_became_peered":"2024-06-11T13:28:15.419738+0000","last_unstale":"2024-06-11T13:28:51.454911+0000","last_undegraded":"2024-06-11T13:28:51.454911+0000","last_fullsized":"2024-06-11T13:28:51.454911+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T18:45:27.891582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.944617+0000","last_change":"2024-06-11T13:28:13.186258+0000","last_active":"2024-06-11T13:28:49.944617+0000","last_peered":"2024-06-11T13:28:49.944617+0000","last_clean":"2024-06-11T13:28:49.944617+0000","last_became_active":"2024-06-11T13:28:13.185858+0000","last_became_peered":"2024-06-11T13:28:13.185858+0000","last_unstale":"2024-06-11T13:28:49.944617+0000","last_undegraded":"2024-06-11T13:28:49.944617+0000","last_fullsized":"2024-06-11T13:28:49.944617+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T18:11:30.855241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.050931+0000","last_change":"2024-06-11T13:28:17.457708+0000","last_active":"2024-06-11T13:28:51.050931+0000","last_peered":"2024-06-11T13:28:51.050931+0000","last_clean":"2024-06-11T13:28:51.050931+0000","last_became_active":"2024-06-11T13:28:17.457371+0000","last_became_peered":"2024-06-11T13:28:17.457371+0000","last_unstale":"2024-06-11T13:28:51.050931+0000","last_undegraded":"2024-06-11T13:28:51.050931+0000","last_fullsized":"2024-06-11T13:28:51.050931+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T23:19:32.764805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.031863+0000","last_change":"2024-06-11T13:28:19.548749+0000","last_active":"2024-06-11T13:28:50.031863+0000","last_peered":"2024-06-11T13:28:50.031863+0000","last_clean":"2024-06-11T13:28:50.031863+0000","last_became_active":"2024-06-11T13:28:19.548339+0000","last_became_peered":"2024-06-11T13:28:19.548339+0000","last_unstale":"2024-06-11T13:28:50.031863+0000","last_undegraded":"2024-06-11T13:28:50.031863+0000","last_fullsized":"2024-06-11T13:28:50.031863+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T15:33:09.658474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"68'14","reported_seq":46,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.454502+0000","last_change":"2024-06-11T13:28:15.419993+0000","last_active":"2024-06-11T13:28:51.454502+0000","last_peered":"2024-06-11T13:28:51.454502+0000","last_clean":"2024-06-11T13:28:51.454502+0000","last_became_active":"2024-06-11T13:28:15.419625+0000","last_became_peered":"2024-06-11T13:28:15.419625+0000","last_unstale":"2024-06-11T13:28:51.454502+0000","last_undegraded":"2024-06-11T13:28:51.454502+0000","last_fullsized":"2024-06-11T13:28:51.454502+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T13:31:14.387479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.549206+0000","last_change":"2024-06-11T13:28:13.184600+0000","last_active":"2024-06-11T13:28:50.549206+0000","last_peered":"2024-06-11T13:28:50.549206+0000","last_clean":"2024-06-11T13:28:50.549206+0000","last_became_active":"2024-06-11T13:28:13.183623+0000","last_became_peered":"2024-06-11T13:28:13.183623+0000","last_unstale":"2024-06-11T13:28:50.549206+0000","last_undegraded":"2024-06-11T13:28:50.549206+0000","last_fullsized":"2024-06-11T13:28:50.549206+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T21:38:39.160094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"68'8","reported_seq":47,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:29:33.244532+0000","last_change":"2024-06-11T13:28:17.455639+0000","last_active":"2024-06-11T13:29:33.244532+0000","last_peered":"2024-06-11T13:29:33.244532+0000","last_clean":"2024-06-11T13:29:33.244532+0000","last_became_active":"2024-06-11T13:28:17.455202+0000","last_became_peered":"2024-06-11T13:28:17.455202+0000","last_unstale":"2024-06-11T13:29:33.244532+0000","last_undegraded":"2024-06-11T13:29:33.244532+0000","last_fullsized":"2024-06-11T13:29:33.244532+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T13:44:30.136745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.944854+0000","last_change":"2024-06-11T13:28:19.549166+0000","last_active":"2024-06-11T13:28:49.944854+0000","last_peered":"2024-06-11T13:28:49.944854+0000","last_clean":"2024-06-11T13:28:49.944854+0000","last_became_active":"2024-06-11T13:28:19.548881+0000","last_became_peered":"2024-06-11T13:28:19.548881+0000","last_unstale":"2024-06-11T13:28:49.944854+0000","last_undegraded":"2024-06-11T13:28:49.944854+0000","last_fullsized":"2024-06-11T13:28:49.944854+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T21:02:42.466664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"68'18","reported_seq":57,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.031497+0000","last_change":"2024-06-11T13:28:15.599521+0000","last_active":"2024-06-11T13:28:50.031497+0000","last_peered":"2024-06-11T13:28:50.031497+0000","last_clean":"2024-06-11T13:28:50.031497+0000","last_became_active":"2024-06-11T13:28:15.599158+0000","last_became_peered":"2024-06-11T13:28:15.599158+0000","last_unstale":"2024-06-11T13:28:50.031497+0000","last_undegraded":"2024-06-11T13:28:50.031497+0000","last_fullsized":"2024-06-11T13:28:50.031497+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T13:51:02.836977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"61'1","reported_seq":30,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.040798+0000","last_change":"2024-06-11T13:28:13.188861+0000","last_active":"2024-06-11T13:28:50.040798+0000","last_peered":"2024-06-11T13:28:50.040798+0000","last_clean":"2024-06-11T13:28:50.040798+0000","last_became_active":"2024-06-11T13:28:13.188576+0000","last_became_peered":"2024-06-11T13:28:13.188576+0000","last_unstale":"2024-06-11T13:28:50.040798+0000","last_undegraded":"2024-06-11T13:28:50.040798+0000","last_fullsized":"2024-06-11T13:28:50.040798+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T21:25:57.212708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.454341+0000","last_change":"2024-06-11T13:28:17.447311+0000","last_active":"2024-06-11T13:28:51.454341+0000","last_peered":"2024-06-11T13:28:51.454341+0000","last_clean":"2024-06-11T13:28:51.454341+0000","last_became_active":"2024-06-11T13:28:17.446472+0000","last_became_peered":"2024-06-11T13:28:17.446472+0000","last_unstale":"2024-06-11T13:28:51.454341+0000","last_undegraded":"2024-06-11T13:28:51.454341+0000","last_fullsized":"2024-06-11T13:28:51.454341+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T16:04:40.986019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.454642+0000","last_change":"2024-06-11T13:28:19.461310+0000","last_active":"2024-06-11T13:28:51.454642+0000","last_peered":"2024-06-11T13:28:51.454642+0000","last_clean":"2024-06-11T13:28:51.454642+0000","last_became_active":"2024-06-11T13:28:19.460964+0000","last_became_peered":"2024-06-11T13:28:19.460964+0000","last_unstale":"2024-06-11T13:28:51.454642+0000","last_undegraded":"2024-06-11T13:28:51.454642+0000","last_fullsized":"2024-06-11T13:28:51.454642+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T20:52:14.875740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"68'19","reported_seq":61,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.943249+0000","last_change":"2024-06-11T13:28:15.597334+0000","last_active":"2024-06-11T13:28:49.943249+0000","last_peered":"2024-06-11T13:28:49.943249+0000","last_clean":"2024-06-11T13:28:49.943249+0000","last_became_active":"2024-06-11T13:28:15.596997+0000","last_became_peered":"2024-06-11T13:28:15.596997+0000","last_unstale":"2024-06-11T13:28:49.943249+0000","last_undegraded":"2024-06-11T13:28:49.943249+0000","last_fullsized":"2024-06-11T13:28:49.943249+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T18:54:34.631686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.945329+0000","last_change":"2024-06-11T13:28:13.187039+0000","last_active":"2024-06-11T13:28:49.945329+0000","last_peered":"2024-06-11T13:28:49.945329+0000","last_clean":"2024-06-11T13:28:49.945329+0000","last_became_active":"2024-06-11T13:28:13.186689+0000","last_became_peered":"2024-06-11T13:28:13.186689+0000","last_unstale":"2024-06-11T13:28:49.945329+0000","last_undegraded":"2024-06-11T13:28:49.945329+0000","last_fullsized":"2024-06-11T13:28:49.945329+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T13:30:37.646586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.548624+0000","last_change":"2024-06-11T13:28:17.453365+0000","last_active":"2024-06-11T13:28:50.548624+0000","last_peered":"2024-06-11T13:28:50.548624+0000","last_clean":"2024-06-11T13:28:50.548624+0000","last_became_active":"2024-06-11T13:28:17.452631+0000","last_became_peered":"2024-06-11T13:28:17.452631+0000","last_unstale":"2024-06-11T13:28:50.548624+0000","last_undegraded":"2024-06-11T13:28:50.548624+0000","last_fullsized":"2024-06-11T13:28:50.548624+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T22:33:41.316200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.039157+0000","last_change":"2024-06-11T13:28:19.548659+0000","last_active":"2024-06-11T13:28:50.039157+0000","last_peered":"2024-06-11T13:28:50.039157+0000","last_clean":"2024-06-11T13:28:50.039157+0000","last_became_active":"2024-06-11T13:28:19.548215+0000","last_became_peered":"2024-06-11T13:28:19.548215+0000","last_unstale":"2024-06-11T13:28:50.039157+0000","last_undegraded":"2024-06-11T13:28:50.039157+0000","last_fullsized":"2024-06-11T13:28:50.039157+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-13T00:24:40.719668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"68'10","reported_seq":40,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.040000+0000","last_change":"2024-06-11T13:28:15.424259+0000","last_active":"2024-06-11T13:28:50.040000+0000","last_peered":"2024-06-11T13:28:50.040000+0000","last_clean":"2024-06-11T13:28:50.040000+0000","last_became_active":"2024-06-11T13:28:15.423884+0000","last_became_peered":"2024-06-11T13:28:15.423884+0000","last_unstale":"2024-06-11T13:28:50.040000+0000","last_undegraded":"2024-06-11T13:28:50.040000+0000","last_fullsized":"2024-06-11T13:28:50.040000+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T23:34:08.611441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"61'1","reported_seq":35,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.040237+0000","last_change":"2024-06-11T13:28:13.189208+0000","last_active":"2024-06-11T13:28:50.040237+0000","last_peered":"2024-06-11T13:28:50.040237+0000","last_clean":"2024-06-11T13:28:50.040237+0000","last_became_active":"2024-06-11T13:28:13.187456+0000","last_became_peered":"2024-06-11T13:28:13.187456+0000","last_unstale":"2024-06-11T13:28:50.040237+0000","last_undegraded":"2024-06-11T13:28:50.040237+0000","last_fullsized":"2024-06-11T13:28:50.040237+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T17:28:59.916170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"68'8","reported_seq":50,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:29:33.244484+0000","last_change":"2024-06-11T13:28:17.454331+0000","last_active":"2024-06-11T13:29:33.244484+0000","last_peered":"2024-06-11T13:29:33.244484+0000","last_clean":"2024-06-11T13:29:33.244484+0000","last_became_active":"2024-06-11T13:28:17.454043+0000","last_became_peered":"2024-06-11T13:28:17.454043+0000","last_unstale":"2024-06-11T13:29:33.244484+0000","last_undegraded":"2024-06-11T13:29:33.244484+0000","last_fullsized":"2024-06-11T13:29:33.244484+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T13:51:30.785359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.942612+0000","last_change":"2024-06-11T13:28:19.463318+0000","last_active":"2024-06-11T13:28:49.942612+0000","last_peered":"2024-06-11T13:28:49.942612+0000","last_clean":"2024-06-11T13:28:49.942612+0000","last_became_active":"2024-06-11T13:28:19.462695+0000","last_became_peered":"2024-06-11T13:28:19.462695+0000","last_unstale":"2024-06-11T13:28:49.942612+0000","last_undegraded":"2024-06-11T13:28:49.942612+0000","last_fullsized":"2024-06-11T13:28:49.942612+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T22:45:18.631444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"68'28","reported_seq":91,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:29:33.244485+0000","last_change":"2024-06-11T13:28:15.428569+0000","last_active":"2024-06-11T13:29:33.244485+0000","last_peered":"2024-06-11T13:29:33.244485+0000","last_clean":"2024-06-11T13:29:33.244485+0000","last_became_active":"2024-06-11T13:28:15.428267+0000","last_became_peered":"2024-06-11T13:28:15.428267+0000","last_unstale":"2024-06-11T13:29:33.244485+0000","last_undegraded":"2024-06-11T13:29:33.244485+0000","last_fullsized":"2024-06-11T13:29:33.244485+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T18:39:04.384388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.050675+0000","last_change":"2024-06-11T13:28:13.182915+0000","last_active":"2024-06-11T13:28:51.050675+0000","last_peered":"2024-06-11T13:28:51.050675+0000","last_clean":"2024-06-11T13:28:51.050675+0000","last_became_active":"2024-06-11T13:28:13.182608+0000","last_became_peered":"2024-06-11T13:28:13.182608+0000","last_unstale":"2024-06-11T13:28:51.050675+0000","last_undegraded":"2024-06-11T13:28:51.050675+0000","last_fullsized":"2024-06-11T13:28:51.050675+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T23:24:44.892880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.943622+0000","last_change":"2024-06-11T13:28:17.460256+0000","last_active":"2024-06-11T13:28:49.943622+0000","last_peered":"2024-06-11T13:28:49.943622+0000","last_clean":"2024-06-11T13:28:49.943622+0000","last_became_active":"2024-06-11T13:28:17.459765+0000","last_became_peered":"2024-06-11T13:28:17.459765+0000","last_unstale":"2024-06-11T13:28:49.943622+0000","last_undegraded":"2024-06-11T13:28:49.943622+0000","last_fullsized":"2024-06-11T13:28:49.943622+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T22:36:22.716109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.033165+0000","last_change":"2024-06-11T13:28:19.459741+0000","last_active":"2024-06-11T13:28:50.033165+0000","last_peered":"2024-06-11T13:28:50.033165+0000","last_clean":"2024-06-11T13:28:50.033165+0000","last_became_active":"2024-06-11T13:28:19.459289+0000","last_became_peered":"2024-06-11T13:28:19.459289+0000","last_unstale":"2024-06-11T13:28:50.033165+0000","last_undegraded":"2024-06-11T13:28:50.033165+0000","last_fullsized":"2024-06-11T13:28:50.033165+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T22:08:28.682276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"68'13","reported_seq":52,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.040991+0000","last_change":"2024-06-11T13:28:15.432290+0000","last_active":"2024-06-11T13:28:50.040991+0000","last_peered":"2024-06-11T13:28:50.040991+0000","last_clean":"2024-06-11T13:28:50.040991+0000","last_became_active":"2024-06-11T13:28:15.431483+0000","last_became_peered":"2024-06-11T13:28:15.431483+0000","last_unstale":"2024-06-11T13:28:50.040991+0000","last_undegraded":"2024-06-11T13:28:50.040991+0000","last_fullsized":"2024-06-11T13:28:50.040991+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T16:52:06.965341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"69'5","reported_seq":121,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:29:39.813039+0000","last_change":"2024-06-11T13:28:17.451284+0000","last_active":"2024-06-11T13:29:39.813039+0000","last_peered":"2024-06-11T13:29:39.813039+0000","last_clean":"2024-06-11T13:29:39.813039+0000","last_became_active":"2024-06-11T13:28:13.188087+0000","last_became_peered":"2024-06-11T13:28:13.188087+0000","last_unstale":"2024-06-11T13:29:39.813039+0000","last_undegraded":"2024-06-11T13:29:39.813039+0000","last_fullsized":"2024-06-11T13:29:39.813039+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T22:58:03.955354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000481383,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":82,"num_read_kb":77,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.944927+0000","last_change":"2024-06-11T13:28:13.176414+0000","last_active":"2024-06-11T13:28:49.944927+0000","last_peered":"2024-06-11T13:28:49.944927+0000","last_clean":"2024-06-11T13:28:49.944927+0000","last_became_active":"2024-06-11T13:28:13.176108+0000","last_became_peered":"2024-06-11T13:28:13.176108+0000","last_unstale":"2024-06-11T13:28:49.944927+0000","last_undegraded":"2024-06-11T13:28:49.944927+0000","last_fullsized":"2024-06-11T13:28:49.944927+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-13T01:08:57.498698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.945323+0000","last_change":"2024-06-11T13:28:17.450721+0000","last_active":"2024-06-11T13:28:49.945323+0000","last_peered":"2024-06-11T13:28:49.945323+0000","last_clean":"2024-06-11T13:28:49.945323+0000","last_became_active":"2024-06-11T13:28:17.450357+0000","last_became_peered":"2024-06-11T13:28:17.450357+0000","last_unstale":"2024-06-11T13:28:49.945323+0000","last_undegraded":"2024-06-11T13:28:49.945323+0000","last_fullsized":"2024-06-11T13:28:49.945323+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T22:36:05.731389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.945158+0000","last_change":"2024-06-11T13:28:19.548117+0000","last_active":"2024-06-11T13:28:49.945158+0000","last_peered":"2024-06-11T13:28:49.945158+0000","last_clean":"2024-06-11T13:28:49.945158+0000","last_became_active":"2024-06-11T13:28:19.547782+0000","last_became_peered":"2024-06-11T13:28:19.547782+0000","last_unstale":"2024-06-11T13:28:49.945158+0000","last_undegraded":"2024-06-11T13:28:49.945158+0000","last_fullsized":"2024-06-11T13:28:49.945158+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T21:25:59.597610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"68'12","reported_seq":43,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.944052+0000","last_change":"2024-06-11T13:28:15.432010+0000","last_active":"2024-06-11T13:28:49.944052+0000","last_peered":"2024-06-11T13:28:49.944052+0000","last_clean":"2024-06-11T13:28:49.944052+0000","last_became_active":"2024-06-11T13:28:15.431515+0000","last_became_peered":"2024-06-11T13:28:15.431515+0000","last_unstale":"2024-06-11T13:28:49.944052+0000","last_undegraded":"2024-06-11T13:28:49.944052+0000","last_fullsized":"2024-06-11T13:28:49.944052+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T13:40:06.929108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":36,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.943001+0000","last_change":"2024-06-11T13:28:17.451445+0000","last_active":"2024-06-11T13:28:49.943001+0000","last_peered":"2024-06-11T13:28:49.943001+0000","last_clean":"2024-06-11T13:28:49.943001+0000","last_became_active":"2024-06-11T13:28:13.177645+0000","last_became_peered":"2024-06-11T13:28:13.177645+0000","last_unstale":"2024-06-11T13:28:49.943001+0000","last_undegraded":"2024-06-11T13:28:49.943001+0000","last_fullsized":"2024-06-11T13:28:49.943001+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T19:31:18.415958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030853080000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.944658+0000","last_change":"2024-06-11T13:28:13.185313+0000","last_active":"2024-06-11T13:28:49.944658+0000","last_peered":"2024-06-11T13:28:49.944658+0000","last_clean":"2024-06-11T13:28:49.944658+0000","last_became_active":"2024-06-11T13:28:13.185045+0000","last_became_peered":"2024-06-11T13:28:13.185045+0000","last_unstale":"2024-06-11T13:28:49.944658+0000","last_undegraded":"2024-06-11T13:28:49.944658+0000","last_fullsized":"2024-06-11T13:28:49.944658+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T18:40:25.221292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.050934+0000","last_change":"2024-06-11T13:28:17.455912+0000","last_active":"2024-06-11T13:28:51.050934+0000","last_peered":"2024-06-11T13:28:51.050934+0000","last_clean":"2024-06-11T13:28:51.050934+0000","last_became_active":"2024-06-11T13:28:17.455445+0000","last_became_peered":"2024-06-11T13:28:17.455445+0000","last_unstale":"2024-06-11T13:28:51.050934+0000","last_undegraded":"2024-06-11T13:28:51.050934+0000","last_fullsized":"2024-06-11T13:28:51.050934+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T21:56:37.105430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.039704+0000","last_change":"2024-06-11T13:28:19.461348+0000","last_active":"2024-06-11T13:28:50.039704+0000","last_peered":"2024-06-11T13:28:50.039704+0000","last_clean":"2024-06-11T13:28:50.039704+0000","last_became_active":"2024-06-11T13:28:19.460988+0000","last_became_peered":"2024-06-11T13:28:19.460988+0000","last_unstale":"2024-06-11T13:28:50.039704+0000","last_undegraded":"2024-06-11T13:28:50.039704+0000","last_fullsized":"2024-06-11T13:28:50.039704+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T23:37:04.637331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"68'16","reported_seq":65,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:29:33.244649+0000","last_change":"2024-06-11T13:28:15.599233+0000","last_active":"2024-06-11T13:29:33.244649+0000","last_peered":"2024-06-11T13:29:33.244649+0000","last_clean":"2024-06-11T13:29:33.244649+0000","last_became_active":"2024-06-11T13:28:15.598906+0000","last_became_peered":"2024-06-11T13:28:15.598906+0000","last_unstale":"2024-06-11T13:29:33.244649+0000","last_undegraded":"2024-06-11T13:29:33.244649+0000","last_fullsized":"2024-06-11T13:29:33.244649+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T17:58:57.982815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"65'2","reported_seq":38,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.031880+0000","last_change":"2024-06-11T13:28:17.440514+0000","last_active":"2024-06-11T13:28:50.031880+0000","last_peered":"2024-06-11T13:28:50.031880+0000","last_clean":"2024-06-11T13:28:50.031880+0000","last_became_active":"2024-06-11T13:28:13.184678+0000","last_became_peered":"2024-06-11T13:28:13.184678+0000","last_unstale":"2024-06-11T13:28:50.031880+0000","last_undegraded":"2024-06-11T13:28:50.031880+0000","last_fullsized":"2024-06-11T13:28:50.031880+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T22:44:49.176865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065293199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.050629+0000","last_change":"2024-06-11T13:28:13.180355+0000","last_active":"2024-06-11T13:28:51.050629+0000","last_peered":"2024-06-11T13:28:51.050629+0000","last_clean":"2024-06-11T13:28:51.050629+0000","last_became_active":"2024-06-11T13:28:13.179885+0000","last_became_peered":"2024-06-11T13:28:13.179885+0000","last_unstale":"2024-06-11T13:28:51.050629+0000","last_undegraded":"2024-06-11T13:28:51.050629+0000","last_fullsized":"2024-06-11T13:28:51.050629+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T19:29:49.164602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"70'128","reported_seq":273,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:52.020828+0000","last_change":"2024-06-11T13:28:00.364232+0000","last_active":"2024-06-11T13:28:52.020828+0000","last_peered":"2024-06-11T13:28:52.020828+0000","last_clean":"2024-06-11T13:28:52.020828+0000","last_became_active":"2024-06-11T13:28:00.363797+0000","last_became_peered":"2024-06-11T13:28:00.363797+0000","last_unstale":"2024-06-11T13:28:52.020828+0000","last_undegraded":"2024-06-11T13:28:52.020828+0000","last_fullsized":"2024-06-11T13:28:52.020828+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:25:59.379125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:25:59.379125+0000","last_clean_scrub_stamp":"2024-06-11T13:25:59.379125+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T18:23:25.764067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.945016+0000","last_change":"2024-06-11T13:28:17.453607+0000","last_active":"2024-06-11T13:28:49.945016+0000","last_peered":"2024-06-11T13:28:49.945016+0000","last_clean":"2024-06-11T13:28:49.945016+0000","last_became_active":"2024-06-11T13:28:17.452331+0000","last_became_peered":"2024-06-11T13:28:17.452331+0000","last_unstale":"2024-06-11T13:28:49.945016+0000","last_undegraded":"2024-06-11T13:28:49.945016+0000","last_fullsized":"2024-06-11T13:28:49.945016+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T17:05:44.373332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.051034+0000","last_change":"2024-06-11T13:28:19.457503+0000","last_active":"2024-06-11T13:28:51.051034+0000","last_peered":"2024-06-11T13:28:51.051034+0000","last_clean":"2024-06-11T13:28:51.051034+0000","last_became_active":"2024-06-11T13:28:19.457112+0000","last_became_peered":"2024-06-11T13:28:19.457112+0000","last_unstale":"2024-06-11T13:28:51.051034+0000","last_undegraded":"2024-06-11T13:28:51.051034+0000","last_fullsized":"2024-06-11T13:28:51.051034+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T19:12:49.788901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"68'19","reported_seq":56,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.549128+0000","last_change":"2024-06-11T13:28:15.597859+0000","last_active":"2024-06-11T13:28:50.549128+0000","last_peered":"2024-06-11T13:28:50.549128+0000","last_clean":"2024-06-11T13:28:50.549128+0000","last_became_active":"2024-06-11T13:28:15.597494+0000","last_became_peered":"2024-06-11T13:28:15.597494+0000","last_unstale":"2024-06-11T13:28:50.549128+0000","last_undegraded":"2024-06-11T13:28:50.549128+0000","last_fullsized":"2024-06-11T13:28:50.549128+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T22:07:14.319927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.947015+0000","last_change":"2024-06-11T13:28:13.175188+0000","last_active":"2024-06-11T13:28:49.947015+0000","last_peered":"2024-06-11T13:28:49.947015+0000","last_clean":"2024-06-11T13:28:49.947015+0000","last_became_active":"2024-06-11T13:28:13.174844+0000","last_became_peered":"2024-06-11T13:28:13.174844+0000","last_unstale":"2024-06-11T13:28:49.947015+0000","last_undegraded":"2024-06-11T13:28:49.947015+0000","last_fullsized":"2024-06-11T13:28:49.947015+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T15:02:06.405290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.946221+0000","last_change":"2024-06-11T13:28:17.454294+0000","last_active":"2024-06-11T13:28:49.946221+0000","last_peered":"2024-06-11T13:28:49.946221+0000","last_clean":"2024-06-11T13:28:49.946221+0000","last_became_active":"2024-06-11T13:28:17.453974+0000","last_became_peered":"2024-06-11T13:28:17.453974+0000","last_unstale":"2024-06-11T13:28:49.946221+0000","last_undegraded":"2024-06-11T13:28:49.946221+0000","last_fullsized":"2024-06-11T13:28:49.946221+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T19:18:14.396961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.944699+0000","last_change":"2024-06-11T13:28:19.461481+0000","last_active":"2024-06-11T13:28:49.944699+0000","last_peered":"2024-06-11T13:28:49.944699+0000","last_clean":"2024-06-11T13:28:49.944699+0000","last_became_active":"2024-06-11T13:28:19.461198+0000","last_became_peered":"2024-06-11T13:28:19.461198+0000","last_unstale":"2024-06-11T13:28:49.944699+0000","last_undegraded":"2024-06-11T13:28:49.944699+0000","last_fullsized":"2024-06-11T13:28:49.944699+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T13:33:08.490127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"68'9","reported_seq":41,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.943210+0000","last_change":"2024-06-11T13:28:15.430307+0000","last_active":"2024-06-11T13:28:49.943210+0000","last_peered":"2024-06-11T13:28:49.943210+0000","last_clean":"2024-06-11T13:28:49.943210+0000","last_became_active":"2024-06-11T13:28:15.429052+0000","last_became_peered":"2024-06-11T13:28:15.429052+0000","last_unstale":"2024-06-11T13:28:49.943210+0000","last_undegraded":"2024-06-11T13:28:49.943210+0000","last_fullsized":"2024-06-11T13:28:49.943210+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T22:16:55.706916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.040105+0000","last_change":"2024-06-11T13:28:13.189710+0000","last_active":"2024-06-11T13:28:50.040105+0000","last_peered":"2024-06-11T13:28:50.040105+0000","last_clean":"2024-06-11T13:28:50.040105+0000","last_became_active":"2024-06-11T13:28:13.189153+0000","last_became_peered":"2024-06-11T13:28:13.189153+0000","last_unstale":"2024-06-11T13:28:50.040105+0000","last_undegraded":"2024-06-11T13:28:50.040105+0000","last_fullsized":"2024-06-11T13:28:50.040105+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T13:43:52.827970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.945047+0000","last_change":"2024-06-11T13:28:17.443853+0000","last_active":"2024-06-11T13:28:49.945047+0000","last_peered":"2024-06-11T13:28:49.945047+0000","last_clean":"2024-06-11T13:28:49.945047+0000","last_became_active":"2024-06-11T13:28:17.443520+0000","last_became_peered":"2024-06-11T13:28:17.443520+0000","last_unstale":"2024-06-11T13:28:49.945047+0000","last_undegraded":"2024-06-11T13:28:49.945047+0000","last_fullsized":"2024-06-11T13:28:49.945047+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T17:44:22.792663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.942888+0000","last_change":"2024-06-11T13:28:19.461977+0000","last_active":"2024-06-11T13:28:49.942888+0000","last_peered":"2024-06-11T13:28:49.942888+0000","last_clean":"2024-06-11T13:28:49.942888+0000","last_became_active":"2024-06-11T13:28:19.460370+0000","last_became_peered":"2024-06-11T13:28:19.460370+0000","last_unstale":"2024-06-11T13:28:49.942888+0000","last_undegraded":"2024-06-11T13:28:49.942888+0000","last_fullsized":"2024-06-11T13:28:49.942888+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T18:19:29.644278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"68'15","reported_seq":50,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.050290+0000","last_change":"2024-06-11T13:28:15.599585+0000","last_active":"2024-06-11T13:28:51.050290+0000","last_peered":"2024-06-11T13:28:51.050290+0000","last_clean":"2024-06-11T13:28:51.050290+0000","last_became_active":"2024-06-11T13:28:15.599247+0000","last_became_peered":"2024-06-11T13:28:15.599247+0000","last_unstale":"2024-06-11T13:28:51.050290+0000","last_undegraded":"2024-06-11T13:28:51.050290+0000","last_fullsized":"2024-06-11T13:28:51.050290+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T16:10:15.007390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.946012+0000","last_change":"2024-06-11T13:28:13.181623+0000","last_active":"2024-06-11T13:28:49.946012+0000","last_peered":"2024-06-11T13:28:49.946012+0000","last_clean":"2024-06-11T13:28:49.946012+0000","last_became_active":"2024-06-11T13:28:13.180442+0000","last_became_peered":"2024-06-11T13:28:13.180442+0000","last_unstale":"2024-06-11T13:28:49.946012+0000","last_undegraded":"2024-06-11T13:28:49.946012+0000","last_fullsized":"2024-06-11T13:28:49.946012+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T21:19:42.863903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"68'8","reported_seq":47,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:29:33.244578+0000","last_change":"2024-06-11T13:28:17.454160+0000","last_active":"2024-06-11T13:29:33.244578+0000","last_peered":"2024-06-11T13:29:33.244578+0000","last_clean":"2024-06-11T13:29:33.244578+0000","last_became_active":"2024-06-11T13:28:17.453879+0000","last_became_peered":"2024-06-11T13:28:17.453879+0000","last_unstale":"2024-06-11T13:29:33.244578+0000","last_undegraded":"2024-06-11T13:29:33.244578+0000","last_fullsized":"2024-06-11T13:29:33.244578+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T22:37:47.435002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.050392+0000","last_change":"2024-06-11T13:28:19.548196+0000","last_active":"2024-06-11T13:28:51.050392+0000","last_peered":"2024-06-11T13:28:51.050392+0000","last_clean":"2024-06-11T13:28:51.050392+0000","last_became_active":"2024-06-11T13:28:19.547915+0000","last_became_peered":"2024-06-11T13:28:19.547915+0000","last_unstale":"2024-06-11T13:28:51.050392+0000","last_undegraded":"2024-06-11T13:28:51.050392+0000","last_fullsized":"2024-06-11T13:28:51.050392+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T17:18:55.545802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"68'12","reported_seq":48,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.454357+0000","last_change":"2024-06-11T13:28:15.423664+0000","last_active":"2024-06-11T13:28:51.454357+0000","last_peered":"2024-06-11T13:28:51.454357+0000","last_clean":"2024-06-11T13:28:51.454357+0000","last_became_active":"2024-06-11T13:28:15.423353+0000","last_became_peered":"2024-06-11T13:28:15.423353+0000","last_unstale":"2024-06-11T13:28:51.454357+0000","last_undegraded":"2024-06-11T13:28:51.454357+0000","last_fullsized":"2024-06-11T13:28:51.454357+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T19:12:37.550194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"61'2","reported_seq":41,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.455075+0000","last_change":"2024-06-11T13:28:13.181088+0000","last_active":"2024-06-11T13:28:51.455075+0000","last_peered":"2024-06-11T13:28:51.455075+0000","last_clean":"2024-06-11T13:28:51.455075+0000","last_became_active":"2024-06-11T13:28:13.179671+0000","last_became_peered":"2024-06-11T13:28:13.179671+0000","last_unstale":"2024-06-11T13:28:51.455075+0000","last_undegraded":"2024-06-11T13:28:51.455075+0000","last_fullsized":"2024-06-11T13:28:51.455075+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T19:33:59.988970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.946208+0000","last_change":"2024-06-11T13:28:17.451962+0000","last_active":"2024-06-11T13:28:49.946208+0000","last_peered":"2024-06-11T13:28:49.946208+0000","last_clean":"2024-06-11T13:28:49.946208+0000","last_became_active":"2024-06-11T13:28:17.451641+0000","last_became_peered":"2024-06-11T13:28:17.451641+0000","last_unstale":"2024-06-11T13:28:49.946208+0000","last_undegraded":"2024-06-11T13:28:49.946208+0000","last_fullsized":"2024-06-11T13:28:49.946208+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T21:30:42.621687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.031943+0000","last_change":"2024-06-11T13:28:19.465914+0000","last_active":"2024-06-11T13:28:50.031943+0000","last_peered":"2024-06-11T13:28:50.031943+0000","last_clean":"2024-06-11T13:28:50.031943+0000","last_became_active":"2024-06-11T13:28:19.464999+0000","last_became_peered":"2024-06-11T13:28:19.464999+0000","last_unstale":"2024-06-11T13:28:50.031943+0000","last_undegraded":"2024-06-11T13:28:50.031943+0000","last_fullsized":"2024-06-11T13:28:50.031943+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T23:36:56.227801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"68'9","reported_seq":41,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.944227+0000","last_change":"2024-06-11T13:28:15.599340+0000","last_active":"2024-06-11T13:28:49.944227+0000","last_peered":"2024-06-11T13:28:49.944227+0000","last_clean":"2024-06-11T13:28:49.944227+0000","last_became_active":"2024-06-11T13:28:15.599052+0000","last_became_peered":"2024-06-11T13:28:15.599052+0000","last_unstale":"2024-06-11T13:28:49.944227+0000","last_undegraded":"2024-06-11T13:28:49.944227+0000","last_fullsized":"2024-06-11T13:28:49.944227+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T21:12:32.237748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.946288+0000","last_change":"2024-06-11T13:28:13.182839+0000","last_active":"2024-06-11T13:28:49.946288+0000","last_peered":"2024-06-11T13:28:49.946288+0000","last_clean":"2024-06-11T13:28:49.946288+0000","last_became_active":"2024-06-11T13:28:13.182011+0000","last_became_peered":"2024-06-11T13:28:13.182011+0000","last_unstale":"2024-06-11T13:28:49.946288+0000","last_undegraded":"2024-06-11T13:28:49.946288+0000","last_fullsized":"2024-06-11T13:28:49.946288+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T20:58:31.725113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.033448+0000","last_change":"2024-06-11T13:28:17.453737+0000","last_active":"2024-06-11T13:28:50.033448+0000","last_peered":"2024-06-11T13:28:50.033448+0000","last_clean":"2024-06-11T13:28:50.033448+0000","last_became_active":"2024-06-11T13:28:17.453353+0000","last_became_peered":"2024-06-11T13:28:17.453353+0000","last_unstale":"2024-06-11T13:28:50.033448+0000","last_undegraded":"2024-06-11T13:28:50.033448+0000","last_fullsized":"2024-06-11T13:28:50.033448+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T16:57:34.452332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.946436+0000","last_change":"2024-06-11T13:28:19.457532+0000","last_active":"2024-06-11T13:28:49.946436+0000","last_peered":"2024-06-11T13:28:49.946436+0000","last_clean":"2024-06-11T13:28:49.946436+0000","last_became_active":"2024-06-11T13:28:19.457210+0000","last_became_peered":"2024-06-11T13:28:19.457210+0000","last_unstale":"2024-06-11T13:28:49.946436+0000","last_undegraded":"2024-06-11T13:28:49.946436+0000","last_fullsized":"2024-06-11T13:28:49.946436+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T23:03:22.695531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.548955+0000","last_change":"2024-06-11T13:28:13.185332+0000","last_active":"2024-06-11T13:28:50.548955+0000","last_peered":"2024-06-11T13:28:50.548955+0000","last_clean":"2024-06-11T13:28:50.548955+0000","last_became_active":"2024-06-11T13:28:13.183098+0000","last_became_peered":"2024-06-11T13:28:13.183098+0000","last_unstale":"2024-06-11T13:28:50.548955+0000","last_undegraded":"2024-06-11T13:28:50.548955+0000","last_fullsized":"2024-06-11T13:28:50.548955+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T19:07:47.442319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"68'6","reported_seq":34,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.033548+0000","last_change":"2024-06-11T13:28:15.598731+0000","last_active":"2024-06-11T13:28:50.033548+0000","last_peered":"2024-06-11T13:28:50.033548+0000","last_clean":"2024-06-11T13:28:50.033548+0000","last_became_active":"2024-06-11T13:28:15.598410+0000","last_became_peered":"2024-06-11T13:28:15.598410+0000","last_unstale":"2024-06-11T13:28:50.033548+0000","last_undegraded":"2024-06-11T13:28:50.033548+0000","last_fullsized":"2024-06-11T13:28:50.033548+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T18:13:52.552294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.050832+0000","last_change":"2024-06-11T13:28:17.456431+0000","last_active":"2024-06-11T13:28:51.050832+0000","last_peered":"2024-06-11T13:28:51.050832+0000","last_clean":"2024-06-11T13:28:51.050832+0000","last_became_active":"2024-06-11T13:28:17.456112+0000","last_became_peered":"2024-06-11T13:28:17.456112+0000","last_unstale":"2024-06-11T13:28:51.050832+0000","last_undegraded":"2024-06-11T13:28:51.050832+0000","last_fullsized":"2024-06-11T13:28:51.050832+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T18:54:14.165022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.945120+0000","last_change":"2024-06-11T13:28:19.548544+0000","last_active":"2024-06-11T13:28:49.945120+0000","last_peered":"2024-06-11T13:28:49.945120+0000","last_clean":"2024-06-11T13:28:49.945120+0000","last_became_active":"2024-06-11T13:28:19.548259+0000","last_became_peered":"2024-06-11T13:28:19.548259+0000","last_unstale":"2024-06-11T13:28:49.945120+0000","last_undegraded":"2024-06-11T13:28:49.945120+0000","last_fullsized":"2024-06-11T13:28:49.945120+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T20:54:44.002109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"68'10","reported_seq":40,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.032738+0000","last_change":"2024-06-11T13:28:15.597642+0000","last_active":"2024-06-11T13:28:50.032738+0000","last_peered":"2024-06-11T13:28:50.032738+0000","last_clean":"2024-06-11T13:28:50.032738+0000","last_became_active":"2024-06-11T13:28:15.597175+0000","last_became_peered":"2024-06-11T13:28:15.597175+0000","last_unstale":"2024-06-11T13:28:50.032738+0000","last_undegraded":"2024-06-11T13:28:50.032738+0000","last_fullsized":"2024-06-11T13:28:50.032738+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T21:39:28.191888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.050623+0000","last_change":"2024-06-11T13:28:13.181755+0000","last_active":"2024-06-11T13:28:51.050623+0000","last_peered":"2024-06-11T13:28:51.050623+0000","last_clean":"2024-06-11T13:28:51.050623+0000","last_became_active":"2024-06-11T13:28:13.180056+0000","last_became_peered":"2024-06-11T13:28:13.180056+0000","last_unstale":"2024-06-11T13:28:51.050623+0000","last_undegraded":"2024-06-11T13:28:51.050623+0000","last_fullsized":"2024-06-11T13:28:51.050623+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T22:04:35.756359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"68'8","reported_seq":47,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:29:33.244577+0000","last_change":"2024-06-11T13:28:17.456221+0000","last_active":"2024-06-11T13:29:33.244577+0000","last_peered":"2024-06-11T13:29:33.244577+0000","last_clean":"2024-06-11T13:29:33.244577+0000","last_became_active":"2024-06-11T13:28:17.455856+0000","last_became_peered":"2024-06-11T13:28:17.455856+0000","last_unstale":"2024-06-11T13:29:33.244577+0000","last_undegraded":"2024-06-11T13:29:33.244577+0000","last_fullsized":"2024-06-11T13:29:33.244577+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T23:48:17.791639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.943805+0000","last_change":"2024-06-11T13:28:19.462632+0000","last_active":"2024-06-11T13:28:49.943805+0000","last_peered":"2024-06-11T13:28:49.943805+0000","last_clean":"2024-06-11T13:28:49.943805+0000","last_became_active":"2024-06-11T13:28:19.462089+0000","last_became_peered":"2024-06-11T13:28:19.462089+0000","last_unstale":"2024-06-11T13:28:49.943805+0000","last_undegraded":"2024-06-11T13:28:49.943805+0000","last_fullsized":"2024-06-11T13:28:49.943805+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-13T00:41:07.090299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"68'9","reported_seq":41,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.050842+0000","last_change":"2024-06-11T13:28:15.598208+0000","last_active":"2024-06-11T13:28:51.050842+0000","last_peered":"2024-06-11T13:28:51.050842+0000","last_clean":"2024-06-11T13:28:51.050842+0000","last_became_active":"2024-06-11T13:28:15.597896+0000","last_became_peered":"2024-06-11T13:28:15.597896+0000","last_unstale":"2024-06-11T13:28:51.050842+0000","last_undegraded":"2024-06-11T13:28:51.050842+0000","last_fullsized":"2024-06-11T13:28:51.050842+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T15:53:46.789334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"61'1","reported_seq":35,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.943443+0000","last_change":"2024-06-11T13:28:13.181673+0000","last_active":"2024-06-11T13:28:49.943443+0000","last_peered":"2024-06-11T13:28:49.943443+0000","last_clean":"2024-06-11T13:28:49.943443+0000","last_became_active":"2024-06-11T13:28:13.181384+0000","last_became_peered":"2024-06-11T13:28:13.181384+0000","last_unstale":"2024-06-11T13:28:49.943443+0000","last_undegraded":"2024-06-11T13:28:49.943443+0000","last_fullsized":"2024-06-11T13:28:49.943443+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T16:20:04.406915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"68'8","reported_seq":47,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:29:33.244592+0000","last_change":"2024-06-11T13:28:17.448787+0000","last_active":"2024-06-11T13:29:33.244592+0000","last_peered":"2024-06-11T13:29:33.244592+0000","last_clean":"2024-06-11T13:29:33.244592+0000","last_became_active":"2024-06-11T13:28:17.448475+0000","last_became_peered":"2024-06-11T13:28:17.448475+0000","last_unstale":"2024-06-11T13:29:33.244592+0000","last_undegraded":"2024-06-11T13:29:33.244592+0000","last_fullsized":"2024-06-11T13:29:33.244592+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T17:44:36.219808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.454963+0000","last_change":"2024-06-11T13:28:19.455787+0000","last_active":"2024-06-11T13:28:51.454963+0000","last_peered":"2024-06-11T13:28:51.454963+0000","last_clean":"2024-06-11T13:28:51.454963+0000","last_became_active":"2024-06-11T13:28:19.455507+0000","last_became_peered":"2024-06-11T13:28:19.455507+0000","last_unstale":"2024-06-11T13:28:51.454963+0000","last_undegraded":"2024-06-11T13:28:51.454963+0000","last_fullsized":"2024-06-11T13:28:51.454963+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T16:57:52.936758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"68'9","reported_seq":41,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.039870+0000","last_change":"2024-06-11T13:28:15.429400+0000","last_active":"2024-06-11T13:28:50.039870+0000","last_peered":"2024-06-11T13:28:50.039870+0000","last_clean":"2024-06-11T13:28:50.039870+0000","last_became_active":"2024-06-11T13:28:15.429093+0000","last_became_peered":"2024-06-11T13:28:15.429093+0000","last_unstale":"2024-06-11T13:28:50.039870+0000","last_undegraded":"2024-06-11T13:28:50.039870+0000","last_fullsized":"2024-06-11T13:28:50.039870+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T13:43:41.146514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.548957+0000","last_change":"2024-06-11T13:28:13.182165+0000","last_active":"2024-06-11T13:28:50.548957+0000","last_peered":"2024-06-11T13:28:50.548957+0000","last_clean":"2024-06-11T13:28:50.548957+0000","last_became_active":"2024-06-11T13:28:13.181895+0000","last_became_peered":"2024-06-11T13:28:13.181895+0000","last_unstale":"2024-06-11T13:28:50.548957+0000","last_undegraded":"2024-06-11T13:28:50.548957+0000","last_fullsized":"2024-06-11T13:28:50.548957+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T13:44:29.552937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.032032+0000","last_change":"2024-06-11T13:28:17.454531+0000","last_active":"2024-06-11T13:28:50.032032+0000","last_peered":"2024-06-11T13:28:50.032032+0000","last_clean":"2024-06-11T13:28:50.032032+0000","last_became_active":"2024-06-11T13:28:17.454160+0000","last_became_peered":"2024-06-11T13:28:17.454160+0000","last_unstale":"2024-06-11T13:28:50.032032+0000","last_undegraded":"2024-06-11T13:28:50.032032+0000","last_fullsized":"2024-06-11T13:28:50.032032+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T13:59:47.847881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.942610+0000","last_change":"2024-06-11T13:28:19.463034+0000","last_active":"2024-06-11T13:28:49.942610+0000","last_peered":"2024-06-11T13:28:49.942610+0000","last_clean":"2024-06-11T13:28:49.942610+0000","last_became_active":"2024-06-11T13:28:19.461138+0000","last_became_peered":"2024-06-11T13:28:19.461138+0000","last_unstale":"2024-06-11T13:28:49.942610+0000","last_undegraded":"2024-06-11T13:28:49.942610+0000","last_fullsized":"2024-06-11T13:28:49.942610+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-13T01:16:08.991874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"68'11","reported_seq":44,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.454463+0000","last_change":"2024-06-11T13:28:15.419950+0000","last_active":"2024-06-11T13:28:51.454463+0000","last_peered":"2024-06-11T13:28:51.454463+0000","last_clean":"2024-06-11T13:28:51.454463+0000","last_became_active":"2024-06-11T13:28:15.419594+0000","last_became_peered":"2024-06-11T13:28:15.419594+0000","last_unstale":"2024-06-11T13:28:51.454463+0000","last_undegraded":"2024-06-11T13:28:51.454463+0000","last_fullsized":"2024-06-11T13:28:51.454463+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T22:27:46.009525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.040019+0000","last_change":"2024-06-11T13:28:13.189433+0000","last_active":"2024-06-11T13:28:50.040019+0000","last_peered":"2024-06-11T13:28:50.040019+0000","last_clean":"2024-06-11T13:28:50.040019+0000","last_became_active":"2024-06-11T13:28:13.189104+0000","last_became_peered":"2024-06-11T13:28:13.189104+0000","last_unstale":"2024-06-11T13:28:50.040019+0000","last_undegraded":"2024-06-11T13:28:50.040019+0000","last_fullsized":"2024-06-11T13:28:50.040019+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T19:37:38.570887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.039713+0000","last_change":"2024-06-11T13:28:17.454941+0000","last_active":"2024-06-11T13:28:50.039713+0000","last_peered":"2024-06-11T13:28:50.039713+0000","last_clean":"2024-06-11T13:28:50.039713+0000","last_became_active":"2024-06-11T13:28:17.454491+0000","last_became_peered":"2024-06-11T13:28:17.454491+0000","last_unstale":"2024-06-11T13:28:50.039713+0000","last_undegraded":"2024-06-11T13:28:50.039713+0000","last_fullsized":"2024-06-11T13:28:50.039713+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T20:07:52.471274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.031368+0000","last_change":"2024-06-11T13:28:19.463446+0000","last_active":"2024-06-11T13:28:50.031368+0000","last_peered":"2024-06-11T13:28:50.031368+0000","last_clean":"2024-06-11T13:28:50.031368+0000","last_became_active":"2024-06-11T13:28:19.463098+0000","last_became_peered":"2024-06-11T13:28:19.463098+0000","last_unstale":"2024-06-11T13:28:50.031368+0000","last_undegraded":"2024-06-11T13:28:50.031368+0000","last_fullsized":"2024-06-11T13:28:50.031368+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T19:28:15.335320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"68'4","reported_seq":31,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.031392+0000","last_change":"2024-06-11T13:28:15.430394+0000","last_active":"2024-06-11T13:28:50.031392+0000","last_peered":"2024-06-11T13:28:50.031392+0000","last_clean":"2024-06-11T13:28:50.031392+0000","last_became_active":"2024-06-11T13:28:15.429952+0000","last_became_peered":"2024-06-11T13:28:15.429952+0000","last_unstale":"2024-06-11T13:28:50.031392+0000","last_undegraded":"2024-06-11T13:28:50.031392+0000","last_fullsized":"2024-06-11T13:28:50.031392+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T18:38:56.317846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.050948+0000","last_change":"2024-06-11T13:28:13.178773+0000","last_active":"2024-06-11T13:28:51.050948+0000","last_peered":"2024-06-11T13:28:51.050948+0000","last_clean":"2024-06-11T13:28:51.050948+0000","last_became_active":"2024-06-11T13:28:13.178476+0000","last_became_peered":"2024-06-11T13:28:13.178476+0000","last_unstale":"2024-06-11T13:28:51.050948+0000","last_undegraded":"2024-06-11T13:28:51.050948+0000","last_fullsized":"2024-06-11T13:28:51.050948+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-13T00:39:08.981141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.548633+0000","last_change":"2024-06-11T13:28:17.455687+0000","last_active":"2024-06-11T13:28:50.548633+0000","last_peered":"2024-06-11T13:28:50.548633+0000","last_clean":"2024-06-11T13:28:50.548633+0000","last_became_active":"2024-06-11T13:28:17.455407+0000","last_became_peered":"2024-06-11T13:28:17.455407+0000","last_unstale":"2024-06-11T13:28:50.548633+0000","last_undegraded":"2024-06-11T13:28:50.548633+0000","last_fullsized":"2024-06-11T13:28:50.548633+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T19:18:38.407996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.944755+0000","last_change":"2024-06-11T13:28:19.459144+0000","last_active":"2024-06-11T13:28:49.944755+0000","last_peered":"2024-06-11T13:28:49.944755+0000","last_clean":"2024-06-11T13:28:49.944755+0000","last_became_active":"2024-06-11T13:28:19.458871+0000","last_became_peered":"2024-06-11T13:28:19.458871+0000","last_unstale":"2024-06-11T13:28:49.944755+0000","last_undegraded":"2024-06-11T13:28:49.944755+0000","last_fullsized":"2024-06-11T13:28:49.944755+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T16:56:25.393539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"68'11","reported_seq":44,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.031489+0000","last_change":"2024-06-11T13:28:15.597576+0000","last_active":"2024-06-11T13:28:50.031489+0000","last_peered":"2024-06-11T13:28:50.031489+0000","last_clean":"2024-06-11T13:28:50.031489+0000","last_became_active":"2024-06-11T13:28:15.597267+0000","last_became_peered":"2024-06-11T13:28:15.597267+0000","last_unstale":"2024-06-11T13:28:50.031489+0000","last_undegraded":"2024-06-11T13:28:50.031489+0000","last_fullsized":"2024-06-11T13:28:50.031489+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T22:36:35.613372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.549140+0000","last_change":"2024-06-11T13:28:13.183716+0000","last_active":"2024-06-11T13:28:50.549140+0000","last_peered":"2024-06-11T13:28:50.549140+0000","last_clean":"2024-06-11T13:28:50.549140+0000","last_became_active":"2024-06-11T13:28:13.182944+0000","last_became_peered":"2024-06-11T13:28:13.182944+0000","last_unstale":"2024-06-11T13:28:50.549140+0000","last_undegraded":"2024-06-11T13:28:50.549140+0000","last_fullsized":"2024-06-11T13:28:50.549140+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T21:46:49.799256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.944452+0000","last_change":"2024-06-11T13:28:17.454563+0000","last_active":"2024-06-11T13:28:49.944452+0000","last_peered":"2024-06-11T13:28:49.944452+0000","last_clean":"2024-06-11T13:28:49.944452+0000","last_became_active":"2024-06-11T13:28:17.453614+0000","last_became_peered":"2024-06-11T13:28:17.453614+0000","last_unstale":"2024-06-11T13:28:49.944452+0000","last_undegraded":"2024-06-11T13:28:49.944452+0000","last_fullsized":"2024-06-11T13:28:49.944452+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T13:40:40.512080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.031895+0000","last_change":"2024-06-11T13:28:19.550177+0000","last_active":"2024-06-11T13:28:50.031895+0000","last_peered":"2024-06-11T13:28:50.031895+0000","last_clean":"2024-06-11T13:28:50.031895+0000","last_became_active":"2024-06-11T13:28:19.549829+0000","last_became_peered":"2024-06-11T13:28:19.549829+0000","last_unstale":"2024-06-11T13:28:50.031895+0000","last_undegraded":"2024-06-11T13:28:50.031895+0000","last_fullsized":"2024-06-11T13:28:50.031895+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T22:13:54.754536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"68'1","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.944917+0000","last_change":"2024-06-11T13:28:19.459347+0000","last_active":"2024-06-11T13:28:49.944917+0000","last_peered":"2024-06-11T13:28:49.944917+0000","last_clean":"2024-06-11T13:28:49.944917+0000","last_became_active":"2024-06-11T13:28:19.458967+0000","last_became_peered":"2024-06-11T13:28:19.458967+0000","last_unstale":"2024-06-11T13:28:49.944917+0000","last_undegraded":"2024-06-11T13:28:49.944917+0000","last_fullsized":"2024-06-11T13:28:49.944917+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T22:42:00.605019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.050322+0000","last_change":"2024-06-11T13:28:13.179416+0000","last_active":"2024-06-11T13:28:51.050322+0000","last_peered":"2024-06-11T13:28:51.050322+0000","last_clean":"2024-06-11T13:28:51.050322+0000","last_became_active":"2024-06-11T13:28:13.178793+0000","last_became_peered":"2024-06-11T13:28:13.178793+0000","last_unstale":"2024-06-11T13:28:51.050322+0000","last_undegraded":"2024-06-11T13:28:51.050322+0000","last_fullsized":"2024-06-11T13:28:51.050322+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T22:28:56.806331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"68'10","reported_seq":40,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.943929+0000","last_change":"2024-06-11T13:28:15.599580+0000","last_active":"2024-06-11T13:28:49.943929+0000","last_peered":"2024-06-11T13:28:49.943929+0000","last_clean":"2024-06-11T13:28:49.943929+0000","last_became_active":"2024-06-11T13:28:15.599285+0000","last_became_peered":"2024-06-11T13:28:15.599285+0000","last_unstale":"2024-06-11T13:28:49.943929+0000","last_undegraded":"2024-06-11T13:28:49.943929+0000","last_fullsized":"2024-06-11T13:28:49.943929+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T17:31:18.047138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"68'8","reported_seq":50,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:29:33.244575+0000","last_change":"2024-06-11T13:28:17.458379+0000","last_active":"2024-06-11T13:29:33.244575+0000","last_peered":"2024-06-11T13:29:33.244575+0000","last_clean":"2024-06-11T13:29:33.244575+0000","last_became_active":"2024-06-11T13:28:17.458051+0000","last_became_peered":"2024-06-11T13:28:17.458051+0000","last_unstale":"2024-06-11T13:29:33.244575+0000","last_undegraded":"2024-06-11T13:29:33.244575+0000","last_fullsized":"2024-06-11T13:29:33.244575+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T16:16:25.577731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.039540+0000","last_change":"2024-06-11T13:28:19.456367+0000","last_active":"2024-06-11T13:28:50.039540+0000","last_peered":"2024-06-11T13:28:50.039540+0000","last_clean":"2024-06-11T13:28:50.039540+0000","last_became_active":"2024-06-11T13:28:19.456042+0000","last_became_peered":"2024-06-11T13:28:19.456042+0000","last_unstale":"2024-06-11T13:28:50.039540+0000","last_undegraded":"2024-06-11T13:28:50.039540+0000","last_fullsized":"2024-06-11T13:28:50.039540+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T18:35:49.905789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"61'1","reported_seq":30,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.031626+0000","last_change":"2024-06-11T13:28:13.178724+0000","last_active":"2024-06-11T13:28:50.031626+0000","last_peered":"2024-06-11T13:28:50.031626+0000","last_clean":"2024-06-11T13:28:50.031626+0000","last_became_active":"2024-06-11T13:28:13.178441+0000","last_became_peered":"2024-06-11T13:28:13.178441+0000","last_unstale":"2024-06-11T13:28:50.031626+0000","last_undegraded":"2024-06-11T13:28:50.031626+0000","last_fullsized":"2024-06-11T13:28:50.031626+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T16:44:02.621263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"68'11","reported_seq":44,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.549090+0000","last_change":"2024-06-11T13:28:15.427408+0000","last_active":"2024-06-11T13:28:50.549090+0000","last_peered":"2024-06-11T13:28:50.549090+0000","last_clean":"2024-06-11T13:28:50.549090+0000","last_became_active":"2024-06-11T13:28:15.427112+0000","last_became_peered":"2024-06-11T13:28:15.427112+0000","last_unstale":"2024-06-11T13:28:50.549090+0000","last_undegraded":"2024-06-11T13:28:50.549090+0000","last_fullsized":"2024-06-11T13:28:50.549090+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T22:17:47.146167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.943834+0000","last_change":"2024-06-11T13:28:17.457781+0000","last_active":"2024-06-11T13:28:49.943834+0000","last_peered":"2024-06-11T13:28:49.943834+0000","last_clean":"2024-06-11T13:28:49.943834+0000","last_became_active":"2024-06-11T13:28:17.457361+0000","last_became_peered":"2024-06-11T13:28:17.457361+0000","last_unstale":"2024-06-11T13:28:49.943834+0000","last_undegraded":"2024-06-11T13:28:49.943834+0000","last_fullsized":"2024-06-11T13:28:49.943834+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-13T00:45:57.100880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.454644+0000","last_change":"2024-06-11T13:28:19.548337+0000","last_active":"2024-06-11T13:28:51.454644+0000","last_peered":"2024-06-11T13:28:51.454644+0000","last_clean":"2024-06-11T13:28:51.454644+0000","last_became_active":"2024-06-11T13:28:19.548053+0000","last_became_peered":"2024-06-11T13:28:19.548053+0000","last_unstale":"2024-06-11T13:28:51.454644+0000","last_undegraded":"2024-06-11T13:28:51.454644+0000","last_fullsized":"2024-06-11T13:28:51.454644+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-13T01:10:51.290117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.549224+0000","last_change":"2024-06-11T13:28:13.185703+0000","last_active":"2024-06-11T13:28:50.549224+0000","last_peered":"2024-06-11T13:28:50.549224+0000","last_clean":"2024-06-11T13:28:50.549224+0000","last_became_active":"2024-06-11T13:28:13.185430+0000","last_became_peered":"2024-06-11T13:28:13.185430+0000","last_unstale":"2024-06-11T13:28:50.549224+0000","last_undegraded":"2024-06-11T13:28:50.549224+0000","last_fullsized":"2024-06-11T13:28:50.549224+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T21:35:22.667919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"68'15","reported_seq":50,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:49.946717+0000","last_change":"2024-06-11T13:28:15.420442+0000","last_active":"2024-06-11T13:28:49.946717+0000","last_peered":"2024-06-11T13:28:49.946717+0000","last_clean":"2024-06-11T13:28:49.946717+0000","last_became_active":"2024-06-11T13:28:15.419791+0000","last_became_peered":"2024-06-11T13:28:15.419791+0000","last_unstale":"2024-06-11T13:28:49.946717+0000","last_undegraded":"2024-06-11T13:28:49.946717+0000","last_fullsized":"2024-06-11T13:28:49.946717+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-13T01:26:10.838592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.039796+0000","last_change":"2024-06-11T13:28:17.458660+0000","last_active":"2024-06-11T13:28:50.039796+0000","last_peered":"2024-06-11T13:28:50.039796+0000","last_clean":"2024-06-11T13:28:50.039796+0000","last_became_active":"2024-06-11T13:28:17.457745+0000","last_became_peered":"2024-06-11T13:28:17.457745+0000","last_unstale":"2024-06-11T13:28:50.039796+0000","last_undegraded":"2024-06-11T13:28:50.039796+0000","last_fullsized":"2024-06-11T13:28:50.039796+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T14:29:05.352069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.033338+0000","last_change":"2024-06-11T13:28:19.550139+0000","last_active":"2024-06-11T13:28:50.033338+0000","last_peered":"2024-06-11T13:28:50.033338+0000","last_clean":"2024-06-11T13:28:50.033338+0000","last_became_active":"2024-06-11T13:28:19.549738+0000","last_became_peered":"2024-06-11T13:28:19.549738+0000","last_unstale":"2024-06-11T13:28:50.033338+0000","last_undegraded":"2024-06-11T13:28:50.033338+0000","last_fullsized":"2024-06-11T13:28:50.033338+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:18.431451+0000","last_clean_scrub_stamp":"2024-06-11T13:28:18.431451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-13T00:10:52.610842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.032229+0000","last_change":"2024-06-11T13:28:13.185050+0000","last_active":"2024-06-11T13:28:50.032229+0000","last_peered":"2024-06-11T13:28:50.032229+0000","last_clean":"2024-06-11T13:28:50.032229+0000","last_became_active":"2024-06-11T13:28:13.184692+0000","last_became_peered":"2024-06-11T13:28:13.184692+0000","last_unstale":"2024-06-11T13:28:50.032229+0000","last_undegraded":"2024-06-11T13:28:50.032229+0000","last_fullsized":"2024-06-11T13:28:50.032229+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:12.149407+0000","last_clean_scrub_stamp":"2024-06-11T13:28:12.149407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T20:46:46.305342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"68'12","reported_seq":48,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:50.032996+0000","last_change":"2024-06-11T13:28:15.429779+0000","last_active":"2024-06-11T13:28:50.032996+0000","last_peered":"2024-06-11T13:28:50.032996+0000","last_clean":"2024-06-11T13:28:50.032996+0000","last_became_active":"2024-06-11T13:28:15.429361+0000","last_became_peered":"2024-06-11T13:28:15.429361+0000","last_unstale":"2024-06-11T13:28:50.032996+0000","last_undegraded":"2024-06-11T13:28:50.032996+0000","last_fullsized":"2024-06-11T13:28:50.032996+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:14.375661+0000","last_clean_scrub_stamp":"2024-06-11T13:28:14.375661+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T15:30:38.304227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-11T13:28:51.454594+0000","last_change":"2024-06-11T13:28:17.442983+0000","last_active":"2024-06-11T13:28:51.454594+0000","last_peered":"2024-06-11T13:28:51.454594+0000","last_clean":"2024-06-11T13:28:51.454594+0000","last_became_active":"2024-06-11T13:28:17.442719+0000","last_became_peered":"2024-06-11T13:28:17.442719+0000","last_unstale":"2024-06-11T13:28:51.454594+0000","last_undegraded":"2024-06-11T13:28:51.454594+0000","last_fullsized":"2024-06-11T13:28:51.454594+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-11T13:28:16.410343+0000","last_clean_scrub_stamp":"2024-06-11T13:28:16.410343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-12T17:06:13.616524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":77,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":55,"seq":236223201304,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32432,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712720,"statfs":{"total":95995035648,"available":95961825280,"internally_reserved":0,"allocated":1286144,"data_stored":858751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462940,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32420,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712732,"statfs":{"total":95995035648,"available":95961837568,"internally_reserved":0,"allocated":1273856,"data_stored":858087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724576,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31836,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713316,"statfs":{"total":95995035648,"available":95962435584,"internally_reserved":0,"allocated":671744,"data_stored":267597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018917,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31860,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713292,"statfs":{"total":95995035648,"available":95962411008,"internally_reserved":0,"allocated":696320,"data_stored":268790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182442,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31876,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713276,"statfs":{"total":95995035648,"available":95962394624,"internally_reserved":0,"allocated":712704,"data_stored":269141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411374,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27672,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717480,"statfs":{"total":95995035648,"available":95966699520,"internally_reserved":0,"allocated":667648,"data_stored":267775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31896,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713256,"statfs":{"total":95995035648,"available":95962374144,"internally_reserved":0,"allocated":733184,"data_stored":269407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738425,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28284,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716868,"statfs":{"total":95995035648,"available":95966072832,"internally_reserved":0,"allocated":1294336,"data_stored":859453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-11T13:29:43.987 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-11T13:29:43.988 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-06-11T13:29:43.988 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-06-11T13:29:43.988 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d shell --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 -- ceph health --format=json 2024-06-11T13:29:44.013 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:29:43 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:29:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:29:44.259 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/mon.a/config 2024-06-11T13:29:44.631 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:44 smithi057 ceph-mon[40588]: from='client.14748 v1:172.21.15.57:0/2348525814' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-11T13:29:44.632 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:44 smithi057 ceph-mon[29805]: from='client.14748 v1:172.21.15.57:0/2348525814' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-11T13:29:44.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:44 smithi107 ceph-mon[33178]: from='client.14748 v1:172.21.15.57:0/2348525814' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-11T13:29:45.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:45 smithi057 ceph-mon[29805]: pgmap v30: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:29:45.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:45 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:45.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:45 smithi057 ceph-mon[40588]: pgmap v30: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:29:45.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:45 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:45.784 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:45 smithi057 bash[75408]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-06-11T13:29:45.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:45 smithi107 ceph-mon[33178]: pgmap v30: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:29:45.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:45 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:46.283 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:45 smithi057 bash[75408]: Writing manifest to image destination 2024-06-11T13:29:46.562 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 podman[75408]: 2024-06-11 13:29:46.22186905 +0000 UTC m=+3.242099666 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-06-11T13:29:46.562 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 podman[75408]: 2024-06-11 13:29:46.462530789 +0000 UTC m=+3.482761452 container create 2a5082206775b601bfb3bab7fc349896df43985a5c73d6ac31a9730a8b387ac1 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-06-11T13:29:47.124 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 podman[75408]: 2024-06-11 13:29:46.893324006 +0000 UTC m=+3.913554624 container init 2a5082206775b601bfb3bab7fc349896df43985a5c73d6ac31a9730a8b387ac1 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-06-11T13:29:47.124 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 podman[75408]: 2024-06-11 13:29:46.896607599 +0000 UTC m=+3.916838204 container start 2a5082206775b601bfb3bab7fc349896df43985a5c73d6ac31a9730a8b387ac1 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-06-11T13:29:47.124 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.900Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-06-11T13:29:47.124 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.900Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-06-11T13:29:47.124 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.901Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-06-11T13:29:47.124 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.901Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-06-11T13:29:47.124 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.901Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-06-11T13:29:47.125 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.901Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-06-11T13:29:47.125 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.901Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-06-11T13:29:47.125 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.902Z caller=node_exporter.go:117 level=info collector=arp 2024-06-11T13:29:47.125 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.902Z caller=node_exporter.go:117 level=info collector=bcache 2024-06-11T13:29:47.125 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.902Z caller=node_exporter.go:117 level=info collector=bonding 2024-06-11T13:29:47.125 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.902Z caller=node_exporter.go:117 level=info collector=btrfs 2024-06-11T13:29:47.125 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.902Z caller=node_exporter.go:117 level=info collector=conntrack 2024-06-11T13:29:47.125 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.902Z caller=node_exporter.go:117 level=info collector=cpu 2024-06-11T13:29:47.125 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.902Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-06-11T13:29:47.126 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.902Z caller=node_exporter.go:117 level=info collector=diskstats 2024-06-11T13:29:47.126 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.902Z caller=node_exporter.go:117 level=info collector=dmi 2024-06-11T13:29:47.126 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.902Z caller=node_exporter.go:117 level=info collector=edac 2024-06-11T13:29:47.126 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.902Z caller=node_exporter.go:117 level=info collector=entropy 2024-06-11T13:29:47.126 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.902Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-06-11T13:29:47.126 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.902Z caller=node_exporter.go:117 level=info collector=filefd 2024-06-11T13:29:47.126 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.902Z caller=node_exporter.go:117 level=info collector=filesystem 2024-06-11T13:29:47.126 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.902Z caller=node_exporter.go:117 level=info collector=hwmon 2024-06-11T13:29:47.126 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.902Z caller=node_exporter.go:117 level=info collector=infiniband 2024-06-11T13:29:47.126 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.902Z caller=node_exporter.go:117 level=info collector=ipvs 2024-06-11T13:29:47.127 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.902Z caller=node_exporter.go:117 level=info collector=loadavg 2024-06-11T13:29:47.127 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.902Z caller=node_exporter.go:117 level=info collector=mdadm 2024-06-11T13:29:47.127 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.902Z caller=node_exporter.go:117 level=info collector=meminfo 2024-06-11T13:29:47.127 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.902Z caller=node_exporter.go:117 level=info collector=netclass 2024-06-11T13:29:47.127 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.902Z caller=node_exporter.go:117 level=info collector=netdev 2024-06-11T13:29:47.127 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.902Z caller=node_exporter.go:117 level=info collector=netstat 2024-06-11T13:29:47.127 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.902Z caller=node_exporter.go:117 level=info collector=nfs 2024-06-11T13:29:47.127 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.902Z caller=node_exporter.go:117 level=info collector=nfsd 2024-06-11T13:29:47.127 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.902Z caller=node_exporter.go:117 level=info collector=nvme 2024-06-11T13:29:47.127 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.902Z caller=node_exporter.go:117 level=info collector=os 2024-06-11T13:29:47.127 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.902Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-06-11T13:29:47.127 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.902Z caller=node_exporter.go:117 level=info collector=pressure 2024-06-11T13:29:47.127 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.902Z caller=node_exporter.go:117 level=info collector=rapl 2024-06-11T13:29:47.128 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.902Z caller=node_exporter.go:117 level=info collector=schedstat 2024-06-11T13:29:47.128 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.902Z caller=node_exporter.go:117 level=info collector=selinux 2024-06-11T13:29:47.128 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.902Z caller=node_exporter.go:117 level=info collector=sockstat 2024-06-11T13:29:47.128 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.902Z caller=node_exporter.go:117 level=info collector=softnet 2024-06-11T13:29:47.128 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.902Z caller=node_exporter.go:117 level=info collector=stat 2024-06-11T13:29:47.128 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.902Z caller=node_exporter.go:117 level=info collector=tapestats 2024-06-11T13:29:47.128 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.902Z caller=node_exporter.go:117 level=info collector=textfile 2024-06-11T13:29:47.128 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.902Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-06-11T13:29:47.128 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.902Z caller=node_exporter.go:117 level=info collector=time 2024-06-11T13:29:47.128 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.902Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-06-11T13:29:47.128 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.902Z caller=node_exporter.go:117 level=info collector=uname 2024-06-11T13:29:47.128 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.902Z caller=node_exporter.go:117 level=info collector=vmstat 2024-06-11T13:29:47.128 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.902Z caller=node_exporter.go:117 level=info collector=xfs 2024-06-11T13:29:47.128 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.902Z caller=node_exporter.go:117 level=info collector=zfs 2024-06-11T13:29:47.129 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.902Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-06-11T13:29:47.129 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a[75588]: ts=2024-06-11T13:29:46.902Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-06-11T13:29:47.129 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 bash[75408]: 2a5082206775b601bfb3bab7fc349896df43985a5c73d6ac31a9730a8b387ac1 2024-06-11T13:29:47.129 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 13:29:46 smithi057 systemd[1]: Started Ceph node-exporter.a for 98a6635e-27f5-11ef-bc9b-c7b262605968. 2024-06-11T13:29:47.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:47.210+0000 7f0a0e33d640 1 -- 172.21.15.57:0/3565644390 >> v1:172.21.15.57:6790/0 conn(0x7f0a08102a70 legacy=0x7f0a08102e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:47.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:47.210+0000 7f0a0e33d640 1 -- 172.21.15.57:0/3565644390 shutdown_connections 2024-06-11T13:29:47.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:47.210+0000 7f0a0e33d640 1 -- 172.21.15.57:0/3565644390 >> 172.21.15.57:0/3565644390 conn(0x7f0a080fc220 msgr2=0x7f0a080fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:29:47.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:47.210+0000 7f0a0e33d640 1 -- 172.21.15.57:0/3565644390 shutdown_connections 2024-06-11T13:29:47.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:47.210+0000 7f0a0e33d640 1 -- 172.21.15.57:0/3565644390 wait complete. 2024-06-11T13:29:47.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:47.211+0000 7f0a0e33d640 1 Processor -- start 2024-06-11T13:29:47.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:47.211+0000 7f0a0e33d640 1 -- start start 2024-06-11T13:29:47.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:47.212+0000 7f0a0e33d640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f0a081a6820 con 0x7f0a08102a70 2024-06-11T13:29:47.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:47.212+0000 7f0a0e33d640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f0a081a7a60 con 0x7f0a0810a880 2024-06-11T13:29:47.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:47.212+0000 7f0a0e33d640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f0a081a8ca0 con 0x7f0a0810e390 2024-06-11T13:29:47.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:47.212+0000 7f0a07fff640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f0a08102a70 0x7f0a0810a090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:38398/0 (socket says 172.21.15.57:38398) 2024-06-11T13:29:47.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:47.212+0000 7f0a07fff640 1 -- 172.21.15.57:0/3923891073 learned_addr learned my addr 172.21.15.57:0/3923891073 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T13:29:47.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:47.212+0000 7f0a057fa640 1 -- 172.21.15.57:0/3923891073 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2022061930 0 0) 0x7f0a081a8ca0 con 0x7f0a0810e390 2024-06-11T13:29:47.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:47.212+0000 7f0a057fa640 1 -- 172.21.15.57:0/3923891073 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f09f0003160 con 0x7f0a0810e390 2024-06-11T13:29:47.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:47.212+0000 7f0a057fa640 1 -- 172.21.15.57:0/3923891073 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 470790244 0 0) 0x7f0a081a6820 con 0x7f0a08102a70 2024-06-11T13:29:47.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:47.212+0000 7f0a057fa640 1 -- 172.21.15.57:0/3923891073 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a081a8ca0 con 0x7f0a08102a70 2024-06-11T13:29:47.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:47.212+0000 7f0a057fa640 1 -- 172.21.15.57:0/3923891073 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3152715040 0 0) 0x7f0a081a7a60 con 0x7f0a0810a880 2024-06-11T13:29:47.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:47.212+0000 7f0a057fa640 1 -- 172.21.15.57:0/3923891073 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a081a6820 con 0x7f0a0810a880 2024-06-11T13:29:47.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:47.212+0000 7f0a057fa640 1 -- 172.21.15.57:0/3923891073 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 679090467 0 0) 0x7f09f0003160 con 0x7f0a0810e390 2024-06-11T13:29:47.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:47.212+0000 7f0a057fa640 1 -- 172.21.15.57:0/3923891073 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0a081a7a60 con 0x7f0a0810e390 2024-06-11T13:29:47.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:47.212+0000 7f0a057fa640 1 -- 172.21.15.57:0/3923891073 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f0a00001c30 con 0x7f0a0810e390 2024-06-11T13:29:47.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:47.212+0000 7f0a057fa640 1 -- 172.21.15.57:0/3923891073 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3340698218 0 0) 0x7f0a081a8ca0 con 0x7f0a08102a70 2024-06-11T13:29:47.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:47.212+0000 7f0a057fa640 1 -- 172.21.15.57:0/3923891073 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f09f0003160 con 0x7f0a08102a70 2024-06-11T13:29:47.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:47.212+0000 7f0a057fa640 1 -- 172.21.15.57:0/3923891073 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3874174494 0 0) 0x7f0a081a6820 con 0x7f0a0810a880 2024-06-11T13:29:47.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:47.212+0000 7f0a057fa640 1 -- 172.21.15.57:0/3923891073 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0a081a8ca0 con 0x7f0a0810a880 2024-06-11T13:29:47.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:47.212+0000 7f0a057fa640 1 -- 172.21.15.57:0/3923891073 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f09f8004320 con 0x7f0a08102a70 2024-06-11T13:29:47.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:47.212+0000 7f0a057fa640 1 -- 172.21.15.57:0/3923891073 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f09fc001840 con 0x7f0a0810a880 2024-06-11T13:29:47.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:47.212+0000 7f0a057fa640 1 -- 172.21.15.57:0/3923891073 <== mon.2 v1:172.21.15.57:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3535149218 0 0) 0x7f0a081a7a60 con 0x7f0a0810e390 2024-06-11T13:29:47.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:47.212+0000 7f0a057fa640 1 -- 172.21.15.57:0/3923891073 >> v1:172.21.15.107:6789/0 conn(0x7f0a0810a880 legacy=0x7f0a081a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:47.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:47.212+0000 7f0a057fa640 1 -- 172.21.15.57:0/3923891073 >> v1:172.21.15.57:6789/0 conn(0x7f0a08102a70 legacy=0x7f0a0810a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:47.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:47.213+0000 7f0a057fa640 1 -- 172.21.15.57:0/3923891073 --> v1:172.21.15.57:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a081a9ee0 con 0x7f0a0810e390 2024-06-11T13:29:47.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:47.213+0000 7f0a0e33d640 1 -- 172.21.15.57:0/3923891073 --> v1:172.21.15.57:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f0a081a6a50 con 0x7f0a0810e390 2024-06-11T13:29:47.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:47.213+0000 7f0a0e33d640 1 -- 172.21.15.57:0/3923891073 --> v1:172.21.15.57:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f0a081a6f10 con 0x7f0a0810e390 2024-06-11T13:29:47.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:47.213+0000 7f0a057fa640 1 -- 172.21.15.57:0/3923891073 <== mon.2 v1:172.21.15.57:6790/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f0a00003e20 con 0x7f0a0810e390 2024-06-11T13:29:47.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:47.213+0000 7f0a057fa640 1 -- 172.21.15.57:0/3923891073 <== mon.2 v1:172.21.15.57:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f0a00004e00 con 0x7f0a0810e390 2024-06-11T13:29:47.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:47.214+0000 7f0a0e33d640 1 -- 172.21.15.57:0/3923891073 --> v1:172.21.15.57:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09f4005080 con 0x7f0a0810e390 2024-06-11T13:29:47.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:47.214+0000 7f0a057fa640 1 -- 172.21.15.57:0/3923891073 <== mon.2 v1:172.21.15.57:6790/0 7 ==== mgrmap(e 22) ==== 92138+0+0 (unknown 311317106 0 0) 0x7f0a00003990 con 0x7f0a0810e390 2024-06-11T13:29:47.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:47.215+0000 7f0a057fa640 1 -- 172.21.15.57:0/3923891073 <== mon.2 v1:172.21.15.57:6790/0 8 ==== osd_map(70..70 src has 1..70) ==== 6172+0+0 (unknown 3813449920 0 0) 0x7f0a00088560 con 0x7f0a0810e390 2024-06-11T13:29:47.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:47.219+0000 7f0a057fa640 1 -- 172.21.15.57:0/3923891073 <== mon.2 v1:172.21.15.57:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f0a00055d20 con 0x7f0a0810e390 2024-06-11T13:29:47.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:47.447+0000 7f0a0e33d640 1 -- 172.21.15.57:0/3923891073 --> v1:172.21.15.57:6790/0 -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f09f4005370 con 0x7f0a0810e390 2024-06-11T13:29:47.444 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:47 smithi057 ceph-mon[40588]: pgmap v31: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:29:47.445 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:47 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:47.445 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:47 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:47.445 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:47 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:47.445 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:47 smithi057 ceph-mon[29805]: pgmap v31: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:29:47.445 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:47 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:47.445 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:47 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:47.445 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:47 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:47.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:47.448+0000 7f0a057fa640 1 -- 172.21.15.57:0/3923891073 <== mon.2 v1:172.21.15.57:6790/0 10 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (unknown 3730786198 0 4185958460) 0x7f0a000599d0 con 0x7f0a0810e390 2024-06-11T13:29:47.446 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-06-11T13:29:47.446 INFO:teuthology.orchestra.run.smithi057.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-06-11T13:29:47.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:47.450+0000 7f0a0e33d640 1 -- 172.21.15.57:0/3923891073 >> v1:172.21.15.57:6800/2077416415 conn(0x7f09f006edd0 legacy=0x7f09f0071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:47.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:47.450+0000 7f0a0e33d640 1 -- 172.21.15.57:0/3923891073 >> v1:172.21.15.57:6790/0 conn(0x7f0a0810e390 legacy=0x7f0a081a4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T13:29:47.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:47.450+0000 7f0a0e33d640 1 -- 172.21.15.57:0/3923891073 shutdown_connections 2024-06-11T13:29:47.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:47.450+0000 7f0a0e33d640 1 -- 172.21.15.57:0/3923891073 >> 172.21.15.57:0/3923891073 conn(0x7f0a080fc220 msgr2=0x7f0a080fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T13:29:47.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:47.450+0000 7f0a0e33d640 1 -- 172.21.15.57:0/3923891073 shutdown_connections 2024-06-11T13:29:47.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-06-11T13:29:47.450+0000 7f0a0e33d640 1 -- 172.21.15.57:0/3923891073 wait complete. 2024-06-11T13:29:47.622 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:47 smithi107 ceph-mon[33178]: pgmap v31: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:29:47.622 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:47 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:47.622 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:47 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:47.622 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:47 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:47.992 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:47 smithi107 systemd[1]: Starting Ceph node-exporter.b for 98a6635e-27f5-11ef-bc9b-c7b262605968... 2024-06-11T13:29:48.276 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-06-11T13:29:48.277 INFO:tasks.cephadm:Setup complete, yielding 2024-06-11T13:29:48.277 INFO:teuthology.run_tasks:Running task workunit... 2024-06-11T13:29:48.290 INFO:tasks.workunit:Pulling workunits from ref 3f51ea3944d39ca13aa40153c9a4ee298306679d 2024-06-11T13:29:48.290 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-06-11T13:29:48.291 DEBUG:teuthology.orchestra.run.smithi057:> stat -- /home/ubuntu/cephtest/mnt.0 2024-06-11T13:29:48.296 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:47 smithi107 bash[63658]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-06-11T13:29:48.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-11T13:29:48.308 INFO:teuthology.orchestra.run.smithi057.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-06-11T13:29:48.309 DEBUG:teuthology.orchestra.run.smithi057:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-06-11T13:29:48.366 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-06-11T13:29:48.366 DEBUG:teuthology.orchestra.run.smithi057:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-06-11T13:29:48.423 INFO:tasks.workunit:timeout=3h 2024-06-11T13:29:48.423 INFO:tasks.workunit:cleanup=True 2024-06-11T13:29:48.424 DEBUG:teuthology.orchestra.run.smithi057:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 3f51ea3944d39ca13aa40153c9a4ee298306679d 2024-06-11T13:29:48.480 INFO:tasks.workunit.client.0.smithi057.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-06-11T13:29:48.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:48 smithi057 ceph-mon[29805]: Deploying daemon node-exporter.b on smithi107 2024-06-11T13:29:48.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:48 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3923891073' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-11T13:29:48.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:48 smithi057 ceph-mon[40588]: Deploying daemon node-exporter.b on smithi107 2024-06-11T13:29:48.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:48 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3923891073' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-11T13:29:48.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:48 smithi107 ceph-mon[33178]: Deploying daemon node-exporter.b on smithi107 2024-06-11T13:29:48.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:48 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3923891073' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-11T13:29:48.797 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:48 smithi107 bash[63658]: Getting image source signatures 2024-06-11T13:29:48.797 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:48 smithi107 bash[63658]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-06-11T13:29:48.797 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:48 smithi107 bash[63658]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-06-11T13:29:48.797 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:48 smithi107 bash[63658]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-06-11T13:29:49.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:49 smithi057 ceph-mon[29805]: pgmap v32: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:29:49.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:49 smithi057 ceph-mon[40588]: pgmap v32: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:29:49.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:49 smithi107 ceph-mon[33178]: pgmap v32: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:29:50.296 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:49 smithi107 bash[63658]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-06-11T13:29:50.296 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 bash[63658]: Writing manifest to image destination 2024-06-11T13:29:50.619 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 podman[63658]: 2024-06-11 13:29:50.297898174 +0000 UTC m=+2.394387694 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-06-11T13:29:50.619 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 podman[63658]: 2024-06-11 13:29:50.388330192 +0000 UTC m=+2.484819709 container create 2332e771e10eb783875ae65259d13f873a0a998b59d44344bbfd02d8757bcc55 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-06-11T13:29:50.620 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 podman[63658]: 2024-06-11 13:29:50.623967287 +0000 UTC m=+2.720456805 container init 2332e771e10eb783875ae65259d13f873a0a998b59d44344bbfd02d8757bcc55 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-06-11T13:29:50.620 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2024-06-11T13:29:50.620 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2024-06-11T13:29:50.620 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2024-06-11T13:29:50.620 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2024-06-11T13:29:50.620 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2024-06-11T13:29:50.620 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2024-06-11T13:29:50.620 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2024-06-11T13:29:50.620 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2024-06-11T13:29:50.620 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:29:50.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:50 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2024-06-11T13:29:50.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:50 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2024-06-11T13:29:50.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:50 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2024-06-11T13:29:50.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:50 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2024-06-11T13:29:50.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:50 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2024-06-11T13:29:50.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:50 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2024-06-11T13:29:50.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:50 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2024-06-11T13:29:50.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:50 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2024-06-11T13:29:50.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:50 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:29:50.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:50 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2024-06-11T13:29:50.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:50 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2024-06-11T13:29:50.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:50 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2024-06-11T13:29:50.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:50 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2024-06-11T13:29:50.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:50 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2024-06-11T13:29:50.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:50 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2024-06-11T13:29:50.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:50 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2024-06-11T13:29:50.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:50 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2024-06-11T13:29:50.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:50 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:29:51.012 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 podman[63658]: 2024-06-11 13:29:50.626689067 +0000 UTC m=+2.723178591 container start 2332e771e10eb783875ae65259d13f873a0a998b59d44344bbfd02d8757bcc55 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-06-11T13:29:51.013 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.630Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-06-11T13:29:51.013 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.630Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-06-11T13:29:51.013 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.630Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-06-11T13:29:51.013 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.631Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-06-11T13:29:51.013 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.631Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-06-11T13:29:51.013 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.631Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-06-11T13:29:51.013 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.631Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-06-11T13:29:51.014 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.631Z caller=node_exporter.go:117 level=info collector=arp 2024-06-11T13:29:51.014 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.631Z caller=node_exporter.go:117 level=info collector=bcache 2024-06-11T13:29:51.014 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.631Z caller=node_exporter.go:117 level=info collector=bonding 2024-06-11T13:29:51.014 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.631Z caller=node_exporter.go:117 level=info collector=btrfs 2024-06-11T13:29:51.014 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.631Z caller=node_exporter.go:117 level=info collector=conntrack 2024-06-11T13:29:51.014 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.631Z caller=node_exporter.go:117 level=info collector=cpu 2024-06-11T13:29:51.014 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.631Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-06-11T13:29:51.014 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.631Z caller=node_exporter.go:117 level=info collector=diskstats 2024-06-11T13:29:51.014 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.631Z caller=node_exporter.go:117 level=info collector=dmi 2024-06-11T13:29:51.015 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.631Z caller=node_exporter.go:117 level=info collector=edac 2024-06-11T13:29:51.015 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.631Z caller=node_exporter.go:117 level=info collector=entropy 2024-06-11T13:29:51.015 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.631Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-06-11T13:29:51.015 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.631Z caller=node_exporter.go:117 level=info collector=filefd 2024-06-11T13:29:51.015 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.631Z caller=node_exporter.go:117 level=info collector=filesystem 2024-06-11T13:29:51.016 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.631Z caller=node_exporter.go:117 level=info collector=hwmon 2024-06-11T13:29:51.016 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.631Z caller=node_exporter.go:117 level=info collector=infiniband 2024-06-11T13:29:51.016 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.631Z caller=node_exporter.go:117 level=info collector=ipvs 2024-06-11T13:29:51.016 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.631Z caller=node_exporter.go:117 level=info collector=loadavg 2024-06-11T13:29:51.016 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.631Z caller=node_exporter.go:117 level=info collector=mdadm 2024-06-11T13:29:51.016 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.631Z caller=node_exporter.go:117 level=info collector=meminfo 2024-06-11T13:29:51.016 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.631Z caller=node_exporter.go:117 level=info collector=netclass 2024-06-11T13:29:51.016 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.631Z caller=node_exporter.go:117 level=info collector=netdev 2024-06-11T13:29:51.016 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.632Z caller=node_exporter.go:117 level=info collector=netstat 2024-06-11T13:29:51.016 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.632Z caller=node_exporter.go:117 level=info collector=nfs 2024-06-11T13:29:51.016 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.632Z caller=node_exporter.go:117 level=info collector=nfsd 2024-06-11T13:29:51.017 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.632Z caller=node_exporter.go:117 level=info collector=nvme 2024-06-11T13:29:51.017 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.632Z caller=node_exporter.go:117 level=info collector=os 2024-06-11T13:29:51.017 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.632Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-06-11T13:29:51.017 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.632Z caller=node_exporter.go:117 level=info collector=pressure 2024-06-11T13:29:51.017 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.632Z caller=node_exporter.go:117 level=info collector=rapl 2024-06-11T13:29:51.017 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.632Z caller=node_exporter.go:117 level=info collector=schedstat 2024-06-11T13:29:51.017 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.632Z caller=node_exporter.go:117 level=info collector=selinux 2024-06-11T13:29:51.017 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.632Z caller=node_exporter.go:117 level=info collector=sockstat 2024-06-11T13:29:51.017 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.632Z caller=node_exporter.go:117 level=info collector=softnet 2024-06-11T13:29:51.018 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.632Z caller=node_exporter.go:117 level=info collector=stat 2024-06-11T13:29:51.018 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.632Z caller=node_exporter.go:117 level=info collector=tapestats 2024-06-11T13:29:51.018 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.632Z caller=node_exporter.go:117 level=info collector=textfile 2024-06-11T13:29:51.018 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.632Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-06-11T13:29:51.018 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.632Z caller=node_exporter.go:117 level=info collector=time 2024-06-11T13:29:51.018 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.632Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-06-11T13:29:51.018 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.632Z caller=node_exporter.go:117 level=info collector=uname 2024-06-11T13:29:51.018 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.632Z caller=node_exporter.go:117 level=info collector=vmstat 2024-06-11T13:29:51.018 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.632Z caller=node_exporter.go:117 level=info collector=xfs 2024-06-11T13:29:51.018 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.632Z caller=node_exporter.go:117 level=info collector=zfs 2024-06-11T13:29:51.018 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.632Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-06-11T13:29:51.019 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b[63708]: ts=2024-06-11T13:29:50.632Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-06-11T13:29:51.019 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 bash[63658]: 2332e771e10eb783875ae65259d13f873a0a998b59d44344bbfd02d8757bcc55 2024-06-11T13:29:51.019 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 13:29:50 smithi107 systemd[1]: Started Ceph node-exporter.b for 98a6635e-27f5-11ef-bc9b-c7b262605968. 2024-06-11T13:29:51.296 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:29:51 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:29:51.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:51 smithi057 ceph-mon[29805]: pgmap v33: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:29:51.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:51 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]': finished 2024-06-11T13:29:51.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:51 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]': finished 2024-06-11T13:29:51.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:51 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]': finished 2024-06-11T13:29:51.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:51 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]': finished 2024-06-11T13:29:51.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:51 smithi057 ceph-mon[29805]: osdmap e71: 8 total, 8 up, 8 in 2024-06-11T13:29:51.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:51 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:51.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:51 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:51.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:51 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:51.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:51 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:51.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:51 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:51.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:51 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:29:51.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:51 smithi057 ceph-mon[40588]: pgmap v33: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:29:51.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:51 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]': finished 2024-06-11T13:29:51.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:51 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]': finished 2024-06-11T13:29:51.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:51 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]': finished 2024-06-11T13:29:51.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:51 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]': finished 2024-06-11T13:29:51.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:51 smithi057 ceph-mon[40588]: osdmap e71: 8 total, 8 up, 8 in 2024-06-11T13:29:51.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:51 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:51.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:51 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:51.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:51 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:51.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:51 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:51.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:51 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:51.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:51 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:29:51.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:51 smithi107 ceph-mon[33178]: pgmap v33: 132 pgs: 132 active+clean; 583 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:29:51.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:51 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]': finished 2024-06-11T13:29:51.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:51 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]': finished 2024-06-11T13:29:51.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:51 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]': finished 2024-06-11T13:29:51.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:51 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]': finished 2024-06-11T13:29:51.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:51 smithi107 ceph-mon[33178]: osdmap e71: 8 total, 8 up, 8 in 2024-06-11T13:29:51.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:51 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:51.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:51 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:51.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:51 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:51.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:51 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:51.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:51 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:51.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:51 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:29:52.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:52 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:29:52.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:52 smithi107 ceph-mon[33178]: osdmap e72: 8 total, 8 up, 8 in 2024-06-11T13:29:52.688 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:52 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:29:52.689 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:52 smithi057 ceph-mon[29805]: osdmap e72: 8 total, 8 up, 8 in 2024-06-11T13:29:52.689 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:52 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:29:52.689 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:52 smithi057 ceph-mon[40588]: osdmap e72: 8 total, 8 up, 8 in 2024-06-11T13:29:53.732 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:53 smithi057 ceph-mon[29805]: pgmap v36: 132 pgs: 4 peering, 128 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:29:53.732 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:53 smithi057 ceph-mon[29805]: Health check failed: Reduced data availability: 1 pg inactive, 3 pgs peering (PG_AVAILABILITY) 2024-06-11T13:29:53.732 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:53 smithi057 ceph-mon[40588]: pgmap v36: 132 pgs: 4 peering, 128 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:29:53.732 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:53 smithi057 ceph-mon[40588]: Health check failed: Reduced data availability: 1 pg inactive, 3 pgs peering (PG_AVAILABILITY) 2024-06-11T13:29:53.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:53 smithi107 ceph-mon[33178]: pgmap v36: 132 pgs: 4 peering, 128 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:29:53.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:53 smithi107 ceph-mon[33178]: Health check failed: Reduced data availability: 1 pg inactive, 3 pgs peering (PG_AVAILABILITY) 2024-06-11T13:29:54.033 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:29:53 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:29:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:29:54.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:54 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:54.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:54 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:55.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:54 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:55.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:54 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:55.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:54 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:55.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:54 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:55.867 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:55 smithi057 ceph-mon[40588]: pgmap v37: 132 pgs: 4 peering, 128 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-06-11T13:29:55.868 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:55 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:55.868 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:55 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:55.868 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:55 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:55.868 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:55 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:29:55.868 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:55 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:29:55.869 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:55 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:55.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:55 smithi057 ceph-mon[29805]: pgmap v37: 132 pgs: 4 peering, 128 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-06-11T13:29:55.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:55 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:55.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:55 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:55.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:55 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:55.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:55 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:29:55.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:55 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:29:55.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:55 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:56.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:55 smithi107 ceph-mon[33178]: pgmap v37: 132 pgs: 4 peering, 128 active+clean; 583 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-06-11T13:29:56.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:55 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:56.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:55 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:56.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:55 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:56.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:55 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:29:56.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:55 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:29:56.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:55 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:29:56.917 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:56 smithi057 ceph-mon[40588]: Reconfiguring alertmanager.a (dependencies changed)... 2024-06-11T13:29:56.917 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:56 smithi057 ceph-mon[40588]: Reconfiguring daemon alertmanager.a on smithi057 2024-06-11T13:29:56.918 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:56 smithi057 ceph-mon[29805]: Reconfiguring alertmanager.a (dependencies changed)... 2024-06-11T13:29:56.918 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:56 smithi057 ceph-mon[29805]: Reconfiguring daemon alertmanager.a on smithi057 2024-06-11T13:29:57.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:56 smithi107 ceph-mon[33178]: Reconfiguring alertmanager.a (dependencies changed)... 2024-06-11T13:29:57.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:56 smithi107 ceph-mon[33178]: Reconfiguring daemon alertmanager.a on smithi057 2024-06-11T13:29:58.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:57 smithi057 ceph-mon[29805]: pgmap v38: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 85 B/s, 2 objects/s recovering 2024-06-11T13:29:58.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:57 smithi057 ceph-mon[29805]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 3 pgs peering) 2024-06-11T13:29:58.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:57 smithi057 ceph-mon[29805]: Cluster is now healthy 2024-06-11T13:29:58.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:57 smithi057 ceph-mon[40588]: pgmap v38: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 85 B/s, 2 objects/s recovering 2024-06-11T13:29:58.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:57 smithi057 ceph-mon[40588]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 3 pgs peering) 2024-06-11T13:29:58.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:57 smithi057 ceph-mon[40588]: Cluster is now healthy 2024-06-11T13:29:58.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:57 smithi107 ceph-mon[33178]: pgmap v38: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 85 B/s, 2 objects/s recovering 2024-06-11T13:29:58.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:57 smithi107 ceph-mon[33178]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 3 pgs peering) 2024-06-11T13:29:58.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:57 smithi107 ceph-mon[33178]: Cluster is now healthy 2024-06-11T13:29:58.783 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 13:29:58 smithi057 systemd[1]: Stopping Ceph alertmanager.a for 98a6635e-27f5-11ef-bc9b-c7b262605968... 2024-06-11T13:29:59.176 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 13:29:58 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-alertmanager-a[74565]: ts=2024-06-11T13:29:58.879Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-06-11T13:29:59.177 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 13:29:58 smithi057 podman[76405]: 2024-06-11 13:29:58.900083371 +0000 UTC m=+0.118389050 container died 86927cc63d428c7777ba9b29d132b989e2740c976814c98ed7d5b4c06dc30a43 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-11T13:29:59.533 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 13:29:59 smithi057 podman[76405]: 2024-06-11 13:29:59.371181255 +0000 UTC m=+0.589486994 container cleanup 86927cc63d428c7777ba9b29d132b989e2740c976814c98ed7d5b4c06dc30a43 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-11T13:29:59.534 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 13:29:59 smithi057 bash[76405]: ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-alertmanager-a 2024-06-11T13:29:59.937 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:29:59 smithi057 ceph-mon[29805]: pgmap v39: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 85 B/s, 2 objects/s recovering 2024-06-11T13:29:59.937 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:29:59 smithi057 ceph-mon[40588]: pgmap v39: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 85 B/s, 2 objects/s recovering 2024-06-11T13:29:59.937 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 13:29:59 smithi057 podman[76417]: 2024-06-11 13:29:59.739423379 +0000 UTC m=+0.834641716 container remove 86927cc63d428c7777ba9b29d132b989e2740c976814c98ed7d5b4c06dc30a43 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-11T13:29:59.937 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 13:29:59 smithi057 podman[76417]: 2024-06-11 13:29:59.823423313 +0000 UTC m=+0.918641634 volume remove 424b7d248055bfb7c82fb5ee2c32e0eefd512d0582cc2690a7d86d07a14c02a5 2024-06-11T13:30:00.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:29:59 smithi107 ceph-mon[33178]: pgmap v39: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 85 B/s, 2 objects/s recovering 2024-06-11T13:30:00.270 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 13:29:59 smithi057 systemd[1]: ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-06-11T13:30:00.271 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 13:29:59 smithi057 systemd[1]: Stopped Ceph alertmanager.a for 98a6635e-27f5-11ef-bc9b-c7b262605968. 2024-06-11T13:30:00.271 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 13:29:59 smithi057 systemd[1]: Starting Ceph alertmanager.a for 98a6635e-27f5-11ef-bc9b-c7b262605968... 2024-06-11T13:30:00.271 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 13:30:00 smithi057 podman[76508]: 2024-06-11 13:30:00.077602987 +0000 UTC m=+0.017932631 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-06-11T13:30:00.271 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 13:30:00 smithi057 podman[76508]: 2024-06-11 13:30:00.191212529 +0000 UTC m=+0.131542160 volume create 2141269c7cae31e8035729c3ce29d08fa3df011245d62474fc2bfc7a7527adaa 2024-06-11T13:30:00.271 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 13:30:00 smithi057 podman[76508]: 2024-06-11 13:30:00.274968303 +0000 UTC m=+0.215297993 container create 2e3bcf3ce9613d04cade40d87295ce529e2695669e59764316820f159a119b43 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-11T13:30:01.023 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:00 smithi107 ceph-mon[33178]: overall HEALTH_OK 2024-06-11T13:30:01.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:00 smithi057 ceph-mon[29805]: overall HEALTH_OK 2024-06-11T13:30:01.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:00 smithi057 ceph-mon[40588]: overall HEALTH_OK 2024-06-11T13:30:01.033 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 13:30:00 smithi057 podman[76508]: 2024-06-11 13:30:00.549506827 +0000 UTC m=+0.489836466 container init 2e3bcf3ce9613d04cade40d87295ce529e2695669e59764316820f159a119b43 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-11T13:30:01.034 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 13:30:00 smithi057 podman[76508]: 2024-06-11 13:30:00.552522985 +0000 UTC m=+0.492852622 container start 2e3bcf3ce9613d04cade40d87295ce529e2695669e59764316820f159a119b43 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-11T13:30:01.034 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 13:30:00 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-alertmanager-a[76524]: ts=2024-06-11T13:30:00.566Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-06-11T13:30:01.034 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 13:30:00 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-alertmanager-a[76524]: ts=2024-06-11T13:30:00.566Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-06-11T13:30:01.034 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 13:30:00 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-alertmanager-a[76524]: ts=2024-06-11T13:30:00.567Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.57 port=9094 2024-06-11T13:30:01.034 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 13:30:00 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-alertmanager-a[76524]: ts=2024-06-11T13:30:00.568Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-06-11T13:30:01.034 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 13:30:00 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-alertmanager-a[76524]: ts=2024-06-11T13:30:00.592Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-06-11T13:30:01.034 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 13:30:00 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-alertmanager-a[76524]: ts=2024-06-11T13:30:00.592Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-06-11T13:30:01.034 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 13:30:00 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-alertmanager-a[76524]: ts=2024-06-11T13:30:00.595Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-06-11T13:30:01.035 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 13:30:00 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-alertmanager-a[76524]: ts=2024-06-11T13:30:00.595Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-06-11T13:30:01.035 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 13:30:00 smithi057 bash[76508]: 2e3bcf3ce9613d04cade40d87295ce529e2695669e59764316820f159a119b43 2024-06-11T13:30:01.035 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 13:30:00 smithi057 systemd[1]: Started Ceph alertmanager.a for 98a6635e-27f5-11ef-bc9b-c7b262605968. 2024-06-11T13:30:01.297 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:30:01 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:30:02.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:01 smithi057 ceph-mon[29805]: pgmap v40: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 71 B/s, 2 objects/s recovering 2024-06-11T13:30:02.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:01 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:30:02.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:01 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:30:02.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:01 smithi057 ceph-mon[29805]: Reconfiguring prometheus.a (dependencies changed)... 2024-06-11T13:30:02.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:01 smithi057 ceph-mon[40588]: pgmap v40: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 71 B/s, 2 objects/s recovering 2024-06-11T13:30:02.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:01 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:30:02.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:01 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:30:02.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:01 smithi057 ceph-mon[40588]: Reconfiguring prometheus.a (dependencies changed)... 2024-06-11T13:30:02.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:01 smithi107 ceph-mon[33178]: pgmap v40: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 71 B/s, 2 objects/s recovering 2024-06-11T13:30:02.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:01 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:30:02.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:01 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:30:02.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:01 smithi107 ceph-mon[33178]: Reconfiguring prometheus.a (dependencies changed)... 2024-06-11T13:30:03.012 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:02 smithi107 ceph-mon[33178]: Reconfiguring daemon prometheus.a on smithi107 2024-06-11T13:30:03.013 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:02 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:30:03.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:02 smithi057 ceph-mon[29805]: Reconfiguring daemon prometheus.a on smithi107 2024-06-11T13:30:03.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:02 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:30:03.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:02 smithi057 ceph-mon[40588]: Reconfiguring daemon prometheus.a on smithi107 2024-06-11T13:30:03.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:02 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:30:03.034 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 13:30:02 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-alertmanager-a[76524]: ts=2024-06-11T13:30:02.569Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000844942s 2024-06-11T13:30:03.874 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:03 smithi107 ceph-mon[33178]: pgmap v41: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 969 B/s rd, 0 op/s; 65 B/s, 2 objects/s recovering 2024-06-11T13:30:04.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:03 smithi057 ceph-mon[29805]: pgmap v41: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 969 B/s rd, 0 op/s; 65 B/s, 2 objects/s recovering 2024-06-11T13:30:04.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:03 smithi057 ceph-mon[40588]: pgmap v41: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 969 B/s rd, 0 op/s; 65 B/s, 2 objects/s recovering 2024-06-11T13:30:04.033 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:30:03 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:30:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:30:04.259 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:04 smithi107 systemd[1]: Stopping Ceph prometheus.a for 98a6635e-27f5-11ef-bc9b-c7b262605968... 2024-06-11T13:30:04.543 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:04 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[61394]: ts=2024-06-11T13:30:04.263Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-06-11T13:30:04.544 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:04 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[61394]: ts=2024-06-11T13:30:04.263Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-06-11T13:30:04.544 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:04 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[61394]: ts=2024-06-11T13:30:04.263Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-06-11T13:30:04.544 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:04 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[61394]: ts=2024-06-11T13:30:04.263Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-06-11T13:30:04.544 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:04 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[61394]: ts=2024-06-11T13:30:04.263Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-06-11T13:30:04.544 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:04 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[61394]: ts=2024-06-11T13:30:04.263Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-06-11T13:30:04.545 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:04 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[61394]: ts=2024-06-11T13:30:04.263Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-06-11T13:30:04.545 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:04 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[61394]: ts=2024-06-11T13:30:04.263Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-06-11T13:30:04.545 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:04 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[61394]: ts=2024-06-11T13:30:04.263Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-06-11T13:30:04.545 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:04 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[61394]: ts=2024-06-11T13:30:04.263Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-06-11T13:30:04.546 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:04 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[61394]: ts=2024-06-11T13:30:04.263Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-06-11T13:30:04.546 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:04 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[61394]: ts=2024-06-11T13:30:04.263Z caller=main.go:1147 level=info msg="See you next time!" 2024-06-11T13:30:04.546 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:04 smithi107 podman[64616]: 2024-06-11 13:30:04.273704925 +0000 UTC m=+0.118585438 container died 7dc6bb13301019745064a621f3c9d4a41d60fb7734d55e245bea28bbbcad91b7 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-11T13:30:04.796 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:04 smithi107 podman[64616]: 2024-06-11 13:30:04.646477716 +0000 UTC m=+0.491358219 container cleanup 7dc6bb13301019745064a621f3c9d4a41d60fb7734d55e245bea28bbbcad91b7 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-11T13:30:04.796 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:04 smithi107 bash[64616]: ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a 2024-06-11T13:30:05.282 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:04 smithi107 podman[64628]: 2024-06-11 13:30:04.906132503 +0000 UTC m=+0.634409859 container remove 7dc6bb13301019745064a621f3c9d4a41d60fb7734d55e245bea28bbbcad91b7 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-11T13:30:05.282 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:05 smithi107 systemd[1]: ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@prometheus.a.service: Deactivated successfully. 2024-06-11T13:30:05.282 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:05 smithi107 systemd[1]: Stopped Ceph prometheus.a for 98a6635e-27f5-11ef-bc9b-c7b262605968. 2024-06-11T13:30:05.282 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:05 smithi107 systemd[1]: ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@prometheus.a.service: Consumed 1.089s CPU time. 2024-06-11T13:30:05.282 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:05 smithi107 systemd[1]: Starting Ceph prometheus.a for 98a6635e-27f5-11ef-bc9b-c7b262605968... 2024-06-11T13:30:05.540 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:05 smithi107 podman[64721]: 2024-06-11 13:30:05.190125322 +0000 UTC m=+0.020198326 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-06-11T13:30:05.541 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:05 smithi107 podman[64721]: 2024-06-11 13:30:05.298991954 +0000 UTC m=+0.129064990 container create 70cb60b78e5536cae7dce2813a4825ca346ea1f725a10897aeafb773a29ddfdf (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-11T13:30:05.541 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:05 smithi107 podman[64721]: 2024-06-11 13:30:05.518388927 +0000 UTC m=+0.348461904 container init 70cb60b78e5536cae7dce2813a4825ca346ea1f725a10897aeafb773a29ddfdf (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-11T13:30:05.541 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:05 smithi107 podman[64721]: 2024-06-11 13:30:05.521348309 +0000 UTC m=+0.351421287 container start 70cb60b78e5536cae7dce2813a4825ca346ea1f725a10897aeafb773a29ddfdf (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-11T13:30:05.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:05 smithi107 ceph-mon[33178]: pgmap v42: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 57 B/s, 1 objects/s recovering 2024-06-11T13:30:05.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:05 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:30:05.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:05 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:30:05.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:05 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:30:05.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:05 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-11T13:30:05.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:05 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi057.front.sepia.ceph.com:9093"}]: dispatch 2024-06-11T13:30:05.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:05 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:30:05.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:05 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-11T13:30:05.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:05 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi107.front.sepia.ceph.com:9095"}]: dispatch 2024-06-11T13:30:05.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:05 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:30:05.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:05 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-11T13:30:05.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:05 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi107.front.sepia.ceph.com:3000"}]: dispatch 2024-06-11T13:30:05.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:05 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:30:05.798 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:05 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[64736]: ts=2024-06-11T13:30:05.545Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-06-11T13:30:05.798 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:05 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[64736]: ts=2024-06-11T13:30:05.545Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-06-11T13:30:05.798 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:05 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[64736]: ts=2024-06-11T13:30:05.545Z caller=main.go:570 level=info host_details="(Linux 5.14.0-452.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024 x86_64 smithi107 (none))" 2024-06-11T13:30:05.798 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:05 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[64736]: ts=2024-06-11T13:30:05.545Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-06-11T13:30:05.798 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:05 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[64736]: ts=2024-06-11T13:30:05.545Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-06-11T13:30:05.799 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:05 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[64736]: ts=2024-06-11T13:30:05.547Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-06-11T13:30:05.799 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:05 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[64736]: ts=2024-06-11T13:30:05.547Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-06-11T13:30:05.799 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:05 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[64736]: ts=2024-06-11T13:30:05.548Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-06-11T13:30:05.799 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:05 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[64736]: ts=2024-06-11T13:30:05.548Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-06-11T13:30:05.799 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:05 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[64736]: ts=2024-06-11T13:30:05.550Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-06-11T13:30:05.799 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:05 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[64736]: ts=2024-06-11T13:30:05.550Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.676µs 2024-06-11T13:30:05.799 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:05 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[64736]: ts=2024-06-11T13:30:05.550Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-06-11T13:30:05.799 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:05 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[64736]: ts=2024-06-11T13:30:05.550Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-06-11T13:30:05.799 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:05 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[64736]: ts=2024-06-11T13:30:05.551Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-06-11T13:30:05.799 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:05 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[64736]: ts=2024-06-11T13:30:05.551Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=21.243µs wal_replay_duration=519.365µs wbl_replay_duration=104ns total_replay_duration=558.409µs 2024-06-11T13:30:05.799 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:05 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[64736]: ts=2024-06-11T13:30:05.551Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-06-11T13:30:05.799 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:05 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[64736]: ts=2024-06-11T13:30:05.551Z caller=main.go:1029 level=info msg="TSDB started" 2024-06-11T13:30:05.799 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:05 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[64736]: ts=2024-06-11T13:30:05.551Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-06-11T13:30:05.800 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:05 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[64736]: ts=2024-06-11T13:30:05.571Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=20.074963ms db_storage=704ns remote_storage=1.022µs web_handler=388ns query_engine=502ns scrape=430.118µs scrape_sd=74.845µs notify=10.044µs notify_sd=11.746µs rules=19.191962ms tracing=5.438µs 2024-06-11T13:30:05.800 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:05 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[64736]: ts=2024-06-11T13:30:05.571Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-06-11T13:30:05.800 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:05 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[64736]: ts=2024-06-11T13:30:05.571Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-06-11T13:30:05.800 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:05 smithi107 bash[64721]: 70cb60b78e5536cae7dce2813a4825ca346ea1f725a10897aeafb773a29ddfdf 2024-06-11T13:30:05.800 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 13:30:05 smithi107 systemd[1]: Started Ceph prometheus.a for 98a6635e-27f5-11ef-bc9b-c7b262605968. 2024-06-11T13:30:05.981 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:05 smithi057 ceph-mon[40588]: pgmap v42: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 57 B/s, 1 objects/s recovering 2024-06-11T13:30:05.981 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:05 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:30:05.981 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:05 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:30:05.981 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:05 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:30:05.981 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:05 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-11T13:30:05.981 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:05 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi057.front.sepia.ceph.com:9093"}]: dispatch 2024-06-11T13:30:05.981 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:05 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:30:05.981 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:05 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-11T13:30:05.981 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:05 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi107.front.sepia.ceph.com:9095"}]: dispatch 2024-06-11T13:30:05.982 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:05 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:30:05.982 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:05 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-11T13:30:05.982 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:05 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi107.front.sepia.ceph.com:3000"}]: dispatch 2024-06-11T13:30:05.982 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:05 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:30:05.982 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:30:05 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: [11/Jun/2024:13:30:05] ENGINE Bus STOPPING 2024-06-11T13:30:05.982 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:30:05 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: [11/Jun/2024:13:30:05] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-06-11T13:30:05.982 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:30:05 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: [11/Jun/2024:13:30:05] ENGINE Bus STOPPED 2024-06-11T13:30:05.982 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:30:05 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: [11/Jun/2024:13:30:05] ENGINE Bus STARTING 2024-06-11T13:30:05.982 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:30:05 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: [11/Jun/2024:13:30:05] ENGINE Serving on http://:::9283 2024-06-11T13:30:05.983 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:30:05 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: [11/Jun/2024:13:30:05] ENGINE Bus STARTED 2024-06-11T13:30:05.983 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:30:05 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: [11/Jun/2024:13:30:05] ENGINE Bus STOPPING 2024-06-11T13:30:05.983 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:30:05 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: [11/Jun/2024:13:30:05] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-06-11T13:30:05.983 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:30:05 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: [11/Jun/2024:13:30:05] ENGINE Bus STOPPED 2024-06-11T13:30:05.983 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:30:05 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: [11/Jun/2024:13:30:05] ENGINE Bus STARTING 2024-06-11T13:30:05.983 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:30:05 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: [11/Jun/2024:13:30:05] ENGINE Serving on http://:::9283 2024-06-11T13:30:05.983 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:30:05 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: [11/Jun/2024:13:30:05] ENGINE Bus STARTED 2024-06-11T13:30:05.983 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:30:05 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: [11/Jun/2024:13:30:05] ENGINE Bus STOPPING 2024-06-11T13:30:05.983 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:05 smithi057 ceph-mon[29805]: pgmap v42: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 57 B/s, 1 objects/s recovering 2024-06-11T13:30:05.983 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:05 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:30:05.983 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:05 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:30:05.983 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:05 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:30:05.983 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:05 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-11T13:30:05.984 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:05 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi057.front.sepia.ceph.com:9093"}]: dispatch 2024-06-11T13:30:05.984 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:05 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:30:05.984 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:05 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-11T13:30:05.984 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:05 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi107.front.sepia.ceph.com:9095"}]: dispatch 2024-06-11T13:30:05.984 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:05 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:30:05.984 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:05 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-11T13:30:05.984 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:05 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi107.front.sepia.ceph.com:3000"}]: dispatch 2024-06-11T13:30:05.984 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:05 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:30:06.729 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:30:06 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: [11/Jun/2024:13:30:06] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-06-11T13:30:06.729 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:30:06 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: [11/Jun/2024:13:30:06] ENGINE Bus STOPPED 2024-06-11T13:30:06.729 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:30:06 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: [11/Jun/2024:13:30:06] ENGINE Bus STARTING 2024-06-11T13:30:06.730 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:30:06 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: [11/Jun/2024:13:30:06] ENGINE Serving on http://:::9283 2024-06-11T13:30:06.730 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:30:06 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: [11/Jun/2024:13:30:06] ENGINE Bus STARTED 2024-06-11T13:30:07.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:06 smithi057 ceph-mon[29805]: from='mon.? v1:172.21.15.57:0/45435016' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-11T13:30:07.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:06 smithi057 ceph-mon[29805]: from='mon.? v1:172.21.15.57:0/45435016' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi057.front.sepia.ceph.com:9093"}]: dispatch 2024-06-11T13:30:07.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:06 smithi057 ceph-mon[29805]: from='mon.? v1:172.21.15.57:0/45435016' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-11T13:30:07.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:06 smithi057 ceph-mon[29805]: from='mon.? v1:172.21.15.57:0/45435016' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi107.front.sepia.ceph.com:9095"}]: dispatch 2024-06-11T13:30:07.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:06 smithi057 ceph-mon[29805]: from='mon.? v1:172.21.15.57:0/45435016' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-11T13:30:07.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:06 smithi057 ceph-mon[29805]: from='mon.? v1:172.21.15.57:0/45435016' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi107.front.sepia.ceph.com:3000"}]: dispatch 2024-06-11T13:30:07.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:06 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:30:07.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:06 smithi057 ceph-mon[40588]: from='mon.? v1:172.21.15.57:0/45435016' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-11T13:30:07.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:06 smithi057 ceph-mon[40588]: from='mon.? v1:172.21.15.57:0/45435016' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi057.front.sepia.ceph.com:9093"}]: dispatch 2024-06-11T13:30:07.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:06 smithi057 ceph-mon[40588]: from='mon.? v1:172.21.15.57:0/45435016' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-11T13:30:07.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:06 smithi057 ceph-mon[40588]: from='mon.? v1:172.21.15.57:0/45435016' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi107.front.sepia.ceph.com:9095"}]: dispatch 2024-06-11T13:30:07.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:06 smithi057 ceph-mon[40588]: from='mon.? v1:172.21.15.57:0/45435016' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-11T13:30:07.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:06 smithi057 ceph-mon[40588]: from='mon.? v1:172.21.15.57:0/45435016' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi107.front.sepia.ceph.com:3000"}]: dispatch 2024-06-11T13:30:07.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:06 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:30:07.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:06 smithi107 ceph-mon[33178]: from='mon.? v1:172.21.15.57:0/45435016' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-11T13:30:07.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:06 smithi107 ceph-mon[33178]: from='mon.? v1:172.21.15.57:0/45435016' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi057.front.sepia.ceph.com:9093"}]: dispatch 2024-06-11T13:30:07.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:06 smithi107 ceph-mon[33178]: from='mon.? v1:172.21.15.57:0/45435016' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-11T13:30:07.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:06 smithi107 ceph-mon[33178]: from='mon.? v1:172.21.15.57:0/45435016' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi107.front.sepia.ceph.com:9095"}]: dispatch 2024-06-11T13:30:07.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:06 smithi107 ceph-mon[33178]: from='mon.? v1:172.21.15.57:0/45435016' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-11T13:30:07.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:06 smithi107 ceph-mon[33178]: from='mon.? v1:172.21.15.57:0/45435016' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi107.front.sepia.ceph.com:3000"}]: dispatch 2024-06-11T13:30:07.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:06 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:30:07.783 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:07 smithi057 ceph-mon[40588]: pgmap v43: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 57 B/s, 1 objects/s recovering 2024-06-11T13:30:07.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:07 smithi057 ceph-mon[29805]: pgmap v43: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 57 B/s, 1 objects/s recovering 2024-06-11T13:30:07.796 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:07 smithi107 ceph-mon[33178]: pgmap v43: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 57 B/s, 1 objects/s recovering 2024-06-11T13:30:10.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:09 smithi057 ceph-mon[29805]: pgmap v44: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:30:10.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:09 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:30:10.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:09 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:30:10.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:09 smithi057 ceph-mon[40588]: pgmap v44: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:30:10.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:09 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:30:10.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:09 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:30:10.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:09 smithi107 ceph-mon[33178]: pgmap v44: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:30:10.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:09 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:30:10.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:09 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:30:11.033 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 13:30:10 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-alertmanager-a[76524]: ts=2024-06-11T13:30:10.572Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.003920278s 2024-06-11T13:30:11.297 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:30:11 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:30:12.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:11 smithi057 ceph-mon[29805]: pgmap v45: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:30:12.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:11 smithi057 ceph-mon[40588]: pgmap v45: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:30:12.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:11 smithi107 ceph-mon[33178]: pgmap v45: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:30:13.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:12 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:30:13.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:12 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:30:13.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:12 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:30:13.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:12 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:30:13.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:12 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:30:13.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:12 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:30:13.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:12 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:30:13.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:12 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:30:13.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:12 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:30:13.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:12 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:30:13.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:12 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:30:13.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:12 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:30:13.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:12 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:30:13.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:12 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:30:13.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:12 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:30:13.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:12 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:30:13.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:12 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:30:13.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:12 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:30:14.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:13 smithi057 ceph-mon[29805]: pgmap v46: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:30:14.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:13 smithi057 ceph-mon[40588]: pgmap v46: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:30:14.034 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:30:13 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:30:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:30:14.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:13 smithi107 ceph-mon[33178]: pgmap v46: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:30:16.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:15 smithi057 ceph-mon[40588]: pgmap v47: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:30:16.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:15 smithi057 ceph-mon[29805]: pgmap v47: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:30:16.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:15 smithi107 ceph-mon[33178]: pgmap v47: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:30:18.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:17 smithi057 ceph-mon[29805]: pgmap v48: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:30:18.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:17 smithi057 ceph-mon[40588]: pgmap v48: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:30:18.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:17 smithi107 ceph-mon[33178]: pgmap v48: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:30:19.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:18 smithi057 ceph-mon[29805]: pgmap v49: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:30:19.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:18 smithi057 ceph-mon[40588]: pgmap v49: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:30:19.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:18 smithi107 ceph-mon[33178]: pgmap v49: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:30:21.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:21 smithi107 ceph-mon[33178]: pgmap v50: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:30:21.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:21 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:30:21.297 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:30:21 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:30:21.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:21 smithi057 ceph-mon[29805]: pgmap v50: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:30:21.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:21 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:30:21.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:21 smithi057 ceph-mon[40588]: pgmap v50: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:30:21.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:21 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:30:22.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:22 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:30:22.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:22 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:30:22.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:22 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:30:23.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:23 smithi057 ceph-mon[29805]: pgmap v51: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:30:23.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:23 smithi057 ceph-mon[40588]: pgmap v51: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:30:23.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:23 smithi107 ceph-mon[33178]: pgmap v51: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:30:24.033 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:30:23 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:30:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:30:25.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:25 smithi057 ceph-mon[29805]: pgmap v52: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:30:25.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:25 smithi057 ceph-mon[40588]: pgmap v52: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:30:25.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:25 smithi107 ceph-mon[33178]: pgmap v52: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:30:27.414 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:27 smithi057 ceph-mon[29805]: pgmap v53: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:30:27.414 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:27 smithi057 ceph-mon[40588]: pgmap v53: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:30:27.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:27 smithi107 ceph-mon[33178]: pgmap v53: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:30:29.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:29 smithi057 ceph-mon[29805]: pgmap v54: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:30:29.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:29 smithi057 ceph-mon[40588]: pgmap v54: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:30:29.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:29 smithi107 ceph-mon[33178]: pgmap v54: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:30:31.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:31 smithi057 ceph-mon[40588]: pgmap v55: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:30:31.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:31 smithi057 ceph-mon[29805]: pgmap v55: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:30:31.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:31 smithi107 ceph-mon[33178]: pgmap v55: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:30:31.546 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:30:31 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:30:32.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:32 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:30:32.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:32 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:30:32.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:32 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:30:33.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:33 smithi057 ceph-mon[40588]: pgmap v56: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:30:33.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:33 smithi057 ceph-mon[29805]: pgmap v56: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:30:33.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:33 smithi107 ceph-mon[33178]: pgmap v56: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:30:34.033 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:30:33 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:30:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:30:35.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:35 smithi057 ceph-mon[40588]: pgmap v57: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:30:35.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:35 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:30:35.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:35 smithi057 ceph-mon[29805]: pgmap v57: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:30:35.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:35 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:30:35.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:35 smithi107 ceph-mon[33178]: pgmap v57: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:30:35.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:35 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:30:37.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:37 smithi057 ceph-mon[40588]: pgmap v58: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:30:37.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:37 smithi057 ceph-mon[29805]: pgmap v58: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:30:37.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:37 smithi107 ceph-mon[33178]: pgmap v58: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:30:39.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:39 smithi057 ceph-mon[29805]: pgmap v59: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:30:39.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:39 smithi057 ceph-mon[40588]: pgmap v59: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:30:39.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:39 smithi107 ceph-mon[33178]: pgmap v59: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:30:41.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:41 smithi057 ceph-mon[29805]: pgmap v60: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:30:41.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:41 smithi057 ceph-mon[40588]: pgmap v60: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:30:41.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:41 smithi107 ceph-mon[33178]: pgmap v60: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:30:41.547 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:30:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:30:42.106 INFO:tasks.workunit.client.0.smithi057.stderr:Updating files: 97% (12217/12576) Updating files: 98% (12325/12576) Updating files: 99% (12451/12576) Updating files: 100% (12576/12576) Updating files: 100% (12576/12576), done. 2024-06-11T13:30:42.420 INFO:tasks.workunit.client.0.smithi057.stderr:Note: switching to '3f51ea3944d39ca13aa40153c9a4ee298306679d'. 2024-06-11T13:30:42.420 INFO:tasks.workunit.client.0.smithi057.stderr: 2024-06-11T13:30:42.420 INFO:tasks.workunit.client.0.smithi057.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-06-11T13:30:42.421 INFO:tasks.workunit.client.0.smithi057.stderr:changes and commit them, and you can discard any commits you make in this 2024-06-11T13:30:42.421 INFO:tasks.workunit.client.0.smithi057.stderr:state without impacting any branches by switching back to a branch. 2024-06-11T13:30:42.421 INFO:tasks.workunit.client.0.smithi057.stderr: 2024-06-11T13:30:42.421 INFO:tasks.workunit.client.0.smithi057.stderr:If you want to create a new branch to retain commits you create, you may 2024-06-11T13:30:42.421 INFO:tasks.workunit.client.0.smithi057.stderr:do so (now or later) by using -c with the switch command. Example: 2024-06-11T13:30:42.421 INFO:tasks.workunit.client.0.smithi057.stderr: 2024-06-11T13:30:42.421 INFO:tasks.workunit.client.0.smithi057.stderr: git switch -c 2024-06-11T13:30:42.421 INFO:tasks.workunit.client.0.smithi057.stderr: 2024-06-11T13:30:42.421 INFO:tasks.workunit.client.0.smithi057.stderr:Or undo this operation with: 2024-06-11T13:30:42.421 INFO:tasks.workunit.client.0.smithi057.stderr: 2024-06-11T13:30:42.421 INFO:tasks.workunit.client.0.smithi057.stderr: git switch - 2024-06-11T13:30:42.421 INFO:tasks.workunit.client.0.smithi057.stderr: 2024-06-11T13:30:42.421 INFO:tasks.workunit.client.0.smithi057.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-06-11T13:30:42.421 INFO:tasks.workunit.client.0.smithi057.stderr: 2024-06-11T13:30:42.422 INFO:tasks.workunit.client.0.smithi057.stderr:HEAD is now at 3f51ea3944d Merge PR #57358 into squid 2024-06-11T13:30:42.425 DEBUG:teuthology.orchestra.run.smithi057:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-06-11T13:30:42.495 INFO:tasks.workunit.client.0.smithi057.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-06-11T13:30:42.498 INFO:tasks.workunit.client.0.smithi057.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-06-11T13:30:42.498 INFO:tasks.workunit.client.0.smithi057.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-06-11T13:30:42.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:42 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:30:42.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:42 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:30:42.546 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:42 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:30:43.236 INFO:tasks.workunit.client.0.smithi057.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-06-11T13:30:43.297 INFO:tasks.workunit.client.0.smithi057.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-06-11T13:30:43.339 INFO:tasks.workunit.client.0.smithi057.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-06-11T13:30:43.341 INFO:tasks.workunit.client.0.smithi057.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-06-11T13:30:43.341 INFO:tasks.workunit.client.0.smithi057.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-06-11T13:30:43.381 INFO:tasks.workunit.client.0.smithi057.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-06-11T13:30:43.385 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-06-11T13:30:43.385 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-06-11T13:30:43.442 INFO:tasks.workunit:Running workunits matching rados/test.sh on client.0... 2024-06-11T13:30:43.443 INFO:tasks.workunit:Running workunit rados/test.sh... 2024-06-11T13:30:43.443 DEBUG:teuthology.orchestra.run.smithi057:workunit test rados/test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3f51ea3944d39ca13aa40153c9a4ee298306679d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh 2024-06-11T13:30:43.511 INFO:tasks.workunit.client.0.smithi057.stderr:+ parallel=1 2024-06-11T13:30:43.511 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' '' = --serial ']' 2024-06-11T13:30:43.511 INFO:tasks.workunit.client.0.smithi057.stderr:+ crimson=0 2024-06-11T13:30:43.511 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' '' = --crimson ']' 2024-06-11T13:30:43.511 INFO:tasks.workunit.client.0.smithi057.stderr:+ color= 2024-06-11T13:30:43.511 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -t 1 ']' 2024-06-11T13:30:43.511 INFO:tasks.workunit.client.0.smithi057.stderr:+ trap cleanup EXIT ERR HUP INT QUIT 2024-06-11T13:30:43.511 INFO:tasks.workunit.client.0.smithi057.stderr:+ GTEST_OUTPUT_DIR=/home/ubuntu/cephtest/archive/unit_test_xml_report 2024-06-11T13:30:43.512 INFO:tasks.workunit.client.0.smithi057.stderr:+ mkdir -p /home/ubuntu/cephtest/archive/unit_test_xml_report 2024-06-11T13:30:43.512 INFO:tasks.workunit.client.0.smithi057.stderr:+ declare -A pids 2024-06-11T13:30:43.513 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-11T13:30:43.513 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' 1 -eq 1 ']' 2024-06-11T13:30:43.513 INFO:tasks.workunit.client.0.smithi057.stderr:++ printf %25s api_aio 2024-06-11T13:30:43.513 INFO:tasks.workunit.client.0.smithi057.stderr:+ r=' api_aio' 2024-06-11T13:30:43.513 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo api_aio 2024-06-11T13:30:43.513 INFO:tasks.workunit.client.0.smithi057.stderr:++ awk '{print $1}' 2024-06-11T13:30:43.514 INFO:tasks.workunit.client.0.smithi057.stderr:+ ff=api_aio 2024-06-11T13:30:43.514 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77275 2024-06-11T13:30:43.515 INFO:tasks.workunit.client.0.smithi057.stdout:test api_aio on pid 77275 2024-06-11T13:30:43.517 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'test api_aio on pid 77275' 2024-06-11T13:30:43.517 INFO:tasks.workunit.client.0.smithi057.stderr:+ pids[$f]=77275 2024-06-11T13:30:43.518 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-11T13:30:43.518 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' 1 -eq 1 ']' 2024-06-11T13:30:43.518 INFO:tasks.workunit.client.0.smithi057.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio.xml 2>&1 | tee ceph_test_rados_api_aio.log | sed "s/^/ api_aio: /"' 2024-06-11T13:30:43.518 INFO:tasks.workunit.client.0.smithi057.stderr:++ printf %25s api_aio_pp 2024-06-11T13:30:43.518 INFO:tasks.workunit.client.0.smithi057.stderr:+ r=' api_aio_pp' 2024-06-11T13:30:43.518 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo api_aio_pp 2024-06-11T13:30:43.518 INFO:tasks.workunit.client.0.smithi057.stderr:++ awk '{print $1}' 2024-06-11T13:30:43.518 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -f /etc/bashrc ']' 2024-06-11T13:30:43.518 INFO:tasks.workunit.client.0.smithi057.stderr:+ . /etc/bashrc 2024-06-11T13:30:43.518 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -z '' ']' 2024-06-11T13:30:43.518 INFO:tasks.workunit.client.0.smithi057.stderr:++ BASHRCSOURCED=Y 2024-06-11T13:30:43.518 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.518 INFO:tasks.workunit.client.0.smithi057.stderr:++ shopt -q login_shell 2024-06-11T13:30:43.519 INFO:tasks.workunit.client.0.smithi057.stderr:+++ umask 2024-06-11T13:30:43.519 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' 0022 -eq 0 ']' 2024-06-11T13:30:43.519 INFO:tasks.workunit.client.0.smithi057.stderr:++ SHELL=/bin/bash 2024-06-11T13:30:43.519 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.519 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-11T13:30:43.519 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.519 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-11T13:30:43.519 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-11T13:30:43.519 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.519 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-11T13:30:43.519 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.519 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-11T13:30:43.519 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.519 INFO:tasks.workunit.client.0.smithi057.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.520 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.520 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-11T13:30:43.520 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.520 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-11T13:30:43.520 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.520 INFO:tasks.workunit.client.0.smithi057.stderr:+ ff=api_aio_pp 2024-06-11T13:30:43.520 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77282 2024-06-11T13:30:43.520 INFO:tasks.workunit.client.0.smithi057.stdout:test api_aio_pp on pid 77282 2024-06-11T13:30:43.524 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'test api_aio_pp on pid 77282' 2024-06-11T13:30:43.524 INFO:tasks.workunit.client.0.smithi057.stderr:+ pids[$f]=77282 2024-06-11T13:30:43.524 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-11T13:30:43.524 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' 1 -eq 1 ']' 2024-06-11T13:30:43.524 INFO:tasks.workunit.client.0.smithi057.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio_pp.xml 2>&1 | tee ceph_test_rados_api_aio_pp.log | sed "s/^/ api_aio_pp: /"' 2024-06-11T13:30:43.524 INFO:tasks.workunit.client.0.smithi057.stderr:++ printf %25s api_io 2024-06-11T13:30:43.524 INFO:tasks.workunit.client.0.smithi057.stderr:+ r=' api_io' 2024-06-11T13:30:43.524 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo api_io 2024-06-11T13:30:43.524 INFO:tasks.workunit.client.0.smithi057.stderr:++ awk '{print $1}' 2024-06-11T13:30:43.524 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -f /etc/bashrc ']' 2024-06-11T13:30:43.524 INFO:tasks.workunit.client.0.smithi057.stderr:+ . /etc/bashrc 2024-06-11T13:30:43.524 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -z '' ']' 2024-06-11T13:30:43.525 INFO:tasks.workunit.client.0.smithi057.stderr:++ BASHRCSOURCED=Y 2024-06-11T13:30:43.525 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.525 INFO:tasks.workunit.client.0.smithi057.stderr:++ shopt -q login_shell 2024-06-11T13:30:43.525 INFO:tasks.workunit.client.0.smithi057.stderr:+++ umask 2024-06-11T13:30:43.525 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' 0022 -eq 0 ']' 2024-06-11T13:30:43.525 INFO:tasks.workunit.client.0.smithi057.stderr:++ SHELL=/bin/bash 2024-06-11T13:30:43.525 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.525 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-11T13:30:43.525 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.525 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-11T13:30:43.525 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-11T13:30:43.525 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.525 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-11T13:30:43.525 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.526 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-11T13:30:43.526 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.526 INFO:tasks.workunit.client.0.smithi057.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.526 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.526 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-11T13:30:43.526 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.526 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-11T13:30:43.526 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.526 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'grep=grep --color=auto' 2024-06-11T13:30:43.526 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-11T13:30:43.526 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-11T13:30:43.526 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.526 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-11T13:30:43.526 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.527 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorls.sh 2024-06-11T13:30:43.527 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' '!' -t 0 ']' 2024-06-11T13:30:43.527 INFO:tasks.workunit.client.0.smithi057.stderr:+++ return 2024-06-11T13:30:43.527 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.527 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-11T13:30:43.527 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.527 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-11T13:30:43.527 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export S_COLORS=auto 2024-06-11T13:30:43.527 INFO:tasks.workunit.client.0.smithi057.stderr:+++ S_COLORS=auto 2024-06-11T13:30:43.527 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.527 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-11T13:30:43.527 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.527 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-11T13:30:43.527 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.527 INFO:tasks.workunit.client.0.smithi057.stderr:+ ff=api_io 2024-06-11T13:30:43.528 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77291 2024-06-11T13:30:43.528 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'test api_io on pid 77291' 2024-06-11T13:30:43.528 INFO:tasks.workunit.client.0.smithi057.stdout:test api_io on pid 77291 2024-06-11T13:30:43.532 INFO:tasks.workunit.client.0.smithi057.stderr:+ pids[$f]=77291 2024-06-11T13:30:43.532 INFO:tasks.workunit.client.0.smithi057.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io.xml 2>&1 | tee ceph_test_rados_api_io.log | sed "s/^/ api_io: /"' 2024-06-11T13:30:43.532 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-11T13:30:43.532 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' 1 -eq 1 ']' 2024-06-11T13:30:43.532 INFO:tasks.workunit.client.0.smithi057.stderr:++ printf %25s api_io_pp 2024-06-11T13:30:43.532 INFO:tasks.workunit.client.0.smithi057.stderr:+ r=' api_io_pp' 2024-06-11T13:30:43.532 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo api_io_pp 2024-06-11T13:30:43.532 INFO:tasks.workunit.client.0.smithi057.stderr:++ awk '{print $1}' 2024-06-11T13:30:43.532 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -f /etc/bashrc ']' 2024-06-11T13:30:43.533 INFO:tasks.workunit.client.0.smithi057.stderr:+ . /etc/bashrc 2024-06-11T13:30:43.533 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -z '' ']' 2024-06-11T13:30:43.533 INFO:tasks.workunit.client.0.smithi057.stderr:++ BASHRCSOURCED=Y 2024-06-11T13:30:43.533 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.533 INFO:tasks.workunit.client.0.smithi057.stderr:++ shopt -q login_shell 2024-06-11T13:30:43.533 INFO:tasks.workunit.client.0.smithi057.stderr:+++ umask 2024-06-11T13:30:43.533 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'grep=grep --color=auto' 2024-06-11T13:30:43.533 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-11T13:30:43.533 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-11T13:30:43.533 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' 0022 -eq 0 ']' 2024-06-11T13:30:43.533 INFO:tasks.workunit.client.0.smithi057.stderr:++ SHELL=/bin/bash 2024-06-11T13:30:43.533 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.533 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-11T13:30:43.533 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.534 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorls.sh 2024-06-11T13:30:43.534 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' '!' -t 0 ']' 2024-06-11T13:30:43.534 INFO:tasks.workunit.client.0.smithi057.stderr:+++ return 2024-06-11T13:30:43.534 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.534 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.534 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-11T13:30:43.534 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-11T13:30:43.534 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.534 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.534 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-11T13:30:43.534 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-11T13:30:43.534 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export S_COLORS=auto 2024-06-11T13:30:43.534 INFO:tasks.workunit.client.0.smithi057.stderr:+++ S_COLORS=auto 2024-06-11T13:30:43.535 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.535 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-11T13:30:43.535 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.535 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-11T13:30:43.535 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.535 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-11T13:30:43.535 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.535 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-11T13:30:43.535 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.535 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-11T13:30:43.535 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.535 INFO:tasks.workunit.client.0.smithi057.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.535 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.535 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-11T13:30:43.535 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.536 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-11T13:30:43.536 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.536 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-11T13:30:43.536 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-11T13:30:43.536 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-11T13:30:43.536 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.536 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-11T13:30:43.536 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.536 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-11T13:30:43.536 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-11T13:30:43.536 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.536 INFO:tasks.workunit.client.0.smithi057.stderr:+ ff=api_io_pp 2024-06-11T13:30:43.536 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77302 2024-06-11T13:30:43.536 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'test api_io_pp on pid 77302' 2024-06-11T13:30:43.537 INFO:tasks.workunit.client.0.smithi057.stdout:test api_io_pp on pid 77302 2024-06-11T13:30:43.541 INFO:tasks.workunit.client.0.smithi057.stderr:+ pids[$f]=77302 2024-06-11T13:30:43.541 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-11T13:30:43.541 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' 1 -eq 1 ']' 2024-06-11T13:30:43.541 INFO:tasks.workunit.client.0.smithi057.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io_pp.xml 2>&1 | tee ceph_test_rados_api_io_pp.log | sed "s/^/ api_io_pp: /"' 2024-06-11T13:30:43.542 INFO:tasks.workunit.client.0.smithi057.stderr:++ printf %25s api_asio 2024-06-11T13:30:43.542 INFO:tasks.workunit.client.0.smithi057.stderr:+ r=' api_asio' 2024-06-11T13:30:43.542 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo api_asio 2024-06-11T13:30:43.542 INFO:tasks.workunit.client.0.smithi057.stderr:++ awk '{print $1}' 2024-06-11T13:30:43.542 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -f /etc/bashrc ']' 2024-06-11T13:30:43.542 INFO:tasks.workunit.client.0.smithi057.stderr:+ . /etc/bashrc 2024-06-11T13:30:43.542 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -z '' ']' 2024-06-11T13:30:43.542 INFO:tasks.workunit.client.0.smithi057.stderr:++ BASHRCSOURCED=Y 2024-06-11T13:30:43.542 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.542 INFO:tasks.workunit.client.0.smithi057.stderr:++ shopt -q login_shell 2024-06-11T13:30:43.542 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-11T13:30:43.542 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-11T13:30:43.542 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-11T13:30:43.543 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'grep=grep --color=auto' 2024-06-11T13:30:43.543 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.543 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-11T13:30:43.543 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.543 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-11T13:30:43.543 INFO:tasks.workunit.client.0.smithi057.stderr:+++ umask 2024-06-11T13:30:43.543 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-11T13:30:43.543 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-11T13:30:43.543 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-11T13:30:43.543 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.543 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.543 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-11T13:30:43.543 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.543 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorls.sh 2024-06-11T13:30:43.544 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' '!' -t 0 ']' 2024-06-11T13:30:43.544 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' 0022 -eq 0 ']' 2024-06-11T13:30:43.544 INFO:tasks.workunit.client.0.smithi057.stderr:+++ return 2024-06-11T13:30:43.544 INFO:tasks.workunit.client.0.smithi057.stderr:++ SHELL=/bin/bash 2024-06-11T13:30:43.544 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.544 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-11T13:30:43.544 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.544 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-11T13:30:43.544 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export S_COLORS=auto 2024-06-11T13:30:43.544 INFO:tasks.workunit.client.0.smithi057.stderr:+++ S_COLORS=auto 2024-06-11T13:30:43.544 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.544 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.544 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-11T13:30:43.544 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-11T13:30:43.545 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.545 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.545 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-11T13:30:43.545 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-11T13:30:43.545 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.545 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-11T13:30:43.545 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.545 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-11T13:30:43.545 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.545 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-11T13:30:43.545 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.545 INFO:tasks.workunit.client.0.smithi057.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.545 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.546 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-11T13:30:43.546 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.546 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-11T13:30:43.546 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.546 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-11T13:30:43.546 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-11T13:30:43.546 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-11T13:30:43.546 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.546 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-11T13:30:43.546 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.546 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-11T13:30:43.546 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-11T13:30:43.546 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.546 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-11T13:30:43.547 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.547 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/gawk.sh 2024-06-11T13:30:43.547 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.547 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-11T13:30:43.547 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.547 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/lang.sh 2024-06-11T13:30:43.547 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup 2024-06-11T13:30:43.547 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/locale 2024-06-11T13:30:43.547 INFO:tasks.workunit.client.0.smithi057.stderr:+ ff=api_asio 2024-06-11T13:30:43.547 INFO:tasks.workunit.client.0.smithi057.stdout:test api_asio on pid 77316 2024-06-11T13:30:43.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:43 smithi107 ceph-mon[33178]: pgmap v61: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:30:43.553 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77316 2024-06-11T13:30:43.553 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'test api_asio on pid 77316' 2024-06-11T13:30:43.553 INFO:tasks.workunit.client.0.smithi057.stderr:+ pids[$f]=77316 2024-06-11T13:30:43.554 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-11T13:30:43.554 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' 1 -eq 1 ']' 2024-06-11T13:30:43.554 INFO:tasks.workunit.client.0.smithi057.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_asio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_asio.xml 2>&1 | tee ceph_test_rados_api_asio.log | sed "s/^/ api_asio: /"' 2024-06-11T13:30:43.554 INFO:tasks.workunit.client.0.smithi057.stderr:++ printf %25s api_list 2024-06-11T13:30:43.554 INFO:tasks.workunit.client.0.smithi057.stderr:+ r=' api_list' 2024-06-11T13:30:43.554 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo api_list 2024-06-11T13:30:43.554 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.554 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-11T13:30:43.554 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.554 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-11T13:30:43.554 INFO:tasks.workunit.client.0.smithi057.stderr:++ awk '{print $1}' 2024-06-11T13:30:43.554 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-11T13:30:43.554 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-11T13:30:43.554 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-11T13:30:43.555 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-11T13:30:43.555 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.555 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-11T13:30:43.555 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.555 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-11T13:30:43.555 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-11T13:30:43.555 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.555 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-11T13:30:43.555 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.555 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/gawk.sh 2024-06-11T13:30:43.555 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -f /etc/bashrc ']' 2024-06-11T13:30:43.555 INFO:tasks.workunit.client.0.smithi057.stderr:+ . /etc/bashrc 2024-06-11T13:30:43.555 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.555 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-11T13:30:43.556 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-11T13:30:43.556 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.556 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/lang.sh 2024-06-11T13:30:43.556 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-11T13:30:43.556 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup 2024-06-11T13:30:43.556 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -z '' ']' 2024-06-11T13:30:43.556 INFO:tasks.workunit.client.0.smithi057.stderr:++ BASHRCSOURCED=Y 2024-06-11T13:30:43.556 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-11T13:30:43.556 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.556 INFO:tasks.workunit.client.0.smithi057.stderr:++ shopt -q login_shell 2024-06-11T13:30:43.556 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-11T13:30:43.556 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.556 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-11T13:30:43.556 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.557 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-11T13:30:43.557 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-11T13:30:43.557 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.557 INFO:tasks.workunit.client.0.smithi057.stderr:+++ umask 2024-06-11T13:30:43.557 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/locale 2024-06-11T13:30:43.557 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' 0022 -eq 0 ']' 2024-06-11T13:30:43.557 INFO:tasks.workunit.client.0.smithi057.stderr:++ SHELL=/bin/bash 2024-06-11T13:30:43.557 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.557 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-11T13:30:43.557 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.557 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-11T13:30:43.557 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'grep=grep --color=auto' 2024-06-11T13:30:43.557 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-11T13:30:43.557 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-11T13:30:43.558 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.558 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-11T13:30:43.558 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.558 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorls.sh 2024-06-11T13:30:43.558 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-11T13:30:43.558 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.558 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-11T13:30:43.558 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' '!' -t 0 ']' 2024-06-11T13:30:43.558 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.558 INFO:tasks.workunit.client.0.smithi057.stderr:+++ return 2024-06-11T13:30:43.558 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-11T13:30:43.558 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.558 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-11T13:30:43.558 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.559 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.559 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-11T13:30:43.559 INFO:tasks.workunit.client.0.smithi057.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.559 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.559 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-11T13:30:43.559 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.559 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export S_COLORS=auto 2024-06-11T13:30:43.559 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-11T13:30:43.559 INFO:tasks.workunit.client.0.smithi057.stderr:+++ S_COLORS=auto 2024-06-11T13:30:43.559 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.559 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-11T13:30:43.559 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.559 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.559 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-11T13:30:43.560 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.560 INFO:tasks.workunit.client.0.smithi057.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-11T13:30:43.560 INFO:tasks.workunit.client.0.smithi057.stderr:++++ export LANG=en_US.UTF-8 2024-06-11T13:30:43.560 INFO:tasks.workunit.client.0.smithi057.stderr:++++ LANG=en_US.UTF-8 2024-06-11T13:30:43.560 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.560 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-11T13:30:43.560 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.560 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG=en_US.UTF-8 2024-06-11T13:30:43.560 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup config 2024-06-11T13:30:43.560 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.560 INFO:tasks.workunit.client.0.smithi057.stderr:+ ff=api_list 2024-06-11T13:30:43.560 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.560 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' dumb = linux ']' 2024-06-11T13:30:43.560 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.560 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-11T13:30:43.561 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.561 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/less.sh 2024-06-11T13:30:43.561 INFO:tasks.workunit.client.0.smithi057.stdout:test api_list on pid 77332 2024-06-11T13:30:43.562 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-11T13:30:43.563 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.563 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-11T13:30:43.563 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.563 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/which2.sh 2024-06-11T13:30:43.563 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77332 2024-06-11T13:30:43.563 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'test api_list on pid 77332' 2024-06-11T13:30:43.563 INFO:tasks.workunit.client.0.smithi057.stderr:+ pids[$f]=77332 2024-06-11T13:30:43.563 INFO:tasks.workunit.client.0.smithi057.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-11T13:30:43.563 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-11T13:30:43.563 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' 1 -eq 1 ']' 2024-06-11T13:30:43.563 INFO:tasks.workunit.client.0.smithi057.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_list --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_list.xml 2>&1 | tee ceph_test_rados_api_list.log | sed "s/^/ api_list: /"' 2024-06-11T13:30:43.563 INFO:tasks.workunit.client.0.smithi057.stderr:++ printf %25s api_lock 2024-06-11T13:30:43.564 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.564 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-11T13:30:43.564 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.564 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-11T13:30:43.564 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-11T13:30:43.564 INFO:tasks.workunit.client.0.smithi057.stderr:+ r=' api_lock' 2024-06-11T13:30:43.564 INFO:tasks.workunit.client.0.smithi057.stderr:+++++ readlink /proc/77275/exe 2024-06-11T13:30:43.564 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-11T13:30:43.564 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-11T13:30:43.564 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-11T13:30:43.564 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-11T13:30:43.564 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.564 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-11T13:30:43.564 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo api_lock 2024-06-11T13:30:43.565 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.565 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-11T13:30:43.565 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-11T13:30:43.565 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.565 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-11T13:30:43.565 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.565 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/gawk.sh 2024-06-11T13:30:43.565 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.565 INFO:tasks.workunit.client.0.smithi057.stderr:++ awk '{print $1}' 2024-06-11T13:30:43.565 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-11T13:30:43.565 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.565 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/lang.sh 2024-06-11T13:30:43.565 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -f /etc/bashrc ']' 2024-06-11T13:30:43.565 INFO:tasks.workunit.client.0.smithi057.stderr:+ . /etc/bashrc 2024-06-11T13:30:43.566 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup 2024-06-11T13:30:43.566 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -z '' ']' 2024-06-11T13:30:43.566 INFO:tasks.workunit.client.0.smithi057.stderr:++ BASHRCSOURCED=Y 2024-06-11T13:30:43.566 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.566 INFO:tasks.workunit.client.0.smithi057.stderr:++ shopt -q login_shell 2024-06-11T13:30:43.566 INFO:tasks.workunit.client.0.smithi057.stderr:++++ basename /usr/bin/bash 2024-06-11T13:30:43.566 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/locale 2024-06-11T13:30:43.566 INFO:tasks.workunit.client.0.smithi057.stderr:+++ umask 2024-06-11T13:30:43.566 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' 0022 -eq 0 ']' 2024-06-11T13:30:43.566 INFO:tasks.workunit.client.0.smithi057.stderr:++ SHELL=/bin/bash 2024-06-11T13:30:43.566 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'grep=grep --color=auto' 2024-06-11T13:30:43.566 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-11T13:30:43.566 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.566 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-11T13:30:43.566 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-11T13:30:43.567 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.567 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-11T13:30:43.567 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.567 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-11T13:30:43.567 INFO:tasks.workunit.client.0.smithi057.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-11T13:30:43.567 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.567 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorls.sh 2024-06-11T13:30:43.567 INFO:tasks.workunit.client.0.smithi057.stderr:++++ export LANG=en_US.UTF-8 2024-06-11T13:30:43.567 INFO:tasks.workunit.client.0.smithi057.stderr:++++ LANG=en_US.UTF-8 2024-06-11T13:30:43.567 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' '!' -t 0 ']' 2024-06-11T13:30:43.567 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.567 INFO:tasks.workunit.client.0.smithi057.stderr:+++ return 2024-06-11T13:30:43.567 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-11T13:30:43.567 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.568 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-11T13:30:43.568 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-11T13:30:43.568 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.568 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.568 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.568 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-11T13:30:43.568 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG=en_US.UTF-8 2024-06-11T13:30:43.568 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-11T13:30:43.568 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.568 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-11T13:30:43.568 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup config 2024-06-11T13:30:43.568 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export S_COLORS=auto 2024-06-11T13:30:43.568 INFO:tasks.workunit.client.0.smithi057.stderr:+++ S_COLORS=auto 2024-06-11T13:30:43.568 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.568 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.569 INFO:tasks.workunit.client.0.smithi057.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.569 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-11T13:30:43.569 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.569 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.569 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-11T13:30:43.569 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-11T13:30:43.569 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.569 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-11T13:30:43.569 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.569 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.569 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:43 smithi057 ceph-mon[29805]: pgmap v61: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:30:43.570 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.570 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-11T13:30:43.571 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-11T13:30:43.571 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-11T13:30:43.571 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.571 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.571 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' dumb = linux ']' 2024-06-11T13:30:43.571 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-11T13:30:43.571 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.571 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.571 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-11T13:30:43.571 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-11T13:30:43.571 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.571 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/less.sh 2024-06-11T13:30:43.572 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:43 smithi057 ceph-mon[40588]: pgmap v61: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:30:43.573 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-11T13:30:43.573 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.573 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-11T13:30:43.573 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.573 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-11T13:30:43.573 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.573 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/which2.sh 2024-06-11T13:30:43.573 INFO:tasks.workunit.client.0.smithi057.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-11T13:30:43.573 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_declare='declare -f' 2024-06-11T13:30:43.573 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_opt=-f 2024-06-11T13:30:43.573 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export which_declare 2024-06-11T13:30:43.573 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export -f which 2024-06-11T13:30:43.573 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset i 2024-06-11T13:30:43.574 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset -f pathmunge 2024-06-11T13:30:43.574 INFO:tasks.workunit.client.0.smithi057.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-11T13:30:43.574 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.574 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-11T13:30:43.574 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.574 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-11T13:30:43.574 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-11T13:30:43.574 INFO:tasks.workunit.client.0.smithi057.stderr:+++++ readlink /proc/77282/exe 2024-06-11T13:30:43.574 INFO:tasks.workunit.client.0.smithi057.stdout:test api_lock on pid 77352 2024-06-11T13:30:43.577 INFO:tasks.workunit.client.0.smithi057.stderr:+ export PATH 2024-06-11T13:30:43.577 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-11T13:30:43.577 INFO:tasks.workunit.client.0.smithi057.stderr:+ unset rc 2024-06-11T13:30:43.577 INFO:tasks.workunit.client.0.smithi057.stderr:+ ff=api_lock 2024-06-11T13:30:43.577 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-11T13:30:43.577 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77352 2024-06-11T13:30:43.577 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'test api_lock on pid 77352' 2024-06-11T13:30:43.577 INFO:tasks.workunit.client.0.smithi057.stderr:+ pids[$f]=77352 2024-06-11T13:30:43.577 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-11T13:30:43.578 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' 1 -eq 1 ']' 2024-06-11T13:30:43.578 INFO:tasks.workunit.client.0.smithi057.stderr:++ printf %25s api_lock_pp 2024-06-11T13:30:43.578 INFO:tasks.workunit.client.0.smithi057.stderr:+ tee ceph_test_rados_api_aio.log 2024-06-11T13:30:43.578 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph_test_rados_api_aio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio.xml 2024-06-11T13:30:43.578 INFO:tasks.workunit.client.0.smithi057.stderr:+ r=' api_lock_pp' 2024-06-11T13:30:43.578 INFO:tasks.workunit.client.0.smithi057.stderr:+ sed 's/^/ api_aio: /' 2024-06-11T13:30:43.578 INFO:tasks.workunit.client.0.smithi057.stderr:++++ basename /usr/bin/bash 2024-06-11T13:30:43.578 INFO:tasks.workunit.client.0.smithi057.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock.xml 2>&1 | tee ceph_test_rados_api_lock.log | sed "s/^/ api_lock: /"' 2024-06-11T13:30:43.578 INFO:tasks.workunit.client.0.smithi057.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-11T13:30:43.578 INFO:tasks.workunit.client.0.smithi057.stderr:++++ export LANG=en_US.UTF-8 2024-06-11T13:30:43.578 INFO:tasks.workunit.client.0.smithi057.stderr:++++ LANG=en_US.UTF-8 2024-06-11T13:30:43.578 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.578 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-11T13:30:43.578 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.579 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG=en_US.UTF-8 2024-06-11T13:30:43.579 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup config 2024-06-11T13:30:43.579 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.579 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_declare='declare -f' 2024-06-11T13:30:43.579 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.579 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_opt=-f 2024-06-11T13:30:43.579 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' dumb = linux ']' 2024-06-11T13:30:43.579 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.579 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export which_declare 2024-06-11T13:30:43.579 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-11T13:30:43.579 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.579 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export -f which 2024-06-11T13:30:43.579 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/less.sh 2024-06-11T13:30:43.580 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset i 2024-06-11T13:30:43.580 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset -f pathmunge 2024-06-11T13:30:43.580 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-11T13:30:43.580 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.580 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-11T13:30:43.580 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.580 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-11T13:30:43.580 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/which2.sh 2024-06-11T13:30:43.580 INFO:tasks.workunit.client.0.smithi057.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-11T13:30:43.580 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-11T13:30:43.580 INFO:tasks.workunit.client.0.smithi057.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-11T13:30:43.580 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-11T13:30:43.580 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.580 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-11T13:30:43.581 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.581 INFO:tasks.workunit.client.0.smithi057.stderr:+ export PATH 2024-06-11T13:30:43.581 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-11T13:30:43.581 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-11T13:30:43.581 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-11T13:30:43.581 INFO:tasks.workunit.client.0.smithi057.stderr:+ unset rc 2024-06-11T13:30:43.581 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.581 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -f /etc/bashrc ']' 2024-06-11T13:30:43.581 INFO:tasks.workunit.client.0.smithi057.stderr:+ . /etc/bashrc 2024-06-11T13:30:43.581 INFO:tasks.workunit.client.0.smithi057.stderr:+++++ readlink /proc/77291/exe 2024-06-11T13:30:43.581 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -z '' ']' 2024-06-11T13:30:43.581 INFO:tasks.workunit.client.0.smithi057.stderr:++ BASHRCSOURCED=Y 2024-06-11T13:30:43.581 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.581 INFO:tasks.workunit.client.0.smithi057.stderr:++ shopt -q login_shell 2024-06-11T13:30:43.582 INFO:tasks.workunit.client.0.smithi057.stderr:+ sed 's/^/ api_aio_pp: /' 2024-06-11T13:30:43.582 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'grep=grep --color=auto' 2024-06-11T13:30:43.582 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-11T13:30:43.582 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-11T13:30:43.582 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.582 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-11T13:30:43.582 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.582 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorls.sh 2024-06-11T13:30:43.582 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' '!' -t 0 ']' 2024-06-11T13:30:43.582 INFO:tasks.workunit.client.0.smithi057.stderr:+++ return 2024-06-11T13:30:43.582 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.582 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-11T13:30:43.582 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.583 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-11T13:30:43.583 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export S_COLORS=auto 2024-06-11T13:30:43.583 INFO:tasks.workunit.client.0.smithi057.stderr:+++ S_COLORS=auto 2024-06-11T13:30:43.583 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.583 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-11T13:30:43.583 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.583 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-11T13:30:43.583 INFO:tasks.workunit.client.0.smithi057.stderr:+++ umask 2024-06-11T13:30:43.583 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.583 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' 0022 -eq 0 ']' 2024-06-11T13:30:43.583 INFO:tasks.workunit.client.0.smithi057.stderr:++ SHELL=/bin/bash 2024-06-11T13:30:43.583 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-11T13:30:43.583 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.583 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-11T13:30:43.583 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-11T13:30:43.584 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.584 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-11T13:30:43.584 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-11T13:30:43.584 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.584 INFO:tasks.workunit.client.0.smithi057.stderr:++++ basename /usr/bin/bash 2024-06-11T13:30:43.584 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-11T13:30:43.584 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.584 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-11T13:30:43.584 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-11T13:30:43.584 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-11T13:30:43.584 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.584 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-11T13:30:43.584 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.584 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.584 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-11T13:30:43.585 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-11T13:30:43.585 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.585 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/gawk.sh 2024-06-11T13:30:43.585 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.585 INFO:tasks.workunit.client.0.smithi057.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.585 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.585 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-11T13:30:43.585 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.585 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-11T13:30:43.585 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.585 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.585 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-11T13:30:43.585 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.585 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/lang.sh 2024-06-11T13:30:43.586 INFO:tasks.workunit.client.0.smithi057.stderr:+ tee ceph_test_rados_api_aio_pp.log 2024-06-11T13:30:43.586 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup 2024-06-11T13:30:43.586 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph_test_rados_api_aio_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio_pp.xml 2024-06-11T13:30:43.586 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo api_lock_pp 2024-06-11T13:30:43.586 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_declare='declare -f' 2024-06-11T13:30:43.586 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_opt=-f 2024-06-11T13:30:43.586 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export which_declare 2024-06-11T13:30:43.586 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export -f which 2024-06-11T13:30:43.586 INFO:tasks.workunit.client.0.smithi057.stderr:++ awk '{print $1}' 2024-06-11T13:30:43.586 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset i 2024-06-11T13:30:43.586 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset -f pathmunge 2024-06-11T13:30:43.586 INFO:tasks.workunit.client.0.smithi057.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-11T13:30:43.586 INFO:tasks.workunit.client.0.smithi057.stderr:+ export PATH 2024-06-11T13:30:43.586 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-11T13:30:43.587 INFO:tasks.workunit.client.0.smithi057.stderr:+ unset rc 2024-06-11T13:30:43.587 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/locale 2024-06-11T13:30:43.587 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-11T13:30:43.587 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-11T13:30:43.587 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-11T13:30:43.587 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.587 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-11T13:30:43.587 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.587 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-11T13:30:43.587 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-11T13:30:43.587 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.587 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-11T13:30:43.587 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.587 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/gawk.sh 2024-06-11T13:30:43.587 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph_test_rados_api_io --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io.xml 2024-06-11T13:30:43.588 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.588 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-11T13:30:43.588 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.588 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/lang.sh 2024-06-11T13:30:43.588 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup 2024-06-11T13:30:43.588 INFO:tasks.workunit.client.0.smithi057.stderr:+ tee ceph_test_rados_api_io.log 2024-06-11T13:30:43.588 INFO:tasks.workunit.client.0.smithi057.stderr:+ sed 's/^/ api_io: /' 2024-06-11T13:30:43.588 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.588 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-11T13:30:43.588 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.588 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-11T13:30:43.588 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-11T13:30:43.588 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-11T13:30:43.588 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/locale 2024-06-11T13:30:43.589 INFO:tasks.workunit.client.0.smithi057.stdout:test api_lock_pp on pid 77382 2024-06-11T13:30:43.592 INFO:tasks.workunit.client.0.smithi057.stderr:+ ff=api_lock_pp 2024-06-11T13:30:43.592 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77382 2024-06-11T13:30:43.592 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'test api_lock_pp on pid 77382' 2024-06-11T13:30:43.592 INFO:tasks.workunit.client.0.smithi057.stderr:+ pids[$f]=77382 2024-06-11T13:30:43.592 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-11T13:30:43.592 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' 1 -eq 1 ']' 2024-06-11T13:30:43.592 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'grep=grep --color=auto' 2024-06-11T13:30:43.592 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-11T13:30:43.592 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-11T13:30:43.592 INFO:tasks.workunit.client.0.smithi057.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock_pp.xml 2>&1 | tee ceph_test_rados_api_lock_pp.log | sed "s/^/ api_lock_pp: /"' 2024-06-11T13:30:43.593 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.593 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-11T13:30:43.593 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.593 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorls.sh 2024-06-11T13:30:43.593 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' '!' -t 0 ']' 2024-06-11T13:30:43.593 INFO:tasks.workunit.client.0.smithi057.stderr:+++ return 2024-06-11T13:30:43.593 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.593 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-11T13:30:43.593 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.593 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-11T13:30:43.593 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export S_COLORS=auto 2024-06-11T13:30:43.593 INFO:tasks.workunit.client.0.smithi057.stderr:+++ S_COLORS=auto 2024-06-11T13:30:43.593 INFO:tasks.workunit.client.0.smithi057.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-11T13:30:43.593 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.594 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-11T13:30:43.594 INFO:tasks.workunit.client.0.smithi057.stderr:++++ export LANG=en_US.UTF-8 2024-06-11T13:30:43.594 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.594 INFO:tasks.workunit.client.0.smithi057.stderr:++++ LANG=en_US.UTF-8 2024-06-11T13:30:43.594 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-11T13:30:43.594 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.594 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.594 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-11T13:30:43.594 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.594 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG=en_US.UTF-8 2024-06-11T13:30:43.594 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup config 2024-06-11T13:30:43.594 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.594 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.594 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' dumb = linux ']' 2024-06-11T13:30:43.595 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.595 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-11T13:30:43.595 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.595 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/less.sh 2024-06-11T13:30:43.595 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-11T13:30:43.595 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.595 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-11T13:30:43.595 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.595 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/which2.sh 2024-06-11T13:30:43.595 INFO:tasks.workunit.client.0.smithi057.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-11T13:30:43.595 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.595 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-11T13:30:43.598 INFO:tasks.workunit.client.0.smithi057.stderr:++ printf %25s api_misc 2024-06-11T13:30:43.598 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.598 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-11T13:30:43.598 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-11T13:30:43.598 INFO:tasks.workunit.client.0.smithi057.stderr:+ r=' api_misc' 2024-06-11T13:30:43.598 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo api_misc 2024-06-11T13:30:43.598 INFO:tasks.workunit.client.0.smithi057.stderr:++ awk '{print $1}' 2024-06-11T13:30:43.598 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-11T13:30:43.598 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-11T13:30:43.598 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-11T13:30:43.598 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-11T13:30:43.598 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.599 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-11T13:30:43.599 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.599 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-11T13:30:43.599 INFO:tasks.workunit.client.0.smithi057.stderr:+++++ readlink /proc/77302/exe 2024-06-11T13:30:43.599 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-11T13:30:43.599 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.599 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -f /etc/bashrc ']' 2024-06-11T13:30:43.599 INFO:tasks.workunit.client.0.smithi057.stderr:+ . /etc/bashrc 2024-06-11T13:30:43.599 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -z '' ']' 2024-06-11T13:30:43.599 INFO:tasks.workunit.client.0.smithi057.stderr:++ BASHRCSOURCED=Y 2024-06-11T13:30:43.599 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.599 INFO:tasks.workunit.client.0.smithi057.stderr:++ shopt -q login_shell 2024-06-11T13:30:43.599 INFO:tasks.workunit.client.0.smithi057.stderr:++++ basename /usr/bin/bash 2024-06-11T13:30:43.599 INFO:tasks.workunit.client.0.smithi057.stderr:+++ umask 2024-06-11T13:30:43.599 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' 0022 -eq 0 ']' 2024-06-11T13:30:43.600 INFO:tasks.workunit.client.0.smithi057.stderr:++ SHELL=/bin/bash 2024-06-11T13:30:43.600 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.600 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-11T13:30:43.600 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.600 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-11T13:30:43.600 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-11T13:30:43.600 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.600 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-11T13:30:43.600 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.600 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-11T13:30:43.600 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.600 INFO:tasks.workunit.client.0.smithi057.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.600 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.600 INFO:tasks.workunit.client.0.smithi057.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-11T13:30:43.601 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-11T13:30:43.601 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.601 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-11T13:30:43.601 INFO:tasks.workunit.client.0.smithi057.stderr:++++ export LANG=en_US.UTF-8 2024-06-11T13:30:43.601 INFO:tasks.workunit.client.0.smithi057.stderr:++++ LANG=en_US.UTF-8 2024-06-11T13:30:43.601 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.601 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.601 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-11T13:30:43.601 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.601 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG=en_US.UTF-8 2024-06-11T13:30:43.601 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup config 2024-06-11T13:30:43.601 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.601 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.601 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' dumb = linux ']' 2024-06-11T13:30:43.601 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.602 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-11T13:30:43.602 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.602 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/less.sh 2024-06-11T13:30:43.602 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-11T13:30:43.602 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.602 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-11T13:30:43.602 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.602 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/which2.sh 2024-06-11T13:30:43.602 INFO:tasks.workunit.client.0.smithi057.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-11T13:30:43.602 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_declare='declare -f' 2024-06-11T13:30:43.602 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_opt=-f 2024-06-11T13:30:43.602 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export which_declare 2024-06-11T13:30:43.602 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export -f which 2024-06-11T13:30:43.602 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset i 2024-06-11T13:30:43.603 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset -f pathmunge 2024-06-11T13:30:43.603 INFO:tasks.workunit.client.0.smithi057.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-11T13:30:43.603 INFO:tasks.workunit.client.0.smithi057.stderr:+ export PATH 2024-06-11T13:30:43.603 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-11T13:30:43.603 INFO:tasks.workunit.client.0.smithi057.stderr:+ unset rc 2024-06-11T13:30:43.603 INFO:tasks.workunit.client.0.smithi057.stdout:test api_misc on pid 77399 2024-06-11T13:30:43.608 INFO:tasks.workunit.client.0.smithi057.stderr:+ ff=api_misc 2024-06-11T13:30:43.608 INFO:tasks.workunit.client.0.smithi057.stderr:+ tee ceph_test_rados_api_io_pp.log 2024-06-11T13:30:43.608 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77399 2024-06-11T13:30:43.608 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'test api_misc on pid 77399' 2024-06-11T13:30:43.608 INFO:tasks.workunit.client.0.smithi057.stderr:+ pids[$f]=77399 2024-06-11T13:30:43.608 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-11T13:30:43.608 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' 1 -eq 1 ']' 2024-06-11T13:30:43.608 INFO:tasks.workunit.client.0.smithi057.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc.xml 2>&1 | tee ceph_test_rados_api_misc.log | sed "s/^/ api_misc: /"' 2024-06-11T13:30:43.608 INFO:tasks.workunit.client.0.smithi057.stderr:++ printf %25s api_misc_pp 2024-06-11T13:30:43.609 INFO:tasks.workunit.client.0.smithi057.stderr:+ r=' api_misc_pp' 2024-06-11T13:30:43.609 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph_test_rados_api_io_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io_pp.xml 2024-06-11T13:30:43.609 INFO:tasks.workunit.client.0.smithi057.stderr:+++++ readlink /proc/77316/exe 2024-06-11T13:30:43.609 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -f /etc/bashrc ']' 2024-06-11T13:30:43.609 INFO:tasks.workunit.client.0.smithi057.stderr:+ . /etc/bashrc 2024-06-11T13:30:43.609 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'grep=grep --color=auto' 2024-06-11T13:30:43.609 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -z '' ']' 2024-06-11T13:30:43.609 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-11T13:30:43.609 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-11T13:30:43.609 INFO:tasks.workunit.client.0.smithi057.stderr:++ BASHRCSOURCED=Y 2024-06-11T13:30:43.609 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.609 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-11T13:30:43.609 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.610 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorls.sh 2024-06-11T13:30:43.610 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.610 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' '!' -t 0 ']' 2024-06-11T13:30:43.610 INFO:tasks.workunit.client.0.smithi057.stderr:+++ return 2024-06-11T13:30:43.610 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.610 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-11T13:30:43.610 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.610 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-11T13:30:43.610 INFO:tasks.workunit.client.0.smithi057.stderr:++ shopt -q login_shell 2024-06-11T13:30:43.610 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export S_COLORS=auto 2024-06-11T13:30:43.610 INFO:tasks.workunit.client.0.smithi057.stderr:+++ S_COLORS=auto 2024-06-11T13:30:43.610 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.610 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-11T13:30:43.610 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.611 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-11T13:30:43.611 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.611 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-11T13:30:43.611 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-11T13:30:43.611 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-11T13:30:43.611 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.611 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-11T13:30:43.611 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.611 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-11T13:30:43.611 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-11T13:30:43.611 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.611 INFO:tasks.workunit.client.0.smithi057.stderr:+++ umask 2024-06-11T13:30:43.611 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-11T13:30:43.611 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-11T13:30:43.612 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-11T13:30:43.612 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.612 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-11T13:30:43.612 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.612 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-11T13:30:43.612 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-11T13:30:43.612 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.612 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-11T13:30:43.612 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.612 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/gawk.sh 2024-06-11T13:30:43.612 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.612 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-11T13:30:43.612 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.612 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/lang.sh 2024-06-11T13:30:43.613 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup 2024-06-11T13:30:43.613 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' 0022 -eq 0 ']' 2024-06-11T13:30:43.613 INFO:tasks.workunit.client.0.smithi057.stderr:++ SHELL=/bin/bash 2024-06-11T13:30:43.613 INFO:tasks.workunit.client.0.smithi057.stderr:++++ basename /usr/bin/bash 2024-06-11T13:30:43.613 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.613 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-11T13:30:43.613 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.613 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-11T13:30:43.613 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-11T13:30:43.613 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.613 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-11T13:30:43.613 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.613 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-11T13:30:43.613 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.613 INFO:tasks.workunit.client.0.smithi057.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.614 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.614 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-11T13:30:43.614 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.614 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-11T13:30:43.614 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.614 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_declare='declare -f' 2024-06-11T13:30:43.614 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_opt=-f 2024-06-11T13:30:43.614 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export which_declare 2024-06-11T13:30:43.614 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export -f which 2024-06-11T13:30:43.614 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset i 2024-06-11T13:30:43.614 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset -f pathmunge 2024-06-11T13:30:43.614 INFO:tasks.workunit.client.0.smithi057.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-11T13:30:43.614 INFO:tasks.workunit.client.0.smithi057.stderr:+ export PATH 2024-06-11T13:30:43.614 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-11T13:30:43.614 INFO:tasks.workunit.client.0.smithi057.stderr:+ unset rc 2024-06-11T13:30:43.615 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph_test_rados_api_asio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_asio.xml 2024-06-11T13:30:43.615 INFO:tasks.workunit.client.0.smithi057.stderr:+ sed 's/^/ api_asio: /' 2024-06-11T13:30:43.615 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/locale 2024-06-11T13:30:43.615 INFO:tasks.workunit.client.0.smithi057.stderr:+ tee ceph_test_rados_api_asio.log 2024-06-11T13:30:43.615 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo api_misc_pp 2024-06-11T13:30:43.615 INFO:tasks.workunit.client.0.smithi057.stderr:++ awk '{print $1}' 2024-06-11T13:30:43.615 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.615 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-11T13:30:43.615 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.615 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-11T13:30:43.615 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-11T13:30:43.616 INFO:tasks.workunit.client.0.smithi057.stdout:test api_misc_pp on pid 77420 2024-06-11T13:30:43.622 INFO:tasks.workunit.client.0.smithi057.stderr:+ ff=api_misc_pp 2024-06-11T13:30:43.623 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77420 2024-06-11T13:30:43.623 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'test api_misc_pp on pid 77420' 2024-06-11T13:30:43.623 INFO:tasks.workunit.client.0.smithi057.stderr:+ pids[$f]=77420 2024-06-11T13:30:43.623 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-11T13:30:43.623 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' 1 -eq 1 ']' 2024-06-11T13:30:43.623 INFO:tasks.workunit.client.0.smithi057.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc_pp.xml 2>&1 | tee ceph_test_rados_api_misc_pp.log | sed "s/^/ api_misc_pp: /"' 2024-06-11T13:30:43.623 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-11T13:30:43.623 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-11T13:30:43.623 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-11T13:30:43.623 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.623 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-11T13:30:43.623 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.624 INFO:tasks.workunit.client.0.smithi057.stderr:+ sed 's/^/ api_io_pp: /' 2024-06-11T13:30:43.624 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-11T13:30:43.624 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-11T13:30:43.624 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.624 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-11T13:30:43.624 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.624 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/gawk.sh 2024-06-11T13:30:43.624 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.624 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-11T13:30:43.624 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.624 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/lang.sh 2024-06-11T13:30:43.624 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup 2024-06-11T13:30:43.625 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/locale 2024-06-11T13:30:43.625 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-11T13:30:43.625 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -f /etc/bashrc ']' 2024-06-11T13:30:43.625 INFO:tasks.workunit.client.0.smithi057.stderr:+ . /etc/bashrc 2024-06-11T13:30:43.625 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -z '' ']' 2024-06-11T13:30:43.625 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-11T13:30:43.625 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-11T13:30:43.625 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-11T13:30:43.625 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.625 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-11T13:30:43.625 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.625 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-11T13:30:43.625 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-11T13:30:43.625 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.626 INFO:tasks.workunit.client.0.smithi057.stderr:++ BASHRCSOURCED=Y 2024-06-11T13:30:43.626 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.626 INFO:tasks.workunit.client.0.smithi057.stderr:++ shopt -q login_shell 2024-06-11T13:30:43.626 INFO:tasks.workunit.client.0.smithi057.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-11T13:30:43.626 INFO:tasks.workunit.client.0.smithi057.stderr:++++ export LANG=en_US.UTF-8 2024-06-11T13:30:43.626 INFO:tasks.workunit.client.0.smithi057.stderr:++++ LANG=en_US.UTF-8 2024-06-11T13:30:43.626 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.626 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-11T13:30:43.626 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.626 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG=en_US.UTF-8 2024-06-11T13:30:43.626 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup config 2024-06-11T13:30:43.626 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.626 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.626 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' dumb = linux ']' 2024-06-11T13:30:43.627 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.627 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-11T13:30:43.627 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.627 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/less.sh 2024-06-11T13:30:43.627 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-11T13:30:43.627 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.627 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-11T13:30:43.627 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.627 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/which2.sh 2024-06-11T13:30:43.627 INFO:tasks.workunit.client.0.smithi057.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-11T13:30:43.627 INFO:tasks.workunit.client.0.smithi057.stderr:+++++ readlink /proc/77332/exe 2024-06-11T13:30:43.627 INFO:tasks.workunit.client.0.smithi057.stderr:++++ basename /usr/bin/bash 2024-06-11T13:30:43.627 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'grep=grep --color=auto' 2024-06-11T13:30:43.627 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-11T13:30:43.628 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-11T13:30:43.628 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.628 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-11T13:30:43.628 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.628 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorls.sh 2024-06-11T13:30:43.628 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' '!' -t 0 ']' 2024-06-11T13:30:43.628 INFO:tasks.workunit.client.0.smithi057.stderr:+++ return 2024-06-11T13:30:43.628 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.628 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-11T13:30:43.628 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.628 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-11T13:30:43.628 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_declare='declare -f' 2024-06-11T13:30:43.628 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_opt=-f 2024-06-11T13:30:43.628 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export S_COLORS=auto 2024-06-11T13:30:43.629 INFO:tasks.workunit.client.0.smithi057.stderr:+++ S_COLORS=auto 2024-06-11T13:30:43.629 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.629 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-11T13:30:43.629 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export which_declare 2024-06-11T13:30:43.629 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.629 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-11T13:30:43.629 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export -f which 2024-06-11T13:30:43.629 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.629 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset i 2024-06-11T13:30:43.629 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset -f pathmunge 2024-06-11T13:30:43.629 INFO:tasks.workunit.client.0.smithi057.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-11T13:30:43.629 INFO:tasks.workunit.client.0.smithi057.stderr:+ export PATH 2024-06-11T13:30:43.629 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-11T13:30:43.629 INFO:tasks.workunit.client.0.smithi057.stderr:+ unset rc 2024-06-11T13:30:43.630 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-11T13:30:43.630 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-11T13:30:43.630 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-11T13:30:43.630 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.630 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-11T13:30:43.630 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.630 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-11T13:30:43.630 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-11T13:30:43.630 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.630 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-11T13:30:43.630 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.630 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/gawk.sh 2024-06-11T13:30:43.630 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.630 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-11T13:30:43.631 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.631 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/lang.sh 2024-06-11T13:30:43.631 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup 2024-06-11T13:30:43.631 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/locale 2024-06-11T13:30:43.631 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.631 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-11T13:30:43.631 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.631 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-11T13:30:43.631 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-11T13:30:43.631 INFO:tasks.workunit.client.0.smithi057.stderr:+ tee ceph_test_rados_api_list.log 2024-06-11T13:30:43.631 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-11T13:30:43.631 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-11T13:30:43.631 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-11T13:30:43.631 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.631 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-11T13:30:43.632 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.632 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-11T13:30:43.632 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-11T13:30:43.632 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-11T13:30:43.632 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.632 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph_test_rados_api_list --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_list.xml 2024-06-11T13:30:43.632 INFO:tasks.workunit.client.0.smithi057.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-11T13:30:43.632 INFO:tasks.workunit.client.0.smithi057.stderr:++++ export LANG=en_US.UTF-8 2024-06-11T13:30:43.632 INFO:tasks.workunit.client.0.smithi057.stderr:++++ LANG=en_US.UTF-8 2024-06-11T13:30:43.632 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.632 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-11T13:30:43.632 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.632 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG=en_US.UTF-8 2024-06-11T13:30:43.632 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup config 2024-06-11T13:30:43.633 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.633 INFO:tasks.workunit.client.0.smithi057.stderr:++ printf %25s api_tier_pp 2024-06-11T13:30:43.633 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.633 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' dumb = linux ']' 2024-06-11T13:30:43.633 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.633 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-11T13:30:43.633 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.633 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/less.sh 2024-06-11T13:30:43.633 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-11T13:30:43.633 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.633 INFO:tasks.workunit.client.0.smithi057.stderr:+ r=' api_tier_pp' 2024-06-11T13:30:43.633 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-11T13:30:43.633 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.633 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/which2.sh 2024-06-11T13:30:43.633 INFO:tasks.workunit.client.0.smithi057.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-11T13:30:43.634 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo api_tier_pp 2024-06-11T13:30:43.634 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-11T13:30:43.634 INFO:tasks.workunit.client.0.smithi057.stderr:+++ umask 2024-06-11T13:30:43.634 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-11T13:30:43.634 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-11T13:30:43.634 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.634 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-11T13:30:43.634 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.634 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-11T13:30:43.634 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-11T13:30:43.634 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.634 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-11T13:30:43.634 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.634 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' 0022 -eq 0 ']' 2024-06-11T13:30:43.635 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/gawk.sh 2024-06-11T13:30:43.635 INFO:tasks.workunit.client.0.smithi057.stderr:++ SHELL=/bin/bash 2024-06-11T13:30:43.635 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.635 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.635 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-11T13:30:43.635 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-11T13:30:43.635 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.635 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/lang.sh 2024-06-11T13:30:43.635 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.635 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-11T13:30:43.635 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup 2024-06-11T13:30:43.635 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-11T13:30:43.635 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.635 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-11T13:30:43.636 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.636 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-11T13:30:43.636 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.636 INFO:tasks.workunit.client.0.smithi057.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.636 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.636 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-11T13:30:43.636 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.636 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-11T13:30:43.636 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.636 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/locale 2024-06-11T13:30:43.636 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.636 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-11T13:30:43.636 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.636 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-11T13:30:43.636 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-11T13:30:43.637 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.637 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-11T13:30:43.637 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.637 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-11T13:30:43.637 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-11T13:30:43.637 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-11T13:30:43.637 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-11T13:30:43.637 INFO:tasks.workunit.client.0.smithi057.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-11T13:30:43.637 INFO:tasks.workunit.client.0.smithi057.stderr:++++ export LANG=en_US.UTF-8 2024-06-11T13:30:43.637 INFO:tasks.workunit.client.0.smithi057.stderr:++++ LANG=en_US.UTF-8 2024-06-11T13:30:43.637 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.637 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-11T13:30:43.637 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.637 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG=en_US.UTF-8 2024-06-11T13:30:43.637 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup config 2024-06-11T13:30:43.638 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.638 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.638 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' dumb = linux ']' 2024-06-11T13:30:43.638 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.638 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-11T13:30:43.638 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.638 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/less.sh 2024-06-11T13:30:43.638 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-11T13:30:43.638 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.638 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-11T13:30:43.638 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.638 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/which2.sh 2024-06-11T13:30:43.638 INFO:tasks.workunit.client.0.smithi057.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-11T13:30:43.639 INFO:tasks.workunit.client.0.smithi057.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-11T13:30:43.639 INFO:tasks.workunit.client.0.smithi057.stderr:++++ export LANG=en_US.UTF-8 2024-06-11T13:30:43.639 INFO:tasks.workunit.client.0.smithi057.stderr:++++ LANG=en_US.UTF-8 2024-06-11T13:30:43.639 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.639 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-11T13:30:43.639 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.639 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG=en_US.UTF-8 2024-06-11T13:30:43.639 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup config 2024-06-11T13:30:43.639 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.639 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.639 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' dumb = linux ']' 2024-06-11T13:30:43.639 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.639 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-11T13:30:43.639 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.639 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/less.sh 2024-06-11T13:30:43.640 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-11T13:30:43.640 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.640 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-11T13:30:43.640 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.640 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/which2.sh 2024-06-11T13:30:43.640 INFO:tasks.workunit.client.0.smithi057.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-11T13:30:43.640 INFO:tasks.workunit.client.0.smithi057.stderr:++ awk '{print $1}' 2024-06-11T13:30:43.640 INFO:tasks.workunit.client.0.smithi057.stderr:+ sed 's/^/ api_list: /' 2024-06-11T13:30:43.640 INFO:tasks.workunit.client.0.smithi057.stderr:+++++ readlink /proc/77352/exe 2024-06-11T13:30:43.640 INFO:tasks.workunit.client.0.smithi057.stderr:++++ basename /usr/bin/bash 2024-06-11T13:30:43.640 INFO:tasks.workunit.client.0.smithi057.stderr:+++++ readlink /proc/77382/exe 2024-06-11T13:30:43.640 INFO:tasks.workunit.client.0.smithi057.stdout:test api_tier_pp on pid 77482 2024-06-11T13:30:43.644 INFO:tasks.workunit.client.0.smithi057.stderr:+ ff=api_tier_pp 2024-06-11T13:30:43.644 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77482 2024-06-11T13:30:43.644 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'test api_tier_pp on pid 77482' 2024-06-11T13:30:43.644 INFO:tasks.workunit.client.0.smithi057.stderr:+ pids[$f]=77482 2024-06-11T13:30:43.644 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-11T13:30:43.644 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' 1 -eq 1 ']' 2024-06-11T13:30:43.644 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_declare='declare -f' 2024-06-11T13:30:43.644 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_opt=-f 2024-06-11T13:30:43.645 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export which_declare 2024-06-11T13:30:43.645 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export -f which 2024-06-11T13:30:43.645 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset i 2024-06-11T13:30:43.645 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset -f pathmunge 2024-06-11T13:30:43.645 INFO:tasks.workunit.client.0.smithi057.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-11T13:30:43.645 INFO:tasks.workunit.client.0.smithi057.stderr:+ export PATH 2024-06-11T13:30:43.645 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-11T13:30:43.645 INFO:tasks.workunit.client.0.smithi057.stderr:+ unset rc 2024-06-11T13:30:43.645 INFO:tasks.workunit.client.0.smithi057.stderr:++ printf %25s api_pool 2024-06-11T13:30:43.645 INFO:tasks.workunit.client.0.smithi057.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_tier_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_tier_pp.xml 2>&1 | tee ceph_test_rados_api_tier_pp.log | sed "s/^/ api_tier_pp: /"' 2024-06-11T13:30:43.645 INFO:tasks.workunit.client.0.smithi057.stderr:++++ basename /usr/bin/bash 2024-06-11T13:30:43.645 INFO:tasks.workunit.client.0.smithi057.stderr:+ r=' api_pool' 2024-06-11T13:30:43.645 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_declare='declare -f' 2024-06-11T13:30:43.645 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_opt=-f 2024-06-11T13:30:43.646 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export which_declare 2024-06-11T13:30:43.646 INFO:tasks.workunit.client.0.smithi057.stderr:+++++ readlink /proc/77399/exe 2024-06-11T13:30:43.646 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export -f which 2024-06-11T13:30:43.646 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset i 2024-06-11T13:30:43.646 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset -f pathmunge 2024-06-11T13:30:43.646 INFO:tasks.workunit.client.0.smithi057.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-11T13:30:43.646 INFO:tasks.workunit.client.0.smithi057.stderr:+ export PATH 2024-06-11T13:30:43.646 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-11T13:30:43.646 INFO:tasks.workunit.client.0.smithi057.stderr:+ unset rc 2024-06-11T13:30:43.646 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph_test_rados_api_lock_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock_pp.xml 2024-06-11T13:30:43.646 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo api_pool 2024-06-11T13:30:43.646 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph_test_rados_api_lock --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock.xml 2024-06-11T13:30:43.646 INFO:tasks.workunit.client.0.smithi057.stderr:+ tee ceph_test_rados_api_lock_pp.log 2024-06-11T13:30:43.647 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'grep=grep --color=auto' 2024-06-11T13:30:43.647 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-11T13:30:43.647 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-11T13:30:43.647 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.647 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-11T13:30:43.647 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.647 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorls.sh 2024-06-11T13:30:43.647 INFO:tasks.workunit.client.0.smithi057.stderr:+ tee ceph_test_rados_api_lock.log 2024-06-11T13:30:43.647 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' '!' -t 0 ']' 2024-06-11T13:30:43.647 INFO:tasks.workunit.client.0.smithi057.stderr:+++ return 2024-06-11T13:30:43.647 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.647 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-11T13:30:43.647 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.647 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-11T13:30:43.647 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export S_COLORS=auto 2024-06-11T13:30:43.648 INFO:tasks.workunit.client.0.smithi057.stderr:+++ S_COLORS=auto 2024-06-11T13:30:43.648 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.648 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-11T13:30:43.648 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.648 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-11T13:30:43.648 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.648 INFO:tasks.workunit.client.0.smithi057.stderr:+ sed 's/^/ api_lock: /' 2024-06-11T13:30:43.648 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -f /etc/bashrc ']' 2024-06-11T13:30:43.648 INFO:tasks.workunit.client.0.smithi057.stderr:+ . /etc/bashrc 2024-06-11T13:30:43.648 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -z '' ']' 2024-06-11T13:30:43.648 INFO:tasks.workunit.client.0.smithi057.stderr:++ BASHRCSOURCED=Y 2024-06-11T13:30:43.648 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.648 INFO:tasks.workunit.client.0.smithi057.stderr:++ shopt -q login_shell 2024-06-11T13:30:43.648 INFO:tasks.workunit.client.0.smithi057.stderr:++ awk '{print $1}' 2024-06-11T13:30:43.649 INFO:tasks.workunit.client.0.smithi057.stderr:+ sed 's/^/ api_lock_pp: /' 2024-06-11T13:30:43.649 INFO:tasks.workunit.client.0.smithi057.stderr:+++ umask 2024-06-11T13:30:43.649 INFO:tasks.workunit.client.0.smithi057.stderr:++++ basename /usr/bin/bash 2024-06-11T13:30:43.649 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' 0022 -eq 0 ']' 2024-06-11T13:30:43.649 INFO:tasks.workunit.client.0.smithi057.stderr:++ SHELL=/bin/bash 2024-06-11T13:30:43.649 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.649 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-11T13:30:43.649 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.649 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-11T13:30:43.649 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-11T13:30:43.649 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.649 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-11T13:30:43.649 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.650 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-11T13:30:43.650 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.650 INFO:tasks.workunit.client.0.smithi057.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.650 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.650 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-11T13:30:43.650 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.650 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-11T13:30:43.650 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.650 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_declare='declare -f' 2024-06-11T13:30:43.650 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_opt=-f 2024-06-11T13:30:43.650 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export which_declare 2024-06-11T13:30:43.650 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export -f which 2024-06-11T13:30:43.650 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset i 2024-06-11T13:30:43.650 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset -f pathmunge 2024-06-11T13:30:43.651 INFO:tasks.workunit.client.0.smithi057.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-11T13:30:43.651 INFO:tasks.workunit.client.0.smithi057.stderr:+ export PATH 2024-06-11T13:30:43.651 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-11T13:30:43.651 INFO:tasks.workunit.client.0.smithi057.stderr:+ unset rc 2024-06-11T13:30:43.651 INFO:tasks.workunit.client.0.smithi057.stderr:+ tee ceph_test_rados_api_misc.log 2024-06-11T13:30:43.651 INFO:tasks.workunit.client.0.smithi057.stdout:test api_pool on pid 77543 2024-06-11T13:30:43.655 INFO:tasks.workunit.client.0.smithi057.stderr:+ ff=api_pool 2024-06-11T13:30:43.655 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77543 2024-06-11T13:30:43.656 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'test api_pool on pid 77543' 2024-06-11T13:30:43.656 INFO:tasks.workunit.client.0.smithi057.stderr:+ pids[$f]=77543 2024-06-11T13:30:43.656 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-11T13:30:43.656 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' 1 -eq 1 ']' 2024-06-11T13:30:43.656 INFO:tasks.workunit.client.0.smithi057.stderr:+ sed 's/^/ api_misc: /' 2024-06-11T13:30:43.656 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph_test_rados_api_misc --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc.xml 2024-06-11T13:30:43.656 INFO:tasks.workunit.client.0.smithi057.stderr:++ printf %25s api_snapshots 2024-06-11T13:30:43.656 INFO:tasks.workunit.client.0.smithi057.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_pool --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_pool.xml 2>&1 | tee ceph_test_rados_api_pool.log | sed "s/^/ api_pool: /"' 2024-06-11T13:30:43.656 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-11T13:30:43.656 INFO:tasks.workunit.client.0.smithi057.stderr:+ r=' api_snapshots' 2024-06-11T13:30:43.656 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-11T13:30:43.656 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-11T13:30:43.656 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.657 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-11T13:30:43.657 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.657 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-11T13:30:43.657 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-11T13:30:43.657 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.657 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -f /etc/bashrc ']' 2024-06-11T13:30:43.657 INFO:tasks.workunit.client.0.smithi057.stderr:+ . /etc/bashrc 2024-06-11T13:30:43.657 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -z '' ']' 2024-06-11T13:30:43.657 INFO:tasks.workunit.client.0.smithi057.stderr:++ BASHRCSOURCED=Y 2024-06-11T13:30:43.657 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.657 INFO:tasks.workunit.client.0.smithi057.stderr:++ shopt -q login_shell 2024-06-11T13:30:43.657 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'grep=grep --color=auto' 2024-06-11T13:30:43.657 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-11T13:30:43.657 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-11T13:30:43.658 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.658 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-11T13:30:43.658 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.658 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorls.sh 2024-06-11T13:30:43.658 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' '!' -t 0 ']' 2024-06-11T13:30:43.658 INFO:tasks.workunit.client.0.smithi057.stderr:+++ return 2024-06-11T13:30:43.658 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.658 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-11T13:30:43.658 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.658 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-11T13:30:43.658 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export S_COLORS=auto 2024-06-11T13:30:43.658 INFO:tasks.workunit.client.0.smithi057.stderr:+++ S_COLORS=auto 2024-06-11T13:30:43.658 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.658 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-11T13:30:43.659 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.659 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-11T13:30:43.659 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.659 INFO:tasks.workunit.client.0.smithi057.stderr:+++ umask 2024-06-11T13:30:43.659 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' 0022 -eq 0 ']' 2024-06-11T13:30:43.659 INFO:tasks.workunit.client.0.smithi057.stderr:++ SHELL=/bin/bash 2024-06-11T13:30:43.659 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.659 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-11T13:30:43.659 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.659 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-11T13:30:43.659 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-11T13:30:43.659 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.659 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-11T13:30:43.659 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.660 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-11T13:30:43.660 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.660 INFO:tasks.workunit.client.0.smithi057.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.660 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.660 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-11T13:30:43.660 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.660 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-11T13:30:43.660 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.660 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-11T13:30:43.660 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-11T13:30:43.660 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-11T13:30:43.660 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.660 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-11T13:30:43.660 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.661 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-11T13:30:43.661 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-11T13:30:43.661 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.661 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'grep=grep --color=auto' 2024-06-11T13:30:43.661 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-11T13:30:43.661 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-11T13:30:43.661 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.661 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-11T13:30:43.661 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.661 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorls.sh 2024-06-11T13:30:43.661 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' '!' -t 0 ']' 2024-06-11T13:30:43.661 INFO:tasks.workunit.client.0.smithi057.stderr:+++ return 2024-06-11T13:30:43.661 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.661 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-11T13:30:43.662 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.662 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-11T13:30:43.662 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export S_COLORS=auto 2024-06-11T13:30:43.662 INFO:tasks.workunit.client.0.smithi057.stderr:+++ S_COLORS=auto 2024-06-11T13:30:43.662 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.662 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-11T13:30:43.662 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.662 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-11T13:30:43.662 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.662 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-11T13:30:43.662 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-11T13:30:43.662 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-11T13:30:43.662 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.663 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-11T13:30:43.663 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.663 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-11T13:30:43.663 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-11T13:30:43.663 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.663 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-11T13:30:43.663 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.663 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/gawk.sh 2024-06-11T13:30:43.663 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.663 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-11T13:30:43.663 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.663 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/lang.sh 2024-06-11T13:30:43.663 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup 2024-06-11T13:30:43.663 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/locale 2024-06-11T13:30:43.663 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.664 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-11T13:30:43.664 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.664 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-11T13:30:43.664 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-11T13:30:43.664 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-11T13:30:43.664 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-11T13:30:43.664 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-11T13:30:43.664 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-11T13:30:43.664 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.664 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-11T13:30:43.664 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.664 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-11T13:30:43.664 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-11T13:30:43.664 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.665 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-11T13:30:43.665 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.665 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/gawk.sh 2024-06-11T13:30:43.665 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.665 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-11T13:30:43.665 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.665 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/lang.sh 2024-06-11T13:30:43.665 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup 2024-06-11T13:30:43.665 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/locale 2024-06-11T13:30:43.665 INFO:tasks.workunit.client.0.smithi057.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-11T13:30:43.665 INFO:tasks.workunit.client.0.smithi057.stderr:++++ export LANG=en_US.UTF-8 2024-06-11T13:30:43.665 INFO:tasks.workunit.client.0.smithi057.stderr:++++ LANG=en_US.UTF-8 2024-06-11T13:30:43.665 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.665 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-11T13:30:43.666 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.666 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG=en_US.UTF-8 2024-06-11T13:30:43.666 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup config 2024-06-11T13:30:43.666 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.666 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.666 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' dumb = linux ']' 2024-06-11T13:30:43.666 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.666 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-11T13:30:43.666 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.666 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/less.sh 2024-06-11T13:30:43.666 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-11T13:30:43.666 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.666 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-11T13:30:43.666 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.667 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/which2.sh 2024-06-11T13:30:43.667 INFO:tasks.workunit.client.0.smithi057.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-11T13:30:43.667 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.667 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-11T13:30:43.667 INFO:tasks.workunit.client.0.smithi057.stderr:+++++ readlink /proc/77420/exe 2024-06-11T13:30:43.667 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.667 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-11T13:30:43.667 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-11T13:30:43.667 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-11T13:30:43.667 INFO:tasks.workunit.client.0.smithi057.stderr:++++ basename /usr/bin/bash 2024-06-11T13:30:43.667 INFO:tasks.workunit.client.0.smithi057.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-11T13:30:43.667 INFO:tasks.workunit.client.0.smithi057.stderr:++++ export LANG=en_US.UTF-8 2024-06-11T13:30:43.667 INFO:tasks.workunit.client.0.smithi057.stderr:++++ LANG=en_US.UTF-8 2024-06-11T13:30:43.667 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_declare='declare -f' 2024-06-11T13:30:43.668 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_opt=-f 2024-06-11T13:30:43.668 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.668 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-11T13:30:43.668 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export which_declare 2024-06-11T13:30:43.668 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.668 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export -f which 2024-06-11T13:30:43.668 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG=en_US.UTF-8 2024-06-11T13:30:43.668 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup config 2024-06-11T13:30:43.668 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset i 2024-06-11T13:30:43.668 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset -f pathmunge 2024-06-11T13:30:43.668 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.668 INFO:tasks.workunit.client.0.smithi057.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-11T13:30:43.668 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.668 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' dumb = linux ']' 2024-06-11T13:30:43.668 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.669 INFO:tasks.workunit.client.0.smithi057.stderr:+ export PATH 2024-06-11T13:30:43.669 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-11T13:30:43.669 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.669 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/less.sh 2024-06-11T13:30:43.669 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-11T13:30:43.669 INFO:tasks.workunit.client.0.smithi057.stderr:+ unset rc 2024-06-11T13:30:43.669 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-11T13:30:43.669 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.669 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-11T13:30:43.669 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.669 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/which2.sh 2024-06-11T13:30:43.669 INFO:tasks.workunit.client.0.smithi057.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-11T13:30:43.669 INFO:tasks.workunit.client.0.smithi057.stderr:++ awk '{print $1}' 2024-06-11T13:30:43.669 INFO:tasks.workunit.client.0.smithi057.stderr:+ sed 's/^/ api_misc_pp: /' 2024-06-11T13:30:43.670 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo api_snapshots 2024-06-11T13:30:43.670 INFO:tasks.workunit.client.0.smithi057.stderr:+ tee ceph_test_rados_api_misc_pp.log 2024-06-11T13:30:43.670 INFO:tasks.workunit.client.0.smithi057.stdout:test api_snapshots on pid 77584 2024-06-11T13:30:43.672 INFO:tasks.workunit.client.0.smithi057.stderr:+ ff=api_snapshots 2024-06-11T13:30:43.672 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77584 2024-06-11T13:30:43.672 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'test api_snapshots on pid 77584' 2024-06-11T13:30:43.672 INFO:tasks.workunit.client.0.smithi057.stderr:+ pids[$f]=77584 2024-06-11T13:30:43.672 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-11T13:30:43.672 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' 1 -eq 1 ']' 2024-06-11T13:30:43.673 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph_test_rados_api_misc_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc_pp.xml 2024-06-11T13:30:43.673 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-11T13:30:43.673 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-11T13:30:43.673 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-11T13:30:43.673 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.673 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-11T13:30:43.673 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.673 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-11T13:30:43.673 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-11T13:30:43.673 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.673 INFO:tasks.workunit.client.0.smithi057.stderr:+++++ readlink /proc/77482/exe 2024-06-11T13:30:43.673 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-11T13:30:43.673 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-11T13:30:43.673 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-11T13:30:43.674 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.674 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-11T13:30:43.674 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.674 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-11T13:30:43.674 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-11T13:30:43.674 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.674 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-11T13:30:43.674 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.674 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/gawk.sh 2024-06-11T13:30:43.674 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.674 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-11T13:30:43.674 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.674 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/lang.sh 2024-06-11T13:30:43.674 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup 2024-06-11T13:30:43.675 INFO:tasks.workunit.client.0.smithi057.stderr:++++ basename /usr/bin/bash 2024-06-11T13:30:43.675 INFO:tasks.workunit.client.0.smithi057.stderr:++ printf %25s api_snapshots_pp 2024-06-11T13:30:43.675 INFO:tasks.workunit.client.0.smithi057.stderr:+ r=' api_snapshots_pp' 2024-06-11T13:30:43.675 INFO:tasks.workunit.client.0.smithi057.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots.xml 2>&1 | tee ceph_test_rados_api_snapshots.log | sed "s/^/ api_snapshots: /"' 2024-06-11T13:30:43.675 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_declare='declare -f' 2024-06-11T13:30:43.675 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_opt=-f 2024-06-11T13:30:43.675 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export which_declare 2024-06-11T13:30:43.675 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export -f which 2024-06-11T13:30:43.675 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset i 2024-06-11T13:30:43.675 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset -f pathmunge 2024-06-11T13:30:43.675 INFO:tasks.workunit.client.0.smithi057.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-11T13:30:43.675 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo api_snapshots_pp 2024-06-11T13:30:43.675 INFO:tasks.workunit.client.0.smithi057.stderr:+ export PATH 2024-06-11T13:30:43.675 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-11T13:30:43.676 INFO:tasks.workunit.client.0.smithi057.stderr:+ unset rc 2024-06-11T13:30:43.676 INFO:tasks.workunit.client.0.smithi057.stderr:+ tee ceph_test_rados_api_tier_pp.log 2024-06-11T13:30:43.676 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -f /etc/bashrc ']' 2024-06-11T13:30:43.676 INFO:tasks.workunit.client.0.smithi057.stderr:+ . /etc/bashrc 2024-06-11T13:30:43.676 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -z '' ']' 2024-06-11T13:30:43.676 INFO:tasks.workunit.client.0.smithi057.stderr:++ BASHRCSOURCED=Y 2024-06-11T13:30:43.676 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.676 INFO:tasks.workunit.client.0.smithi057.stderr:++ shopt -q login_shell 2024-06-11T13:30:43.676 INFO:tasks.workunit.client.0.smithi057.stderr:+ sed 's/^/ api_tier_pp: /' 2024-06-11T13:30:43.676 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph_test_rados_api_tier_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_tier_pp.xml 2024-06-11T13:30:43.676 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/locale 2024-06-11T13:30:43.676 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.676 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-11T13:30:43.676 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.676 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-11T13:30:43.677 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-11T13:30:43.677 INFO:tasks.workunit.client.0.smithi057.stderr:++ awk '{print $1}' 2024-06-11T13:30:43.677 INFO:tasks.workunit.client.0.smithi057.stdout:test api_snapshots_pp on pid 77618 2024-06-11T13:30:43.680 INFO:tasks.workunit.client.0.smithi057.stderr:+ ff=api_snapshots_pp 2024-06-11T13:30:43.680 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77618 2024-06-11T13:30:43.680 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'test api_snapshots_pp on pid 77618' 2024-06-11T13:30:43.680 INFO:tasks.workunit.client.0.smithi057.stderr:+ pids[$f]=77618 2024-06-11T13:30:43.680 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-11T13:30:43.681 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' 1 -eq 1 ']' 2024-06-11T13:30:43.681 INFO:tasks.workunit.client.0.smithi057.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots_pp.xml 2>&1 | tee ceph_test_rados_api_snapshots_pp.log | sed "s/^/ api_snapshots_pp: /"' 2024-06-11T13:30:43.681 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-11T13:30:43.681 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -f /etc/bashrc ']' 2024-06-11T13:30:43.681 INFO:tasks.workunit.client.0.smithi057.stderr:+++ umask 2024-06-11T13:30:43.681 INFO:tasks.workunit.client.0.smithi057.stderr:+ . /etc/bashrc 2024-06-11T13:30:43.681 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' 0022 -eq 0 ']' 2024-06-11T13:30:43.681 INFO:tasks.workunit.client.0.smithi057.stderr:++ SHELL=/bin/bash 2024-06-11T13:30:43.681 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -z '' ']' 2024-06-11T13:30:43.681 INFO:tasks.workunit.client.0.smithi057.stderr:++ BASHRCSOURCED=Y 2024-06-11T13:30:43.681 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.681 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-11T13:30:43.681 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.682 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-11T13:30:43.682 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.682 INFO:tasks.workunit.client.0.smithi057.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-11T13:30:43.682 INFO:tasks.workunit.client.0.smithi057.stderr:++++ export LANG=en_US.UTF-8 2024-06-11T13:30:43.682 INFO:tasks.workunit.client.0.smithi057.stderr:++++ LANG=en_US.UTF-8 2024-06-11T13:30:43.682 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-11T13:30:43.682 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.682 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-11T13:30:43.682 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.682 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-11T13:30:43.682 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.682 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.682 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG=en_US.UTF-8 2024-06-11T13:30:43.682 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-11T13:30:43.683 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup config 2024-06-11T13:30:43.683 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.683 INFO:tasks.workunit.client.0.smithi057.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.683 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.683 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.683 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-11T13:30:43.683 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.683 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-11T13:30:43.683 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.683 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.683 INFO:tasks.workunit.client.0.smithi057.stderr:++ shopt -q login_shell 2024-06-11T13:30:43.683 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' dumb = linux ']' 2024-06-11T13:30:43.683 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.683 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-11T13:30:43.684 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.684 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/less.sh 2024-06-11T13:30:43.684 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-11T13:30:43.684 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.684 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-11T13:30:43.684 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.684 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/which2.sh 2024-06-11T13:30:43.684 INFO:tasks.workunit.client.0.smithi057.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-11T13:30:43.684 INFO:tasks.workunit.client.0.smithi057.stderr:++ printf %25s api_stat 2024-06-11T13:30:43.684 INFO:tasks.workunit.client.0.smithi057.stderr:+ r=' api_stat' 2024-06-11T13:30:43.684 INFO:tasks.workunit.client.0.smithi057.stderr:+++ umask 2024-06-11T13:30:43.684 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo api_stat 2024-06-11T13:30:43.684 INFO:tasks.workunit.client.0.smithi057.stderr:+++++ readlink /proc/77543/exe 2024-06-11T13:30:43.684 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' 0022 -eq 0 ']' 2024-06-11T13:30:43.684 INFO:tasks.workunit.client.0.smithi057.stderr:++ SHELL=/bin/bash 2024-06-11T13:30:43.685 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.685 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-11T13:30:43.685 INFO:tasks.workunit.client.0.smithi057.stderr:++++ basename /usr/bin/bash 2024-06-11T13:30:43.685 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.685 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-11T13:30:43.685 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-11T13:30:43.685 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.685 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-11T13:30:43.685 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.685 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-11T13:30:43.685 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.685 INFO:tasks.workunit.client.0.smithi057.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.685 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.685 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-11T13:30:43.686 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.686 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-11T13:30:43.686 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.686 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_declare='declare -f' 2024-06-11T13:30:43.686 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_opt=-f 2024-06-11T13:30:43.686 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export which_declare 2024-06-11T13:30:43.686 INFO:tasks.workunit.client.0.smithi057.stderr:++ awk '{print $1}' 2024-06-11T13:30:43.686 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export -f which 2024-06-11T13:30:43.686 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset i 2024-06-11T13:30:43.686 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset -f pathmunge 2024-06-11T13:30:43.686 INFO:tasks.workunit.client.0.smithi057.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-11T13:30:43.686 INFO:tasks.workunit.client.0.smithi057.stderr:+ export PATH 2024-06-11T13:30:43.686 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-11T13:30:43.686 INFO:tasks.workunit.client.0.smithi057.stderr:+ unset rc 2024-06-11T13:30:43.687 INFO:tasks.workunit.client.0.smithi057.stderr:+ sed 's/^/ api_pool: /' 2024-06-11T13:30:43.687 INFO:tasks.workunit.client.0.smithi057.stderr:+ ff=api_stat 2024-06-11T13:30:43.687 INFO:tasks.workunit.client.0.smithi057.stdout:test api_stat on pid 77644 2024-06-11T13:30:43.688 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77644 2024-06-11T13:30:43.688 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'test api_stat on pid 77644' 2024-06-11T13:30:43.688 INFO:tasks.workunit.client.0.smithi057.stderr:+ pids[$f]=77644 2024-06-11T13:30:43.688 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-11T13:30:43.689 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' 1 -eq 1 ']' 2024-06-11T13:30:43.689 INFO:tasks.workunit.client.0.smithi057.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat.xml 2>&1 | tee ceph_test_rados_api_stat.log | sed "s/^/ api_stat: /"' 2024-06-11T13:30:43.689 INFO:tasks.workunit.client.0.smithi057.stderr:++ printf %25s api_stat_pp 2024-06-11T13:30:43.689 INFO:tasks.workunit.client.0.smithi057.stderr:+ r=' api_stat_pp' 2024-06-11T13:30:43.689 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo api_stat_pp 2024-06-11T13:30:43.689 INFO:tasks.workunit.client.0.smithi057.stderr:++ awk '{print $1}' 2024-06-11T13:30:43.689 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -f /etc/bashrc ']' 2024-06-11T13:30:43.689 INFO:tasks.workunit.client.0.smithi057.stderr:+ . /etc/bashrc 2024-06-11T13:30:43.689 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -z '' ']' 2024-06-11T13:30:43.689 INFO:tasks.workunit.client.0.smithi057.stderr:++ BASHRCSOURCED=Y 2024-06-11T13:30:43.689 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.689 INFO:tasks.workunit.client.0.smithi057.stderr:++ shopt -q login_shell 2024-06-11T13:30:43.689 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph_test_rados_api_pool --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_pool.xml 2024-06-11T13:30:43.690 INFO:tasks.workunit.client.0.smithi057.stderr:+ tee ceph_test_rados_api_pool.log 2024-06-11T13:30:43.690 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'grep=grep --color=auto' 2024-06-11T13:30:43.690 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-11T13:30:43.690 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-11T13:30:43.690 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.690 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-11T13:30:43.690 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.690 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorls.sh 2024-06-11T13:30:43.690 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' '!' -t 0 ']' 2024-06-11T13:30:43.690 INFO:tasks.workunit.client.0.smithi057.stderr:+++ return 2024-06-11T13:30:43.690 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.690 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-11T13:30:43.690 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.691 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-11T13:30:43.691 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export S_COLORS=auto 2024-06-11T13:30:43.691 INFO:tasks.workunit.client.0.smithi057.stderr:+++ S_COLORS=auto 2024-06-11T13:30:43.691 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.691 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-11T13:30:43.691 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.691 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-11T13:30:43.691 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.691 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'grep=grep --color=auto' 2024-06-11T13:30:43.691 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-11T13:30:43.691 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-11T13:30:43.691 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.691 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-11T13:30:43.691 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.691 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorls.sh 2024-06-11T13:30:43.692 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' '!' -t 0 ']' 2024-06-11T13:30:43.692 INFO:tasks.workunit.client.0.smithi057.stderr:+++ return 2024-06-11T13:30:43.692 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.692 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-11T13:30:43.692 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.692 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-11T13:30:43.692 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export S_COLORS=auto 2024-06-11T13:30:43.692 INFO:tasks.workunit.client.0.smithi057.stderr:+++ S_COLORS=auto 2024-06-11T13:30:43.692 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.692 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-11T13:30:43.692 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.692 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-11T13:30:43.692 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.693 INFO:tasks.workunit.client.0.smithi057.stdout:test api_stat_pp on pid 77658 2024-06-11T13:30:43.696 INFO:tasks.workunit.client.0.smithi057.stderr:+ ff=api_stat_pp 2024-06-11T13:30:43.696 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77658 2024-06-11T13:30:43.696 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'test api_stat_pp on pid 77658' 2024-06-11T13:30:43.696 INFO:tasks.workunit.client.0.smithi057.stderr:+ pids[$f]=77658 2024-06-11T13:30:43.696 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-11T13:30:43.696 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' 1 -eq 1 ']' 2024-06-11T13:30:43.696 INFO:tasks.workunit.client.0.smithi057.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat_pp.xml 2>&1 | tee ceph_test_rados_api_stat_pp.log | sed "s/^/ api_stat_pp: /"' 2024-06-11T13:30:43.696 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -f /etc/bashrc ']' 2024-06-11T13:30:43.697 INFO:tasks.workunit.client.0.smithi057.stderr:+ . /etc/bashrc 2024-06-11T13:30:43.697 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -z '' ']' 2024-06-11T13:30:43.697 INFO:tasks.workunit.client.0.smithi057.stderr:++ BASHRCSOURCED=Y 2024-06-11T13:30:43.697 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.697 INFO:tasks.workunit.client.0.smithi057.stderr:++ shopt -q login_shell 2024-06-11T13:30:43.697 INFO:tasks.workunit.client.0.smithi057.stderr:+++ umask 2024-06-11T13:30:43.697 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' 0022 -eq 0 ']' 2024-06-11T13:30:43.697 INFO:tasks.workunit.client.0.smithi057.stderr:++ SHELL=/bin/bash 2024-06-11T13:30:43.697 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.697 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-11T13:30:43.697 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.697 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-11T13:30:43.697 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-11T13:30:43.697 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.698 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-11T13:30:43.698 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.698 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-11T13:30:43.698 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.698 INFO:tasks.workunit.client.0.smithi057.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.698 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.698 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-11T13:30:43.698 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.698 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-11T13:30:43.698 INFO:tasks.workunit.client.0.smithi057.stderr:+++ umask 2024-06-11T13:30:43.698 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.698 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' 0022 -eq 0 ']' 2024-06-11T13:30:43.698 INFO:tasks.workunit.client.0.smithi057.stderr:++ SHELL=/bin/bash 2024-06-11T13:30:43.698 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.699 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-11T13:30:43.699 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.699 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-11T13:30:43.699 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-11T13:30:43.699 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.699 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-11T13:30:43.699 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.699 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-11T13:30:43.699 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.699 INFO:tasks.workunit.client.0.smithi057.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.699 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.699 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-11T13:30:43.699 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.700 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-11T13:30:43.700 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.700 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-11T13:30:43.700 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-11T13:30:43.700 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-11T13:30:43.700 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.700 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-11T13:30:43.700 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.700 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-11T13:30:43.700 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-11T13:30:43.700 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.700 INFO:tasks.workunit.client.0.smithi057.stderr:++ printf %25s api_watch_notify 2024-06-11T13:30:43.700 INFO:tasks.workunit.client.0.smithi057.stderr:+ r=' api_watch_notify' 2024-06-11T13:30:43.701 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo api_watch_notify 2024-06-11T13:30:43.701 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-11T13:30:43.701 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-11T13:30:43.701 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-11T13:30:43.701 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.701 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-11T13:30:43.701 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.701 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-11T13:30:43.701 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-11T13:30:43.701 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.701 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-11T13:30:43.701 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.701 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/gawk.sh 2024-06-11T13:30:43.701 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.702 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-11T13:30:43.702 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.702 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/lang.sh 2024-06-11T13:30:43.702 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup 2024-06-11T13:30:43.702 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/locale 2024-06-11T13:30:43.702 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-11T13:30:43.702 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-11T13:30:43.702 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-11T13:30:43.702 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.702 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-11T13:30:43.702 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.702 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-11T13:30:43.702 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-11T13:30:43.703 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.703 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.703 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-11T13:30:43.703 INFO:tasks.workunit.client.0.smithi057.stderr:++ awk '{print $1}' 2024-06-11T13:30:43.703 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.703 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-11T13:30:43.703 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-11T13:30:43.703 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-11T13:30:43.703 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'grep=grep --color=auto' 2024-06-11T13:30:43.703 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-11T13:30:43.703 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-11T13:30:43.703 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.703 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-11T13:30:43.703 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.703 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorls.sh 2024-06-11T13:30:43.704 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' '!' -t 0 ']' 2024-06-11T13:30:43.704 INFO:tasks.workunit.client.0.smithi057.stderr:+++ return 2024-06-11T13:30:43.704 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.704 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-11T13:30:43.704 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.704 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-11T13:30:43.704 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export S_COLORS=auto 2024-06-11T13:30:43.704 INFO:tasks.workunit.client.0.smithi057.stderr:+++ S_COLORS=auto 2024-06-11T13:30:43.704 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.704 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-11T13:30:43.704 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.704 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-11T13:30:43.704 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.704 INFO:tasks.workunit.client.0.smithi057.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-11T13:30:43.705 INFO:tasks.workunit.client.0.smithi057.stderr:++++ export LANG=en_US.UTF-8 2024-06-11T13:30:43.705 INFO:tasks.workunit.client.0.smithi057.stderr:++++ LANG=en_US.UTF-8 2024-06-11T13:30:43.705 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.705 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-11T13:30:43.705 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.705 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG=en_US.UTF-8 2024-06-11T13:30:43.705 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup config 2024-06-11T13:30:43.705 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.705 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.705 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' dumb = linux ']' 2024-06-11T13:30:43.705 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.705 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-11T13:30:43.705 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.705 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/less.sh 2024-06-11T13:30:43.706 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-11T13:30:43.706 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.706 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-11T13:30:43.706 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.706 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/which2.sh 2024-06-11T13:30:43.706 INFO:tasks.workunit.client.0.smithi057.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-11T13:30:43.706 INFO:tasks.workunit.client.0.smithi057.stdout:test api_watch_notify on pid 77708 2024-06-11T13:30:43.710 INFO:tasks.workunit.client.0.smithi057.stderr:+ ff=api_watch_notify 2024-06-11T13:30:43.710 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77708 2024-06-11T13:30:43.711 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'test api_watch_notify on pid 77708' 2024-06-11T13:30:43.711 INFO:tasks.workunit.client.0.smithi057.stderr:+ pids[$f]=77708 2024-06-11T13:30:43.711 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-11T13:30:43.711 INFO:tasks.workunit.client.0.smithi057.stderr:+++++ readlink /proc/77618/exe 2024-06-11T13:30:43.711 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' 1 -eq 1 ']' 2024-06-11T13:30:43.711 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-11T13:30:43.711 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-11T13:30:43.711 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-11T13:30:43.711 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.711 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-11T13:30:43.711 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.711 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-11T13:30:43.711 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-11T13:30:43.711 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.712 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-11T13:30:43.712 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.712 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/gawk.sh 2024-06-11T13:30:43.712 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.712 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-11T13:30:43.712 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.712 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/lang.sh 2024-06-11T13:30:43.712 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup 2024-06-11T13:30:43.712 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'grep=grep --color=auto' 2024-06-11T13:30:43.712 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-11T13:30:43.712 INFO:tasks.workunit.client.0.smithi057.stderr:++++ basename /usr/bin/bash 2024-06-11T13:30:43.712 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-11T13:30:43.712 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.712 INFO:tasks.workunit.client.0.smithi057.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify.xml 2>&1 | tee ceph_test_rados_api_watch_notify.log | sed "s/^/ api_watch_notify: /"' 2024-06-11T13:30:43.713 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-11T13:30:43.713 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.713 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorls.sh 2024-06-11T13:30:43.713 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' '!' -t 0 ']' 2024-06-11T13:30:43.713 INFO:tasks.workunit.client.0.smithi057.stderr:+++ return 2024-06-11T13:30:43.713 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.713 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-11T13:30:43.713 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.713 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-11T13:30:43.713 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export S_COLORS=auto 2024-06-11T13:30:43.713 INFO:tasks.workunit.client.0.smithi057.stderr:+++ S_COLORS=auto 2024-06-11T13:30:43.713 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.713 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-11T13:30:43.713 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.713 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-11T13:30:43.714 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.714 INFO:tasks.workunit.client.0.smithi057.stderr:++ printf %25s api_watch_notify_pp 2024-06-11T13:30:43.714 INFO:tasks.workunit.client.0.smithi057.stderr:+ r=' api_watch_notify_pp' 2024-06-11T13:30:43.714 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_declare='declare -f' 2024-06-11T13:30:43.714 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_opt=-f 2024-06-11T13:30:43.714 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export which_declare 2024-06-11T13:30:43.714 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export -f which 2024-06-11T13:30:43.714 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset i 2024-06-11T13:30:43.714 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset -f pathmunge 2024-06-11T13:30:43.714 INFO:tasks.workunit.client.0.smithi057.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-11T13:30:43.714 INFO:tasks.workunit.client.0.smithi057.stderr:+ export PATH 2024-06-11T13:30:43.714 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-11T13:30:43.714 INFO:tasks.workunit.client.0.smithi057.stderr:+ unset rc 2024-06-11T13:30:43.714 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo api_watch_notify_pp 2024-06-11T13:30:43.715 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -f /etc/bashrc ']' 2024-06-11T13:30:43.715 INFO:tasks.workunit.client.0.smithi057.stderr:+ . /etc/bashrc 2024-06-11T13:30:43.715 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph_test_rados_api_snapshots_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots_pp.xml 2024-06-11T13:30:43.715 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -z '' ']' 2024-06-11T13:30:43.715 INFO:tasks.workunit.client.0.smithi057.stderr:++ BASHRCSOURCED=Y 2024-06-11T13:30:43.715 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.715 INFO:tasks.workunit.client.0.smithi057.stderr:++ shopt -q login_shell 2024-06-11T13:30:43.715 INFO:tasks.workunit.client.0.smithi057.stderr:+++ umask 2024-06-11T13:30:43.715 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' 0022 -eq 0 ']' 2024-06-11T13:30:43.715 INFO:tasks.workunit.client.0.smithi057.stderr:++ SHELL=/bin/bash 2024-06-11T13:30:43.715 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.715 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-11T13:30:43.715 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.716 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-11T13:30:43.716 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-11T13:30:43.716 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.716 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-11T13:30:43.716 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.716 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-11T13:30:43.716 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.716 INFO:tasks.workunit.client.0.smithi057.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.716 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.716 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-11T13:30:43.716 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.716 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-11T13:30:43.716 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.716 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/locale 2024-06-11T13:30:43.716 INFO:tasks.workunit.client.0.smithi057.stderr:++ awk '{print $1}' 2024-06-11T13:30:43.717 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-11T13:30:43.717 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-11T13:30:43.717 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-11T13:30:43.717 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.717 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-11T13:30:43.717 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.717 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-11T13:30:43.717 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-11T13:30:43.717 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.717 INFO:tasks.workunit.client.0.smithi057.stderr:+ tee ceph_test_rados_api_snapshots_pp.log 2024-06-11T13:30:43.717 INFO:tasks.workunit.client.0.smithi057.stderr:+ sed 's/^/ api_snapshots_pp: /' 2024-06-11T13:30:43.717 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-11T13:30:43.717 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-11T13:30:43.717 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-11T13:30:43.717 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.717 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-11T13:30:43.718 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.718 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-11T13:30:43.718 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-11T13:30:43.718 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.718 INFO:tasks.workunit.client.0.smithi057.stderr:+ ff=api_watch_notify_pp 2024-06-11T13:30:43.718 INFO:tasks.workunit.client.0.smithi057.stdout:test api_watch_notify_pp on pid 77733 2024-06-11T13:30:43.721 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77733 2024-06-11T13:30:43.721 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'test api_watch_notify_pp on pid 77733' 2024-06-11T13:30:43.721 INFO:tasks.workunit.client.0.smithi057.stderr:+ pids[$f]=77733 2024-06-11T13:30:43.721 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-11T13:30:43.721 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' 1 -eq 1 ']' 2024-06-11T13:30:43.721 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-11T13:30:43.721 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-11T13:30:43.721 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-11T13:30:43.721 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.721 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-11T13:30:43.722 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.722 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-11T13:30:43.722 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-11T13:30:43.722 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.722 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-11T13:30:43.722 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.722 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/gawk.sh 2024-06-11T13:30:43.722 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.722 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-11T13:30:43.722 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.722 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/lang.sh 2024-06-11T13:30:43.722 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup 2024-06-11T13:30:43.722 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/locale 2024-06-11T13:30:43.722 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.723 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-11T13:30:43.723 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.723 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-11T13:30:43.723 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-11T13:30:43.723 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-11T13:30:43.723 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.723 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-11T13:30:43.723 INFO:tasks.workunit.client.0.smithi057.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify_pp.xml 2>&1 | tee ceph_test_rados_api_watch_notify_pp.log | sed "s/^/ api_watch_notify_pp: /"' 2024-06-11T13:30:43.723 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.723 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-11T13:30:43.723 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-11T13:30:43.723 INFO:tasks.workunit.client.0.smithi057.stderr:++ printf %25s api_cmd 2024-06-11T13:30:43.723 INFO:tasks.workunit.client.0.smithi057.stderr:+ r=' api_cmd' 2024-06-11T13:30:43.723 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'grep=grep --color=auto' 2024-06-11T13:30:43.724 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-11T13:30:43.724 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-11T13:30:43.724 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.724 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-11T13:30:43.724 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.724 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorls.sh 2024-06-11T13:30:43.724 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' '!' -t 0 ']' 2024-06-11T13:30:43.724 INFO:tasks.workunit.client.0.smithi057.stderr:+++ return 2024-06-11T13:30:43.724 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.724 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-11T13:30:43.724 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.724 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-11T13:30:43.724 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export S_COLORS=auto 2024-06-11T13:30:43.725 INFO:tasks.workunit.client.0.smithi057.stderr:+++ S_COLORS=auto 2024-06-11T13:30:43.725 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.725 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-11T13:30:43.725 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.725 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-11T13:30:43.725 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.725 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -f /etc/bashrc ']' 2024-06-11T13:30:43.725 INFO:tasks.workunit.client.0.smithi057.stderr:+ . /etc/bashrc 2024-06-11T13:30:43.725 INFO:tasks.workunit.client.0.smithi057.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-11T13:30:43.725 INFO:tasks.workunit.client.0.smithi057.stderr:++++ export LANG=en_US.UTF-8 2024-06-11T13:30:43.725 INFO:tasks.workunit.client.0.smithi057.stderr:++++ LANG=en_US.UTF-8 2024-06-11T13:30:43.725 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -z '' ']' 2024-06-11T13:30:43.725 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.725 INFO:tasks.workunit.client.0.smithi057.stderr:++ BASHRCSOURCED=Y 2024-06-11T13:30:43.726 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-11T13:30:43.726 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.726 INFO:tasks.workunit.client.0.smithi057.stderr:++ shopt -q login_shell 2024-06-11T13:30:43.726 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.726 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG=en_US.UTF-8 2024-06-11T13:30:43.726 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup config 2024-06-11T13:30:43.726 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.726 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-11T13:30:43.726 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-11T13:30:43.726 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.726 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' dumb = linux ']' 2024-06-11T13:30:43.726 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-11T13:30:43.726 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.727 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-11T13:30:43.727 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.727 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.727 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/less.sh 2024-06-11T13:30:43.727 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-11T13:30:43.727 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.727 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-11T13:30:43.727 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-11T13:30:43.727 INFO:tasks.workunit.client.0.smithi057.stderr:+++ umask 2024-06-11T13:30:43.727 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.727 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-11T13:30:43.727 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.727 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/which2.sh 2024-06-11T13:30:43.727 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-11T13:30:43.728 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.728 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-11T13:30:43.728 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.728 INFO:tasks.workunit.client.0.smithi057.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-11T13:30:43.728 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/gawk.sh 2024-06-11T13:30:43.728 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' 0022 -eq 0 ']' 2024-06-11T13:30:43.728 INFO:tasks.workunit.client.0.smithi057.stderr:++ SHELL=/bin/bash 2024-06-11T13:30:43.728 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.728 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-11T13:30:43.728 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.728 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/lang.sh 2024-06-11T13:30:43.728 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup 2024-06-11T13:30:43.728 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.728 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-11T13:30:43.729 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.729 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-11T13:30:43.729 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-11T13:30:43.729 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.729 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-11T13:30:43.729 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-11T13:30:43.729 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.729 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-11T13:30:43.729 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.729 INFO:tasks.workunit.client.0.smithi057.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.729 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.729 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-11T13:30:43.730 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.730 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-11T13:30:43.730 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.730 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/locale 2024-06-11T13:30:43.730 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo api_cmd 2024-06-11T13:30:43.730 INFO:tasks.workunit.client.0.smithi057.stderr:+++++ readlink /proc/77644/exe 2024-06-11T13:30:43.730 INFO:tasks.workunit.client.0.smithi057.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-11T13:30:43.730 INFO:tasks.workunit.client.0.smithi057.stderr:++++ export LANG=en_US.UTF-8 2024-06-11T13:30:43.730 INFO:tasks.workunit.client.0.smithi057.stderr:++++ LANG=en_US.UTF-8 2024-06-11T13:30:43.730 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.730 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-11T13:30:43.730 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.730 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG=en_US.UTF-8 2024-06-11T13:30:43.730 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup config 2024-06-11T13:30:43.731 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.731 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.731 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' dumb = linux ']' 2024-06-11T13:30:43.731 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.731 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-11T13:30:43.731 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.731 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/less.sh 2024-06-11T13:30:43.731 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-11T13:30:43.731 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.731 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-11T13:30:43.731 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.731 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/which2.sh 2024-06-11T13:30:43.731 INFO:tasks.workunit.client.0.smithi057.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-11T13:30:43.731 INFO:tasks.workunit.client.0.smithi057.stderr:++++ basename /usr/bin/bash 2024-06-11T13:30:43.732 INFO:tasks.workunit.client.0.smithi057.stderr:++ awk '{print $1}' 2024-06-11T13:30:43.732 INFO:tasks.workunit.client.0.smithi057.stderr:+++++ readlink /proc/77584/exe 2024-06-11T13:30:43.733 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_declare='declare -f' 2024-06-11T13:30:43.733 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_opt=-f 2024-06-11T13:30:43.733 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export which_declare 2024-06-11T13:30:43.733 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export -f which 2024-06-11T13:30:43.733 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset i 2024-06-11T13:30:43.733 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset -f pathmunge 2024-06-11T13:30:43.733 INFO:tasks.workunit.client.0.smithi057.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-11T13:30:43.733 INFO:tasks.workunit.client.0.smithi057.stderr:+ export PATH 2024-06-11T13:30:43.733 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-11T13:30:43.733 INFO:tasks.workunit.client.0.smithi057.stderr:+ unset rc 2024-06-11T13:30:43.733 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph_test_rados_api_stat --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat.xml 2024-06-11T13:30:43.733 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'grep=grep --color=auto' 2024-06-11T13:30:43.734 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-11T13:30:43.734 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-11T13:30:43.734 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.734 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-11T13:30:43.734 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.734 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorls.sh 2024-06-11T13:30:43.734 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' '!' -t 0 ']' 2024-06-11T13:30:43.734 INFO:tasks.workunit.client.0.smithi057.stderr:+++ return 2024-06-11T13:30:43.734 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.734 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-11T13:30:43.734 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.734 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-11T13:30:43.734 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export S_COLORS=auto 2024-06-11T13:30:43.734 INFO:tasks.workunit.client.0.smithi057.stderr:+++ S_COLORS=auto 2024-06-11T13:30:43.735 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.735 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-11T13:30:43.735 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.735 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-11T13:30:43.735 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.735 INFO:tasks.workunit.client.0.smithi057.stderr:++++ basename /usr/bin/bash 2024-06-11T13:30:43.735 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-11T13:30:43.735 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-11T13:30:43.735 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-11T13:30:43.735 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.735 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-11T13:30:43.735 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.735 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-11T13:30:43.735 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-11T13:30:43.736 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.736 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.736 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-11T13:30:43.736 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.736 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-11T13:30:43.736 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-11T13:30:43.736 INFO:tasks.workunit.client.0.smithi057.stderr:+ sed 's/^/ api_stat: /' 2024-06-11T13:30:43.736 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_declare='declare -f' 2024-06-11T13:30:43.736 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_opt=-f 2024-06-11T13:30:43.736 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export which_declare 2024-06-11T13:30:43.736 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export -f which 2024-06-11T13:30:43.736 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset i 2024-06-11T13:30:43.736 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset -f pathmunge 2024-06-11T13:30:43.736 INFO:tasks.workunit.client.0.smithi057.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-11T13:30:43.736 INFO:tasks.workunit.client.0.smithi057.stderr:+ export PATH 2024-06-11T13:30:43.737 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-11T13:30:43.737 INFO:tasks.workunit.client.0.smithi057.stderr:+ unset rc 2024-06-11T13:30:43.737 INFO:tasks.workunit.client.0.smithi057.stderr:+ tee ceph_test_rados_api_stat.log 2024-06-11T13:30:43.737 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph_test_rados_api_snapshots --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots.xml 2024-06-11T13:30:43.737 INFO:tasks.workunit.client.0.smithi057.stderr:+ tee ceph_test_rados_api_snapshots.log 2024-06-11T13:30:43.737 INFO:tasks.workunit.client.0.smithi057.stdout:test api_cmd on pid 77805 2024-06-11T13:30:43.739 INFO:tasks.workunit.client.0.smithi057.stderr:+ ff=api_cmd 2024-06-11T13:30:43.739 INFO:tasks.workunit.client.0.smithi057.stderr:+ sed 's/^/ api_snapshots: /' 2024-06-11T13:30:43.739 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77805 2024-06-11T13:30:43.739 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-11T13:30:43.740 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'test api_cmd on pid 77805' 2024-06-11T13:30:43.740 INFO:tasks.workunit.client.0.smithi057.stderr:+ pids[$f]=77805 2024-06-11T13:30:43.740 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-11T13:30:43.740 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' 1 -eq 1 ']' 2024-06-11T13:30:43.740 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-11T13:30:43.740 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-11T13:30:43.740 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-11T13:30:43.740 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.740 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-11T13:30:43.740 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.740 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-11T13:30:43.740 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-11T13:30:43.740 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.740 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-11T13:30:43.741 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.741 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/gawk.sh 2024-06-11T13:30:43.741 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.741 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-11T13:30:43.741 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.741 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/lang.sh 2024-06-11T13:30:43.741 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup 2024-06-11T13:30:43.741 INFO:tasks.workunit.client.0.smithi057.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-11T13:30:43.741 INFO:tasks.workunit.client.0.smithi057.stderr:++++ export LANG=en_US.UTF-8 2024-06-11T13:30:43.741 INFO:tasks.workunit.client.0.smithi057.stderr:++++ LANG=en_US.UTF-8 2024-06-11T13:30:43.741 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.741 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-11T13:30:43.741 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.741 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG=en_US.UTF-8 2024-06-11T13:30:43.741 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup config 2024-06-11T13:30:43.742 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.742 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.742 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' dumb = linux ']' 2024-06-11T13:30:43.742 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.742 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-11T13:30:43.742 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.742 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/less.sh 2024-06-11T13:30:43.742 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-11T13:30:43.742 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.742 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-11T13:30:43.742 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.742 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/which2.sh 2024-06-11T13:30:43.742 INFO:tasks.workunit.client.0.smithi057.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-11T13:30:43.743 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-11T13:30:43.743 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-11T13:30:43.743 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-11T13:30:43.743 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.743 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-11T13:30:43.743 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.743 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-11T13:30:43.743 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-11T13:30:43.743 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.743 INFO:tasks.workunit.client.0.smithi057.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd.xml 2>&1 | tee ceph_test_rados_api_cmd.log | sed "s/^/ api_cmd: /"' 2024-06-11T13:30:43.743 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -f /etc/bashrc ']' 2024-06-11T13:30:43.743 INFO:tasks.workunit.client.0.smithi057.stderr:++ printf %25s api_cmd_pp 2024-06-11T13:30:43.743 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/locale 2024-06-11T13:30:43.743 INFO:tasks.workunit.client.0.smithi057.stderr:+ r=' api_cmd_pp' 2024-06-11T13:30:43.743 INFO:tasks.workunit.client.0.smithi057.stderr:+ . /etc/bashrc 2024-06-11T13:30:43.744 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -z '' ']' 2024-06-11T13:30:43.744 INFO:tasks.workunit.client.0.smithi057.stderr:++ BASHRCSOURCED=Y 2024-06-11T13:30:43.744 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.744 INFO:tasks.workunit.client.0.smithi057.stderr:++ shopt -q login_shell 2024-06-11T13:30:43.744 INFO:tasks.workunit.client.0.smithi057.stderr:+++++ readlink /proc/77658/exe 2024-06-11T13:30:43.747 INFO:tasks.workunit.client.0.smithi057.stderr:+++ umask 2024-06-11T13:30:43.747 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' 0022 -eq 0 ']' 2024-06-11T13:30:43.747 INFO:tasks.workunit.client.0.smithi057.stderr:++ SHELL=/bin/bash 2024-06-11T13:30:43.747 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.747 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-11T13:30:43.747 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.747 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-11T13:30:43.747 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-11T13:30:43.747 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.748 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-11T13:30:43.748 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.748 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-11T13:30:43.748 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.748 INFO:tasks.workunit.client.0.smithi057.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.748 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.748 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-11T13:30:43.748 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.748 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-11T13:30:43.748 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.748 INFO:tasks.workunit.client.0.smithi057.stderr:++++ basename /usr/bin/bash 2024-06-11T13:30:43.748 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_declare='declare -f' 2024-06-11T13:30:43.748 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_opt=-f 2024-06-11T13:30:43.748 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export which_declare 2024-06-11T13:30:43.749 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export -f which 2024-06-11T13:30:43.749 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset i 2024-06-11T13:30:43.749 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset -f pathmunge 2024-06-11T13:30:43.749 INFO:tasks.workunit.client.0.smithi057.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-11T13:30:43.749 INFO:tasks.workunit.client.0.smithi057.stderr:+ export PATH 2024-06-11T13:30:43.749 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-11T13:30:43.749 INFO:tasks.workunit.client.0.smithi057.stderr:+ unset rc 2024-06-11T13:30:43.749 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.749 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-11T13:30:43.749 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.749 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-11T13:30:43.749 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-11T13:30:43.749 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph_test_rados_api_stat_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat_pp.xml 2024-06-11T13:30:43.749 INFO:tasks.workunit.client.0.smithi057.stderr:+ tee ceph_test_rados_api_stat_pp.log 2024-06-11T13:30:43.749 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo api_cmd_pp 2024-06-11T13:30:43.750 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-11T13:30:43.751 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-11T13:30:43.751 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-11T13:30:43.752 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-11T13:30:43.752 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.752 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-11T13:30:43.752 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.752 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-11T13:30:43.752 INFO:tasks.workunit.client.0.smithi057.stderr:+ sed 's/^/ api_stat_pp: /' 2024-06-11T13:30:43.752 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-11T13:30:43.752 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.752 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-11T13:30:43.752 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.752 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/gawk.sh 2024-06-11T13:30:43.752 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.752 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-11T13:30:43.752 INFO:tasks.workunit.client.0.smithi057.stderr:++ awk '{print $1}' 2024-06-11T13:30:43.753 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.753 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/lang.sh 2024-06-11T13:30:43.753 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup 2024-06-11T13:30:43.753 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/locale 2024-06-11T13:30:43.753 INFO:tasks.workunit.client.0.smithi057.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-11T13:30:43.753 INFO:tasks.workunit.client.0.smithi057.stderr:++++ export LANG=en_US.UTF-8 2024-06-11T13:30:43.753 INFO:tasks.workunit.client.0.smithi057.stderr:++++ LANG=en_US.UTF-8 2024-06-11T13:30:43.753 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.753 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-11T13:30:43.753 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.753 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG=en_US.UTF-8 2024-06-11T13:30:43.753 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup config 2024-06-11T13:30:43.753 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.753 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.754 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' dumb = linux ']' 2024-06-11T13:30:43.754 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.754 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-11T13:30:43.754 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.754 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/less.sh 2024-06-11T13:30:43.754 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-11T13:30:43.754 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.754 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-11T13:30:43.754 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.754 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/which2.sh 2024-06-11T13:30:43.754 INFO:tasks.workunit.client.0.smithi057.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-11T13:30:43.754 INFO:tasks.workunit.client.0.smithi057.stdout:test api_cmd_pp on pid 77847 2024-06-11T13:30:43.757 INFO:tasks.workunit.client.0.smithi057.stderr:+ ff=api_cmd_pp 2024-06-11T13:30:43.758 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77847 2024-06-11T13:30:43.758 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'test api_cmd_pp on pid 77847' 2024-06-11T13:30:43.758 INFO:tasks.workunit.client.0.smithi057.stderr:+ pids[$f]=77847 2024-06-11T13:30:43.758 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-11T13:30:43.758 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' 1 -eq 1 ']' 2024-06-11T13:30:43.758 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.758 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-11T13:30:43.758 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.758 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-11T13:30:43.758 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-11T13:30:43.758 INFO:tasks.workunit.client.0.smithi057.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd_pp.xml 2>&1 | tee ceph_test_rados_api_cmd_pp.log | sed "s/^/ api_cmd_pp: /"' 2024-06-11T13:30:43.758 INFO:tasks.workunit.client.0.smithi057.stderr:++ printf %25s api_service 2024-06-11T13:30:43.759 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-11T13:30:43.759 INFO:tasks.workunit.client.0.smithi057.stderr:+ r=' api_service' 2024-06-11T13:30:43.759 INFO:tasks.workunit.client.0.smithi057.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-11T13:30:43.759 INFO:tasks.workunit.client.0.smithi057.stderr:++++ export LANG=en_US.UTF-8 2024-06-11T13:30:43.759 INFO:tasks.workunit.client.0.smithi057.stderr:++++ LANG=en_US.UTF-8 2024-06-11T13:30:43.759 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.759 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-11T13:30:43.759 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.759 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG=en_US.UTF-8 2024-06-11T13:30:43.759 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup config 2024-06-11T13:30:43.759 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.759 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.759 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' dumb = linux ']' 2024-06-11T13:30:43.759 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.759 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-11T13:30:43.760 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.760 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/less.sh 2024-06-11T13:30:43.760 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -f /etc/bashrc ']' 2024-06-11T13:30:43.760 INFO:tasks.workunit.client.0.smithi057.stderr:+ . /etc/bashrc 2024-06-11T13:30:43.760 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-11T13:30:43.760 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.760 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-11T13:30:43.760 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.760 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/which2.sh 2024-06-11T13:30:43.760 INFO:tasks.workunit.client.0.smithi057.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-11T13:30:43.760 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'grep=grep --color=auto' 2024-06-11T13:30:43.760 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-11T13:30:43.760 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -z '' ']' 2024-06-11T13:30:43.760 INFO:tasks.workunit.client.0.smithi057.stderr:++ BASHRCSOURCED=Y 2024-06-11T13:30:43.761 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-11T13:30:43.761 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.761 INFO:tasks.workunit.client.0.smithi057.stderr:++ shopt -q login_shell 2024-06-11T13:30:43.761 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.761 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-11T13:30:43.761 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.761 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorls.sh 2024-06-11T13:30:43.761 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' '!' -t 0 ']' 2024-06-11T13:30:43.761 INFO:tasks.workunit.client.0.smithi057.stderr:+++ return 2024-06-11T13:30:43.761 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.761 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-11T13:30:43.761 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.761 INFO:tasks.workunit.client.0.smithi057.stderr:++ awk '{print $1}' 2024-06-11T13:30:43.761 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-11T13:30:43.762 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export S_COLORS=auto 2024-06-11T13:30:43.762 INFO:tasks.workunit.client.0.smithi057.stderr:+++ S_COLORS=auto 2024-06-11T13:30:43.762 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.762 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-11T13:30:43.762 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.762 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-11T13:30:43.762 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.762 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo api_service 2024-06-11T13:30:43.762 INFO:tasks.workunit.client.0.smithi057.stderr:+++ umask 2024-06-11T13:30:43.762 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' 0022 -eq 0 ']' 2024-06-11T13:30:43.762 INFO:tasks.workunit.client.0.smithi057.stderr:++ SHELL=/bin/bash 2024-06-11T13:30:43.762 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.762 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-11T13:30:43.762 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.763 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-11T13:30:43.763 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-11T13:30:43.763 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.763 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-11T13:30:43.763 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.763 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-11T13:30:43.763 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.763 INFO:tasks.workunit.client.0.smithi057.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.763 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.763 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-11T13:30:43.763 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.763 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-11T13:30:43.763 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.764 INFO:tasks.workunit.client.0.smithi057.stdout:test api_service on pid 77861 2024-06-11T13:30:43.767 INFO:tasks.workunit.client.0.smithi057.stderr:+ ff=api_service 2024-06-11T13:30:43.767 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77861 2024-06-11T13:30:43.767 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'test api_service on pid 77861' 2024-06-11T13:30:43.767 INFO:tasks.workunit.client.0.smithi057.stderr:+ pids[$f]=77861 2024-06-11T13:30:43.767 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-11T13:30:43.767 INFO:tasks.workunit.client.0.smithi057.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service.xml 2>&1 | tee ceph_test_rados_api_service.log | sed "s/^/ api_service: /"' 2024-06-11T13:30:43.767 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' 1 -eq 1 ']' 2024-06-11T13:30:43.767 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'grep=grep --color=auto' 2024-06-11T13:30:43.767 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-11T13:30:43.767 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-11T13:30:43.767 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.767 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-11T13:30:43.767 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.768 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorls.sh 2024-06-11T13:30:43.768 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' '!' -t 0 ']' 2024-06-11T13:30:43.768 INFO:tasks.workunit.client.0.smithi057.stderr:+++ return 2024-06-11T13:30:43.768 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.768 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-11T13:30:43.768 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.768 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-11T13:30:43.768 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export S_COLORS=auto 2024-06-11T13:30:43.768 INFO:tasks.workunit.client.0.smithi057.stderr:+++ S_COLORS=auto 2024-06-11T13:30:43.768 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.768 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-11T13:30:43.768 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.768 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-11T13:30:43.768 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.769 INFO:tasks.workunit.client.0.smithi057.stderr:+++++ readlink /proc/77708/exe 2024-06-11T13:30:43.769 INFO:tasks.workunit.client.0.smithi057.stderr:++ printf %25s api_service_pp 2024-06-11T13:30:43.769 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -f /etc/bashrc ']' 2024-06-11T13:30:43.769 INFO:tasks.workunit.client.0.smithi057.stderr:+ . /etc/bashrc 2024-06-11T13:30:43.769 INFO:tasks.workunit.client.0.smithi057.stderr:+ r=' api_service_pp' 2024-06-11T13:30:43.769 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -z '' ']' 2024-06-11T13:30:43.769 INFO:tasks.workunit.client.0.smithi057.stderr:++ BASHRCSOURCED=Y 2024-06-11T13:30:43.769 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.769 INFO:tasks.workunit.client.0.smithi057.stderr:++ shopt -q login_shell 2024-06-11T13:30:43.769 INFO:tasks.workunit.client.0.smithi057.stderr:+++ umask 2024-06-11T13:30:43.769 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo api_service_pp 2024-06-11T13:30:43.769 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' 0022 -eq 0 ']' 2024-06-11T13:30:43.769 INFO:tasks.workunit.client.0.smithi057.stderr:++ SHELL=/bin/bash 2024-06-11T13:30:43.769 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.769 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-11T13:30:43.770 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.770 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-11T13:30:43.770 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-11T13:30:43.770 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.770 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-11T13:30:43.770 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.770 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-11T13:30:43.770 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.770 INFO:tasks.workunit.client.0.smithi057.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.770 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.770 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-11T13:30:43.770 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.770 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-11T13:30:43.770 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.770 INFO:tasks.workunit.client.0.smithi057.stderr:++++ basename /usr/bin/bash 2024-06-11T13:30:43.771 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-11T13:30:43.771 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-11T13:30:43.771 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-11T13:30:43.771 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.771 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-11T13:30:43.771 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.771 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-11T13:30:43.771 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-11T13:30:43.771 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.771 INFO:tasks.workunit.client.0.smithi057.stderr:++ awk '{print $1}' 2024-06-11T13:30:43.771 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_declare='declare -f' 2024-06-11T13:30:43.771 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_opt=-f 2024-06-11T13:30:43.771 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export which_declare 2024-06-11T13:30:43.771 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export -f which 2024-06-11T13:30:43.771 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset i 2024-06-11T13:30:43.772 INFO:tasks.workunit.client.0.smithi057.stdout:test api_service_pp on pid 77877 2024-06-11T13:30:43.775 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset -f pathmunge 2024-06-11T13:30:43.776 INFO:tasks.workunit.client.0.smithi057.stderr:+++++ readlink /proc/77733/exe 2024-06-11T13:30:43.776 INFO:tasks.workunit.client.0.smithi057.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-11T13:30:43.776 INFO:tasks.workunit.client.0.smithi057.stderr:+ export PATH 2024-06-11T13:30:43.776 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-11T13:30:43.776 INFO:tasks.workunit.client.0.smithi057.stderr:+ unset rc 2024-06-11T13:30:43.776 INFO:tasks.workunit.client.0.smithi057.stderr:+ ff=api_service_pp 2024-06-11T13:30:43.776 INFO:tasks.workunit.client.0.smithi057.stderr:++++ basename /usr/bin/bash 2024-06-11T13:30:43.776 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77877 2024-06-11T13:30:43.776 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'test api_service_pp on pid 77877' 2024-06-11T13:30:43.776 INFO:tasks.workunit.client.0.smithi057.stderr:+ pids[$f]=77877 2024-06-11T13:30:43.776 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-11T13:30:43.776 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' 1 -eq 1 ']' 2024-06-11T13:30:43.776 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-11T13:30:43.777 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-11T13:30:43.777 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-11T13:30:43.777 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.777 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-11T13:30:43.777 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.777 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-11T13:30:43.777 INFO:tasks.workunit.client.0.smithi057.stderr:++ printf %25s api_c_write_operations 2024-06-11T13:30:43.777 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-11T13:30:43.777 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.777 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-11T13:30:43.777 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.777 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/gawk.sh 2024-06-11T13:30:43.777 INFO:tasks.workunit.client.0.smithi057.stderr:+ r=' api_c_write_operations' 2024-06-11T13:30:43.777 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.777 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-11T13:30:43.778 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.778 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/lang.sh 2024-06-11T13:30:43.778 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup 2024-06-11T13:30:43.778 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_declare='declare -f' 2024-06-11T13:30:43.778 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_opt=-f 2024-06-11T13:30:43.778 INFO:tasks.workunit.client.0.smithi057.stderr:+ sed 's/^/ api_watch_notify: /' 2024-06-11T13:30:43.778 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export which_declare 2024-06-11T13:30:43.778 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export -f which 2024-06-11T13:30:43.778 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset i 2024-06-11T13:30:43.778 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset -f pathmunge 2024-06-11T13:30:43.778 INFO:tasks.workunit.client.0.smithi057.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-11T13:30:43.778 INFO:tasks.workunit.client.0.smithi057.stderr:+ export PATH 2024-06-11T13:30:43.778 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/locale 2024-06-11T13:30:43.778 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-11T13:30:43.779 INFO:tasks.workunit.client.0.smithi057.stderr:+ unset rc 2024-06-11T13:30:43.779 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph_test_rados_api_watch_notify_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify_pp.xml 2024-06-11T13:30:43.779 INFO:tasks.workunit.client.0.smithi057.stderr:+ tee ceph_test_rados_api_watch_notify_pp.log 2024-06-11T13:30:43.779 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'grep=grep --color=auto' 2024-06-11T13:30:43.779 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-11T13:30:43.779 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-11T13:30:43.779 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.779 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-11T13:30:43.779 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.779 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorls.sh 2024-06-11T13:30:43.779 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' '!' -t 0 ']' 2024-06-11T13:30:43.779 INFO:tasks.workunit.client.0.smithi057.stderr:+++ return 2024-06-11T13:30:43.779 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.779 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-11T13:30:43.780 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.780 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-11T13:30:43.780 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export S_COLORS=auto 2024-06-11T13:30:43.780 INFO:tasks.workunit.client.0.smithi057.stderr:+++ S_COLORS=auto 2024-06-11T13:30:43.780 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.780 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-11T13:30:43.780 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.780 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-11T13:30:43.780 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.780 INFO:tasks.workunit.client.0.smithi057.stderr:+ sed 's/^/ api_watch_notify_pp: /' 2024-06-11T13:30:43.780 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.780 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-11T13:30:43.780 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.780 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-11T13:30:43.781 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-11T13:30:43.781 INFO:tasks.workunit.client.0.smithi057.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service_pp.xml 2>&1 | tee ceph_test_rados_api_service_pp.log | sed "s/^/ api_service_pp: /"' 2024-06-11T13:30:43.781 INFO:tasks.workunit.client.0.smithi057.stderr:+ tee ceph_test_rados_api_watch_notify.log 2024-06-11T13:30:43.781 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -f /etc/bashrc ']' 2024-06-11T13:30:43.781 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo api_c_write_operations 2024-06-11T13:30:43.781 INFO:tasks.workunit.client.0.smithi057.stderr:+ . /etc/bashrc 2024-06-11T13:30:43.781 INFO:tasks.workunit.client.0.smithi057.stderr:++ awk '{print $1}' 2024-06-11T13:30:43.781 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -z '' ']' 2024-06-11T13:30:43.781 INFO:tasks.workunit.client.0.smithi057.stderr:++ BASHRCSOURCED=Y 2024-06-11T13:30:43.781 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.781 INFO:tasks.workunit.client.0.smithi057.stderr:++ shopt -q login_shell 2024-06-11T13:30:43.781 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-11T13:30:43.781 INFO:tasks.workunit.client.0.smithi057.stderr:+++ umask 2024-06-11T13:30:43.781 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' 0022 -eq 0 ']' 2024-06-11T13:30:43.781 INFO:tasks.workunit.client.0.smithi057.stderr:++ SHELL=/bin/bash 2024-06-11T13:30:43.782 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.782 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-11T13:30:43.782 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-11T13:30:43.782 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.782 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-11T13:30:43.782 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-11T13:30:43.782 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-11T13:30:43.782 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.782 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-11T13:30:43.782 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.782 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-11T13:30:43.782 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-11T13:30:43.782 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.782 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-11T13:30:43.783 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-11T13:30:43.783 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.783 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.783 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-11T13:30:43.783 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.783 INFO:tasks.workunit.client.0.smithi057.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.783 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.783 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-11T13:30:43.783 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.783 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-11T13:30:43.783 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.783 INFO:tasks.workunit.client.0.smithi057.stdout:test api_c_write_operations on pid 77908 2024-06-11T13:30:43.786 INFO:tasks.workunit.client.0.smithi057.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-11T13:30:43.787 INFO:tasks.workunit.client.0.smithi057.stderr:++++ export LANG=en_US.UTF-8 2024-06-11T13:30:43.787 INFO:tasks.workunit.client.0.smithi057.stderr:++++ LANG=en_US.UTF-8 2024-06-11T13:30:43.787 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.787 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-11T13:30:43.787 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.787 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG=en_US.UTF-8 2024-06-11T13:30:43.787 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup config 2024-06-11T13:30:43.787 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.787 INFO:tasks.workunit.client.0.smithi057.stderr:+ ff=api_c_write_operations 2024-06-11T13:30:43.787 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.787 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' dumb = linux ']' 2024-06-11T13:30:43.787 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.787 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-11T13:30:43.787 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.788 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/less.sh 2024-06-11T13:30:43.788 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-11T13:30:43.788 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.788 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-11T13:30:43.788 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.788 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/which2.sh 2024-06-11T13:30:43.788 INFO:tasks.workunit.client.0.smithi057.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-11T13:30:43.788 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77908 2024-06-11T13:30:43.788 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'test api_c_write_operations on pid 77908' 2024-06-11T13:30:43.788 INFO:tasks.workunit.client.0.smithi057.stderr:+ pids[$f]=77908 2024-06-11T13:30:43.788 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-11T13:30:43.788 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' 1 -eq 1 ']' 2024-06-11T13:30:43.788 INFO:tasks.workunit.client.0.smithi057.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_write_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_write_operations.xml 2>&1 | tee ceph_test_rados_api_c_write_operations.log | sed "s/^/ api_c_write_operations: /"' 2024-06-11T13:30:43.789 INFO:tasks.workunit.client.0.smithi057.stderr:++ printf %25s api_c_read_operations 2024-06-11T13:30:43.789 INFO:tasks.workunit.client.0.smithi057.stderr:+ r=' api_c_read_operations' 2024-06-11T13:30:43.789 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo api_c_read_operations 2024-06-11T13:30:43.789 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -f /etc/bashrc ']' 2024-06-11T13:30:43.789 INFO:tasks.workunit.client.0.smithi057.stderr:+ . /etc/bashrc 2024-06-11T13:30:43.789 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -z '' ']' 2024-06-11T13:30:43.789 INFO:tasks.workunit.client.0.smithi057.stderr:++ BASHRCSOURCED=Y 2024-06-11T13:30:43.789 INFO:tasks.workunit.client.0.smithi057.stderr:++ awk '{print $1}' 2024-06-11T13:30:43.789 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.789 INFO:tasks.workunit.client.0.smithi057.stderr:++ shopt -q login_shell 2024-06-11T13:30:43.789 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'grep=grep --color=auto' 2024-06-11T13:30:43.789 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-11T13:30:43.789 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-11T13:30:43.789 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.790 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-11T13:30:43.790 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.790 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorls.sh 2024-06-11T13:30:43.790 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-11T13:30:43.790 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' '!' -t 0 ']' 2024-06-11T13:30:43.790 INFO:tasks.workunit.client.0.smithi057.stderr:+++ return 2024-06-11T13:30:43.790 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-11T13:30:43.790 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.790 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-11T13:30:43.790 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-11T13:30:43.790 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.790 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-11T13:30:43.790 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.790 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph_test_rados_api_watch_notify --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify.xml 2024-06-11T13:30:43.791 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-11T13:30:43.791 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export S_COLORS=auto 2024-06-11T13:30:43.791 INFO:tasks.workunit.client.0.smithi057.stderr:+++ S_COLORS=auto 2024-06-11T13:30:43.791 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.791 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-11T13:30:43.791 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.791 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-11T13:30:43.791 INFO:tasks.workunit.client.0.smithi057.stderr:+++ umask 2024-06-11T13:30:43.791 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.791 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-11T13:30:43.791 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-11T13:30:43.791 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.791 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.792 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-11T13:30:43.792 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.792 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/gawk.sh 2024-06-11T13:30:43.792 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.792 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-11T13:30:43.792 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.792 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' 0022 -eq 0 ']' 2024-06-11T13:30:43.792 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/lang.sh 2024-06-11T13:30:43.792 INFO:tasks.workunit.client.0.smithi057.stderr:++ SHELL=/bin/bash 2024-06-11T13:30:43.792 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup 2024-06-11T13:30:43.792 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.792 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-11T13:30:43.792 INFO:tasks.workunit.client.0.smithi057.stderr:+++++ readlink /proc/77805/exe 2024-06-11T13:30:43.792 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.792 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-11T13:30:43.793 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-11T13:30:43.793 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.793 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-11T13:30:43.793 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.793 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-11T13:30:43.793 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.793 INFO:tasks.workunit.client.0.smithi057.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.793 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.793 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-11T13:30:43.793 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.793 INFO:tasks.workunit.client.0.smithi057.stderr:++++ basename /usr/bin/bash 2024-06-11T13:30:43.793 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-11T13:30:43.793 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.794 INFO:tasks.workunit.client.0.smithi057.stdout:test api_c_read_operations on pid 77921 2024-06-11T13:30:43.796 INFO:tasks.workunit.client.0.smithi057.stderr:+ ff=api_c_read_operations 2024-06-11T13:30:43.796 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77921 2024-06-11T13:30:43.796 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'test api_c_read_operations on pid 77921' 2024-06-11T13:30:43.796 INFO:tasks.workunit.client.0.smithi057.stderr:+ pids[$f]=77921 2024-06-11T13:30:43.797 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-11T13:30:43.797 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' 1 -eq 1 ']' 2024-06-11T13:30:43.797 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/locale 2024-06-11T13:30:43.797 INFO:tasks.workunit.client.0.smithi057.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_read_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_read_operations.xml 2>&1 | tee ceph_test_rados_api_c_read_operations.log | sed "s/^/ api_c_read_operations: /"' 2024-06-11T13:30:43.797 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_declare='declare -f' 2024-06-11T13:30:43.797 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_opt=-f 2024-06-11T13:30:43.797 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export which_declare 2024-06-11T13:30:43.797 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export -f which 2024-06-11T13:30:43.797 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset i 2024-06-11T13:30:43.797 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset -f pathmunge 2024-06-11T13:30:43.797 INFO:tasks.workunit.client.0.smithi057.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-11T13:30:43.797 INFO:tasks.workunit.client.0.smithi057.stderr:+ export PATH 2024-06-11T13:30:43.797 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-11T13:30:43.797 INFO:tasks.workunit.client.0.smithi057.stderr:+ unset rc 2024-06-11T13:30:43.797 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph_test_rados_api_cmd --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd.xml 2024-06-11T13:30:43.798 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.798 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -f /etc/bashrc ']' 2024-06-11T13:30:43.798 INFO:tasks.workunit.client.0.smithi057.stderr:+ . /etc/bashrc 2024-06-11T13:30:43.798 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-11T13:30:43.798 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.798 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-11T13:30:43.798 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-11T13:30:43.798 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-11T13:30:43.798 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-11T13:30:43.798 INFO:tasks.workunit.client.0.smithi057.stderr:+ sed 's/^/ api_cmd: /' 2024-06-11T13:30:43.798 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-11T13:30:43.798 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.798 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-11T13:30:43.798 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.799 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-11T13:30:43.799 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-11T13:30:43.799 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.799 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -z '' ']' 2024-06-11T13:30:43.799 INFO:tasks.workunit.client.0.smithi057.stderr:++ printf %25s api_cls_remote_reads 2024-06-11T13:30:43.799 INFO:tasks.workunit.client.0.smithi057.stderr:+ r=' api_cls_remote_reads' 2024-06-11T13:30:43.799 INFO:tasks.workunit.client.0.smithi057.stderr:++ BASHRCSOURCED=Y 2024-06-11T13:30:43.799 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.799 INFO:tasks.workunit.client.0.smithi057.stderr:++ shopt -q login_shell 2024-06-11T13:30:43.799 INFO:tasks.workunit.client.0.smithi057.stderr:+ tee ceph_test_rados_api_cmd.log 2024-06-11T13:30:43.799 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo api_cls_remote_reads 2024-06-11T13:30:43.799 INFO:tasks.workunit.client.0.smithi057.stderr:++ awk '{print $1}' 2024-06-11T13:30:43.799 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-11T13:30:43.799 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-11T13:30:43.800 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-11T13:30:43.800 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.800 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-11T13:30:43.800 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.800 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-11T13:30:43.800 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-11T13:30:43.800 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.800 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-11T13:30:43.800 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.800 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/gawk.sh 2024-06-11T13:30:43.800 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.800 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-11T13:30:43.800 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.800 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/lang.sh 2024-06-11T13:30:43.800 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup 2024-06-11T13:30:43.801 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-11T13:30:43.801 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-11T13:30:43.801 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-11T13:30:43.801 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.801 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-11T13:30:43.801 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.801 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-11T13:30:43.801 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-11T13:30:43.801 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.801 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-11T13:30:43.801 INFO:tasks.workunit.client.0.smithi057.stderr:+++ umask 2024-06-11T13:30:43.801 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/locale 2024-06-11T13:30:43.801 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' 0022 -eq 0 ']' 2024-06-11T13:30:43.801 INFO:tasks.workunit.client.0.smithi057.stderr:++ SHELL=/bin/bash 2024-06-11T13:30:43.801 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.802 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-11T13:30:43.802 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.802 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-11T13:30:43.802 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-11T13:30:43.802 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.802 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-11T13:30:43.802 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.802 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-11T13:30:43.802 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.802 INFO:tasks.workunit.client.0.smithi057.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.802 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.802 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-11T13:30:43.802 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.802 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-11T13:30:43.803 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.803 INFO:tasks.workunit.client.0.smithi057.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-11T13:30:43.803 INFO:tasks.workunit.client.0.smithi057.stderr:++++ export LANG=en_US.UTF-8 2024-06-11T13:30:43.803 INFO:tasks.workunit.client.0.smithi057.stderr:++++ LANG=en_US.UTF-8 2024-06-11T13:30:43.803 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.803 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-11T13:30:43.803 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.803 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG=en_US.UTF-8 2024-06-11T13:30:43.803 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup config 2024-06-11T13:30:43.803 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.803 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.803 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' dumb = linux ']' 2024-06-11T13:30:43.803 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.803 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-11T13:30:43.804 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.804 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/less.sh 2024-06-11T13:30:43.804 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-11T13:30:43.804 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.804 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-11T13:30:43.804 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.804 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/which2.sh 2024-06-11T13:30:43.804 INFO:tasks.workunit.client.0.smithi057.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-11T13:30:43.804 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.804 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-11T13:30:43.804 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.804 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-11T13:30:43.804 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-11T13:30:43.804 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-11T13:30:43.805 INFO:tasks.workunit.client.0.smithi057.stdout:test api_cls_remote_reads on pid 77963 2024-06-11T13:30:43.806 INFO:tasks.workunit.client.0.smithi057.stderr:+ ff=api_cls_remote_reads 2024-06-11T13:30:43.806 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77963 2024-06-11T13:30:43.806 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'test api_cls_remote_reads on pid 77963' 2024-06-11T13:30:43.806 INFO:tasks.workunit.client.0.smithi057.stderr:+ pids[$f]=77963 2024-06-11T13:30:43.806 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-11T13:30:43.806 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' 1 -eq 1 ']' 2024-06-11T13:30:43.806 INFO:tasks.workunit.client.0.smithi057.stderr:++ printf %25s list_parallel 2024-06-11T13:30:43.806 INFO:tasks.workunit.client.0.smithi057.stderr:+ r=' list_parallel' 2024-06-11T13:30:43.807 INFO:tasks.workunit.client.0.smithi057.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-11T13:30:43.807 INFO:tasks.workunit.client.0.smithi057.stderr:++++ export LANG=en_US.UTF-8 2024-06-11T13:30:43.807 INFO:tasks.workunit.client.0.smithi057.stderr:++++ LANG=en_US.UTF-8 2024-06-11T13:30:43.807 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.807 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-11T13:30:43.807 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.807 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG=en_US.UTF-8 2024-06-11T13:30:43.807 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup config 2024-06-11T13:30:43.807 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.807 INFO:tasks.workunit.client.0.smithi057.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cls_remote_reads --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cls_remote_reads.xml 2>&1 | tee ceph_test_rados_api_cls_remote_reads.log | sed "s/^/ api_cls_remote_reads: /"' 2024-06-11T13:30:43.807 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo list_parallel 2024-06-11T13:30:43.807 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.807 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' dumb = linux ']' 2024-06-11T13:30:43.807 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.808 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-11T13:30:43.808 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.808 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/less.sh 2024-06-11T13:30:43.808 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-11T13:30:43.808 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.808 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-11T13:30:43.808 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.808 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/which2.sh 2024-06-11T13:30:43.808 INFO:tasks.workunit.client.0.smithi057.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-11T13:30:43.808 INFO:tasks.workunit.client.0.smithi057.stderr:++ awk '{print $1}' 2024-06-11T13:30:43.808 INFO:tasks.workunit.client.0.smithi057.stderr:+++++ readlink /proc/77847/exe 2024-06-11T13:30:43.808 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'grep=grep --color=auto' 2024-06-11T13:30:43.808 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-11T13:30:43.808 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-11T13:30:43.809 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.809 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-11T13:30:43.809 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.809 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorls.sh 2024-06-11T13:30:43.809 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' '!' -t 0 ']' 2024-06-11T13:30:43.809 INFO:tasks.workunit.client.0.smithi057.stderr:+++ return 2024-06-11T13:30:43.809 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.809 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-11T13:30:43.809 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.809 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-11T13:30:43.809 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export S_COLORS=auto 2024-06-11T13:30:43.809 INFO:tasks.workunit.client.0.smithi057.stderr:+++ S_COLORS=auto 2024-06-11T13:30:43.809 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.809 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-11T13:30:43.809 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.810 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-11T13:30:43.810 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.810 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -f /etc/bashrc ']' 2024-06-11T13:30:43.810 INFO:tasks.workunit.client.0.smithi057.stderr:+ . /etc/bashrc 2024-06-11T13:30:43.810 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -z '' ']' 2024-06-11T13:30:43.810 INFO:tasks.workunit.client.0.smithi057.stderr:++ BASHRCSOURCED=Y 2024-06-11T13:30:43.810 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.810 INFO:tasks.workunit.client.0.smithi057.stderr:++ shopt -q login_shell 2024-06-11T13:30:43.810 INFO:tasks.workunit.client.0.smithi057.stderr:++++ basename /usr/bin/bash 2024-06-11T13:30:43.810 INFO:tasks.workunit.client.0.smithi057.stderr:+++++ readlink /proc/77861/exe 2024-06-11T13:30:43.810 INFO:tasks.workunit.client.0.smithi057.stderr:+++ umask 2024-06-11T13:30:43.810 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' 0022 -eq 0 ']' 2024-06-11T13:30:43.810 INFO:tasks.workunit.client.0.smithi057.stderr:++ SHELL=/bin/bash 2024-06-11T13:30:43.810 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.811 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-11T13:30:43.811 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.811 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-11T13:30:43.811 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-11T13:30:43.811 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.811 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-11T13:30:43.811 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.811 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-11T13:30:43.811 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.811 INFO:tasks.workunit.client.0.smithi057.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.811 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.811 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_declare='declare -f' 2024-06-11T13:30:43.811 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-11T13:30:43.811 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_opt=-f 2024-06-11T13:30:43.811 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.812 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-11T13:30:43.812 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.812 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export which_declare 2024-06-11T13:30:43.812 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export -f which 2024-06-11T13:30:43.812 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset i 2024-06-11T13:30:43.812 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset -f pathmunge 2024-06-11T13:30:43.812 INFO:tasks.workunit.client.0.smithi057.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-11T13:30:43.812 INFO:tasks.workunit.client.0.smithi057.stdout:test list_parallel on pid 77977 2024-06-11T13:30:43.815 INFO:tasks.workunit.client.0.smithi057.stderr:+ export PATH 2024-06-11T13:30:43.815 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-11T13:30:43.815 INFO:tasks.workunit.client.0.smithi057.stderr:+ unset rc 2024-06-11T13:30:43.815 INFO:tasks.workunit.client.0.smithi057.stderr:+ ff=list_parallel 2024-06-11T13:30:43.816 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph_test_rados_api_cmd_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd_pp.xml 2024-06-11T13:30:43.816 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77977 2024-06-11T13:30:43.816 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'test list_parallel on pid 77977' 2024-06-11T13:30:43.816 INFO:tasks.workunit.client.0.smithi057.stderr:+ pids[$f]=77977 2024-06-11T13:30:43.816 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-11T13:30:43.816 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' 1 -eq 1 ']' 2024-06-11T13:30:43.816 INFO:tasks.workunit.client.0.smithi057.stderr:++++ basename /usr/bin/bash 2024-06-11T13:30:43.816 INFO:tasks.workunit.client.0.smithi057.stderr:++ printf %25s open_pools_parallel 2024-06-11T13:30:43.816 INFO:tasks.workunit.client.0.smithi057.stderr:+ r=' open_pools_parallel' 2024-06-11T13:30:43.816 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_declare='declare -f' 2024-06-11T13:30:43.816 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_opt=-f 2024-06-11T13:30:43.816 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export which_declare 2024-06-11T13:30:43.816 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export -f which 2024-06-11T13:30:43.816 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset i 2024-06-11T13:30:43.817 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset -f pathmunge 2024-06-11T13:30:43.817 INFO:tasks.workunit.client.0.smithi057.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-11T13:30:43.817 INFO:tasks.workunit.client.0.smithi057.stderr:+ export PATH 2024-06-11T13:30:43.817 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-11T13:30:43.817 INFO:tasks.workunit.client.0.smithi057.stderr:+ unset rc 2024-06-11T13:30:43.817 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'grep=grep --color=auto' 2024-06-11T13:30:43.817 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-11T13:30:43.817 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-11T13:30:43.817 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.817 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-11T13:30:43.817 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.817 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorls.sh 2024-06-11T13:30:43.817 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' '!' -t 0 ']' 2024-06-11T13:30:43.817 INFO:tasks.workunit.client.0.smithi057.stderr:+++ return 2024-06-11T13:30:43.818 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.818 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-11T13:30:43.818 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.818 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-11T13:30:43.818 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export S_COLORS=auto 2024-06-11T13:30:43.818 INFO:tasks.workunit.client.0.smithi057.stderr:+++ S_COLORS=auto 2024-06-11T13:30:43.818 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.818 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-11T13:30:43.818 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.818 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-11T13:30:43.818 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.818 INFO:tasks.workunit.client.0.smithi057.stderr:+ sed 's/^/ api_service: /' 2024-06-11T13:30:43.818 INFO:tasks.workunit.client.0.smithi057.stderr:+ tee ceph_test_rados_api_cmd_pp.log 2024-06-11T13:30:43.819 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-11T13:30:43.819 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-11T13:30:43.819 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-11T13:30:43.819 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.819 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-11T13:30:43.819 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.819 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-11T13:30:43.819 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-11T13:30:43.819 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.819 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-11T13:30:43.819 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.819 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/gawk.sh 2024-06-11T13:30:43.819 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.819 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-11T13:30:43.819 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.820 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/lang.sh 2024-06-11T13:30:43.820 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup 2024-06-11T13:30:43.820 INFO:tasks.workunit.client.0.smithi057.stderr:+ bash -o pipefail -exc 'ceph_test_rados_list_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/list_parallel.xml 2>&1 | tee ceph_test_rados_list_parallel.log | sed "s/^/ list_parallel: /"' 2024-06-11T13:30:43.820 INFO:tasks.workunit.client.0.smithi057.stderr:+ sed 's/^/ api_cmd_pp: /' 2024-06-11T13:30:43.820 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'grep=grep --color=auto' 2024-06-11T13:30:43.820 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-11T13:30:43.820 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-11T13:30:43.820 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.820 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-11T13:30:43.820 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.820 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorls.sh 2024-06-11T13:30:43.820 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' '!' -t 0 ']' 2024-06-11T13:30:43.820 INFO:tasks.workunit.client.0.smithi057.stderr:+++ return 2024-06-11T13:30:43.820 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.820 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-11T13:30:43.821 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.821 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/locale 2024-06-11T13:30:43.821 INFO:tasks.workunit.client.0.smithi057.stderr:+ tee ceph_test_rados_api_service.log 2024-06-11T13:30:43.821 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-11T13:30:43.821 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export S_COLORS=auto 2024-06-11T13:30:43.821 INFO:tasks.workunit.client.0.smithi057.stderr:+++ S_COLORS=auto 2024-06-11T13:30:43.821 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.821 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-11T13:30:43.821 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo open_pools_parallel 2024-06-11T13:30:43.821 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.821 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-11T13:30:43.821 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.821 INFO:tasks.workunit.client.0.smithi057.stderr:++ awk '{print $1}' 2024-06-11T13:30:43.821 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph_test_rados_api_service --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service.xml 2024-06-11T13:30:43.822 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-11T13:30:43.822 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-11T13:30:43.822 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-11T13:30:43.822 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.822 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-11T13:30:43.822 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.822 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-11T13:30:43.822 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-11T13:30:43.822 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.822 INFO:tasks.workunit.client.0.smithi057.stdout:test open_pools_parallel on pid 77996 2024-06-11T13:30:43.825 INFO:tasks.workunit.client.0.smithi057.stderr:+ ff=open_pools_parallel 2024-06-11T13:30:43.825 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77996 2024-06-11T13:30:43.826 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'test open_pools_parallel on pid 77996' 2024-06-11T13:30:43.826 INFO:tasks.workunit.client.0.smithi057.stderr:+ pids[$f]=77996 2024-06-11T13:30:43.826 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-06-11T13:30:43.826 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' 1 -eq 1 ']' 2024-06-11T13:30:43.826 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -f /etc/bashrc ']' 2024-06-11T13:30:43.826 INFO:tasks.workunit.client.0.smithi057.stderr:+ . /etc/bashrc 2024-06-11T13:30:43.826 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -z '' ']' 2024-06-11T13:30:43.826 INFO:tasks.workunit.client.0.smithi057.stderr:++ BASHRCSOURCED=Y 2024-06-11T13:30:43.826 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.826 INFO:tasks.workunit.client.0.smithi057.stderr:++ printf %25s delete_pools_parallel 2024-06-11T13:30:43.826 INFO:tasks.workunit.client.0.smithi057.stderr:++ shopt -q login_shell 2024-06-11T13:30:43.826 INFO:tasks.workunit.client.0.smithi057.stderr:+ r=' delete_pools_parallel' 2024-06-11T13:30:43.826 INFO:tasks.workunit.client.0.smithi057.stderr:+++ umask 2024-06-11T13:30:43.826 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' 0022 -eq 0 ']' 2024-06-11T13:30:43.827 INFO:tasks.workunit.client.0.smithi057.stderr:++ SHELL=/bin/bash 2024-06-11T13:30:43.827 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.827 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-11T13:30:43.827 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.827 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-11T13:30:43.827 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-11T13:30:43.827 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-11T13:30:43.827 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-11T13:30:43.827 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.827 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-11T13:30:43.827 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.827 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-11T13:30:43.827 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-11T13:30:43.827 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.828 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-11T13:30:43.828 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-11T13:30:43.828 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.828 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-11T13:30:43.828 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.828 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.828 INFO:tasks.workunit.client.0.smithi057.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.828 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.828 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-11T13:30:43.828 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.828 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-11T13:30:43.828 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.828 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-11T13:30:43.829 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-11T13:30:43.829 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-11T13:30:43.829 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.829 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-11T13:30:43.829 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.829 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-11T13:30:43.829 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-11T13:30:43.829 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.829 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-11T13:30:43.829 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.829 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/gawk.sh 2024-06-11T13:30:43.829 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.829 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-11T13:30:43.829 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.829 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/lang.sh 2024-06-11T13:30:43.830 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup 2024-06-11T13:30:43.830 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.830 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-11T13:30:43.830 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.830 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-11T13:30:43.830 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-11T13:30:43.830 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-11T13:30:43.830 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-11T13:30:43.830 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-11T13:30:43.830 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.830 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-11T13:30:43.830 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.830 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-11T13:30:43.831 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-11T13:30:43.831 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.831 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-11T13:30:43.831 INFO:tasks.workunit.client.0.smithi057.stderr:+ bash -o pipefail -exc 'ceph_test_rados_open_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/open_pools_parallel.xml 2>&1 | tee ceph_test_rados_open_pools_parallel.log | sed "s/^/ open_pools_parallel: /"' 2024-06-11T13:30:43.831 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'grep=grep --color=auto' 2024-06-11T13:30:43.831 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-11T13:30:43.831 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-11T13:30:43.831 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.831 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-11T13:30:43.831 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.831 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorls.sh 2024-06-11T13:30:43.831 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' '!' -t 0 ']' 2024-06-11T13:30:43.831 INFO:tasks.workunit.client.0.smithi057.stderr:+++ return 2024-06-11T13:30:43.831 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.832 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-11T13:30:43.832 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.832 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-11T13:30:43.832 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-11T13:30:43.832 INFO:tasks.workunit.client.0.smithi057.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-11T13:30:43.832 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-11T13:30:43.832 INFO:tasks.workunit.client.0.smithi057.stderr:++++ export LANG=en_US.UTF-8 2024-06-11T13:30:43.832 INFO:tasks.workunit.client.0.smithi057.stderr:++++ LANG=en_US.UTF-8 2024-06-11T13:30:43.832 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export S_COLORS=auto 2024-06-11T13:30:43.832 INFO:tasks.workunit.client.0.smithi057.stderr:+++ S_COLORS=auto 2024-06-11T13:30:43.832 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-11T13:30:43.832 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.832 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-11T13:30:43.832 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.832 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.833 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-11T13:30:43.833 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.833 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-11T13:30:43.833 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-11T13:30:43.833 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.833 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.833 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.833 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-11T13:30:43.833 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG=en_US.UTF-8 2024-06-11T13:30:43.833 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup config 2024-06-11T13:30:43.833 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.833 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-11T13:30:43.833 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -f /etc/bashrc ']' 2024-06-11T13:30:43.833 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.834 INFO:tasks.workunit.client.0.smithi057.stderr:+ . /etc/bashrc 2024-06-11T13:30:43.834 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-11T13:30:43.834 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.834 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/gawk.sh 2024-06-11T13:30:43.834 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.834 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' dumb = linux ']' 2024-06-11T13:30:43.834 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.834 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-11T13:30:43.834 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.834 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.834 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/less.sh 2024-06-11T13:30:43.834 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-11T13:30:43.834 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -z '' ']' 2024-06-11T13:30:43.834 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.834 INFO:tasks.workunit.client.0.smithi057.stderr:++ BASHRCSOURCED=Y 2024-06-11T13:30:43.835 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/lang.sh 2024-06-11T13:30:43.835 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.835 INFO:tasks.workunit.client.0.smithi057.stderr:++ shopt -q login_shell 2024-06-11T13:30:43.835 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-11T13:30:43.835 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.835 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup 2024-06-11T13:30:43.835 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-11T13:30:43.835 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.835 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/which2.sh 2024-06-11T13:30:43.835 INFO:tasks.workunit.client.0.smithi057.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-11T13:30:43.835 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/locale 2024-06-11T13:30:43.835 INFO:tasks.workunit.client.0.smithi057.stderr:++ awk '{print $1}' 2024-06-11T13:30:43.837 INFO:tasks.workunit.client.0.smithi057.stderr:+++++ readlink /proc/77877/exe 2024-06-11T13:30:43.837 INFO:tasks.workunit.client.0.smithi057.stderr:++++ basename /usr/bin/bash 2024-06-11T13:30:43.837 INFO:tasks.workunit.client.0.smithi057.stderr:+++ umask 2024-06-11T13:30:43.837 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-11T13:30:43.837 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-11T13:30:43.838 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-11T13:30:43.838 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.838 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-11T13:30:43.838 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.838 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-11T13:30:43.838 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-11T13:30:43.838 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.838 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' 0022 -eq 0 ']' 2024-06-11T13:30:43.838 INFO:tasks.workunit.client.0.smithi057.stderr:++ SHELL=/bin/bash 2024-06-11T13:30:43.838 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo delete_pools_parallel 2024-06-11T13:30:43.838 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_declare='declare -f' 2024-06-11T13:30:43.838 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_opt=-f 2024-06-11T13:30:43.838 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.838 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-11T13:30:43.839 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.839 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-11T13:30:43.839 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export which_declare 2024-06-11T13:30:43.839 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export -f which 2024-06-11T13:30:43.839 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset i 2024-06-11T13:30:43.839 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset -f pathmunge 2024-06-11T13:30:43.839 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-11T13:30:43.839 INFO:tasks.workunit.client.0.smithi057.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-11T13:30:43.839 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.839 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-11T13:30:43.839 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.839 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-11T13:30:43.839 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.839 INFO:tasks.workunit.client.0.smithi057.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.840 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.840 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-11T13:30:43.840 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.840 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-11T13:30:43.840 INFO:tasks.workunit.client.0.smithi057.stderr:+ export PATH 2024-06-11T13:30:43.840 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.840 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-11T13:30:43.840 INFO:tasks.workunit.client.0.smithi057.stderr:+ unset rc 2024-06-11T13:30:43.840 INFO:tasks.workunit.client.0.smithi057.stdout:test delete_pools_parallel on pid 78054 2024-06-11T13:30:43.846 INFO:tasks.workunit.client.0.smithi057.stderr:+ ff=delete_pools_parallel 2024-06-11T13:30:43.846 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=78054 2024-06-11T13:30:43.846 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'test delete_pools_parallel on pid 78054' 2024-06-11T13:30:43.846 INFO:tasks.workunit.client.0.smithi057.stderr:+ pids[$f]=78054 2024-06-11T13:30:43.846 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph_test_rados_api_service_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service_pp.xml 2024-06-11T13:30:43.846 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-06-11T13:30:43.846 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' 1 -eq 1 ']' 2024-06-11T13:30:43.846 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/locale 2024-06-11T13:30:43.846 INFO:tasks.workunit.client.0.smithi057.stderr:++ printf %25s cls 2024-06-11T13:30:43.846 INFO:tasks.workunit.client.0.smithi057.stderr:+ r=' cls' 2024-06-11T13:30:43.847 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.847 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-11T13:30:43.847 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.847 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-11T13:30:43.847 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-11T13:30:43.847 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.847 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-11T13:30:43.847 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-11T13:30:43.847 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'grep=grep --color=auto' 2024-06-11T13:30:43.847 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-11T13:30:43.847 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.847 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-11T13:30:43.847 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-11T13:30:43.847 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-11T13:30:43.848 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.848 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-11T13:30:43.848 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.848 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorls.sh 2024-06-11T13:30:43.848 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' '!' -t 0 ']' 2024-06-11T13:30:43.848 INFO:tasks.workunit.client.0.smithi057.stderr:+++ return 2024-06-11T13:30:43.848 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.848 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-11T13:30:43.848 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.848 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-11T13:30:43.848 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export S_COLORS=auto 2024-06-11T13:30:43.848 INFO:tasks.workunit.client.0.smithi057.stderr:+++ S_COLORS=auto 2024-06-11T13:30:43.848 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.848 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-11T13:30:43.849 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.849 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-11T13:30:43.849 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.849 INFO:tasks.workunit.client.0.smithi057.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-11T13:30:43.849 INFO:tasks.workunit.client.0.smithi057.stderr:++++ export LANG=en_US.UTF-8 2024-06-11T13:30:43.849 INFO:tasks.workunit.client.0.smithi057.stderr:++++ LANG=en_US.UTF-8 2024-06-11T13:30:43.849 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.849 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-11T13:30:43.849 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.849 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG=en_US.UTF-8 2024-06-11T13:30:43.849 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup config 2024-06-11T13:30:43.849 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.849 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.849 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' dumb = linux ']' 2024-06-11T13:30:43.850 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.850 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-11T13:30:43.850 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.850 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/less.sh 2024-06-11T13:30:43.850 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-11T13:30:43.850 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.850 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-11T13:30:43.850 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.850 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/which2.sh 2024-06-11T13:30:43.850 INFO:tasks.workunit.client.0.smithi057.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-11T13:30:43.850 INFO:tasks.workunit.client.0.smithi057.stderr:+++++ readlink /proc/77921/exe 2024-06-11T13:30:43.850 INFO:tasks.workunit.client.0.smithi057.stderr:+ bash -o pipefail -exc 'ceph_test_rados_delete_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/delete_pools_parallel.xml 2>&1 | tee ceph_test_rados_delete_pools_parallel.log | sed "s/^/ delete_pools_parallel: /"' 2024-06-11T13:30:43.850 INFO:tasks.workunit.client.0.smithi057.stderr:++++ basename /usr/bin/bash 2024-06-11T13:30:43.850 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-11T13:30:43.851 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-11T13:30:43.851 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-11T13:30:43.851 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.851 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-11T13:30:43.851 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.851 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-11T13:30:43.851 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-11T13:30:43.851 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.851 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-11T13:30:43.851 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.851 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/gawk.sh 2024-06-11T13:30:43.851 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-11T13:30:43.851 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-11T13:30:43.851 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-11T13:30:43.852 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.852 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.852 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-11T13:30:43.852 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-11T13:30:43.852 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.852 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.852 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/lang.sh 2024-06-11T13:30:43.852 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-11T13:30:43.852 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-11T13:30:43.852 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup 2024-06-11T13:30:43.852 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.852 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/locale 2024-06-11T13:30:43.852 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo cls 2024-06-11T13:30:43.852 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -f /etc/bashrc ']' 2024-06-11T13:30:43.853 INFO:tasks.workunit.client.0.smithi057.stderr:+ . /etc/bashrc 2024-06-11T13:30:43.853 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -z '' ']' 2024-06-11T13:30:43.853 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.853 INFO:tasks.workunit.client.0.smithi057.stderr:++ BASHRCSOURCED=Y 2024-06-11T13:30:43.853 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.853 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-11T13:30:43.853 INFO:tasks.workunit.client.0.smithi057.stderr:++ shopt -q login_shell 2024-06-11T13:30:43.853 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.853 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-11T13:30:43.853 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-11T13:30:43.853 INFO:tasks.workunit.client.0.smithi057.stderr:+ tee ceph_test_rados_api_service_pp.log 2024-06-11T13:30:43.853 INFO:tasks.workunit.client.0.smithi057.stderr:++ awk '{print $1}' 2024-06-11T13:30:43.853 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-11T13:30:43.853 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_declare='declare -f' 2024-06-11T13:30:43.853 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_opt=-f 2024-06-11T13:30:43.854 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export which_declare 2024-06-11T13:30:43.854 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export -f which 2024-06-11T13:30:43.854 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset i 2024-06-11T13:30:43.854 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset -f pathmunge 2024-06-11T13:30:43.854 INFO:tasks.workunit.client.0.smithi057.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-11T13:30:43.854 INFO:tasks.workunit.client.0.smithi057.stderr:+ export PATH 2024-06-11T13:30:43.854 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-11T13:30:43.854 INFO:tasks.workunit.client.0.smithi057.stderr:+ unset rc 2024-06-11T13:30:43.854 INFO:tasks.workunit.client.0.smithi057.stderr:+ sed 's/^/ api_c_read_operations: /' 2024-06-11T13:30:43.854 INFO:tasks.workunit.client.0.smithi057.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-11T13:30:43.854 INFO:tasks.workunit.client.0.smithi057.stderr:++++ export LANG=en_US.UTF-8 2024-06-11T13:30:43.854 INFO:tasks.workunit.client.0.smithi057.stderr:++++ LANG=en_US.UTF-8 2024-06-11T13:30:43.854 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.854 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-11T13:30:43.855 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.855 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG=en_US.UTF-8 2024-06-11T13:30:43.855 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup config 2024-06-11T13:30:43.855 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.855 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.855 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' dumb = linux ']' 2024-06-11T13:30:43.855 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.855 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-11T13:30:43.855 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.855 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/less.sh 2024-06-11T13:30:43.855 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-11T13:30:43.855 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.855 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-11T13:30:43.855 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.856 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/which2.sh 2024-06-11T13:30:43.856 INFO:tasks.workunit.client.0.smithi057.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-11T13:30:43.856 INFO:tasks.workunit.client.0.smithi057.stderr:+++++ readlink /proc/77908/exe 2024-06-11T13:30:43.856 INFO:tasks.workunit.client.0.smithi057.stderr:+ tee ceph_test_rados_api_c_read_operations.log 2024-06-11T13:30:43.856 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph_test_rados_api_c_read_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_read_operations.xml 2024-06-11T13:30:43.856 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-11T13:30:43.856 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-11T13:30:43.856 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-11T13:30:43.856 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.856 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-11T13:30:43.856 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.856 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-11T13:30:43.856 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-11T13:30:43.856 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.857 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-11T13:30:43.857 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.857 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/gawk.sh 2024-06-11T13:30:43.857 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.857 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-11T13:30:43.857 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.857 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/lang.sh 2024-06-11T13:30:43.857 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup 2024-06-11T13:30:43.857 INFO:tasks.workunit.client.0.smithi057.stderr:+ sed 's/^/ api_service_pp: /' 2024-06-11T13:30:43.857 INFO:tasks.workunit.client.0.smithi057.stderr:+++ umask 2024-06-11T13:30:43.858 INFO:tasks.workunit.client.0.smithi057.stdout:test cls on pid 78106 2024-06-11T13:30:43.861 INFO:tasks.workunit.client.0.smithi057.stderr:++++ basename /usr/bin/bash 2024-06-11T13:30:43.861 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' 0022 -eq 0 ']' 2024-06-11T13:30:43.861 INFO:tasks.workunit.client.0.smithi057.stderr:++ SHELL=/bin/bash 2024-06-11T13:30:43.861 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.861 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-11T13:30:43.861 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.861 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-11T13:30:43.862 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-11T13:30:43.862 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.862 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-11T13:30:43.862 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.862 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-11T13:30:43.862 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.862 INFO:tasks.workunit.client.0.smithi057.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.862 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.862 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-11T13:30:43.862 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.862 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-11T13:30:43.862 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.863 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_declare='declare -f' 2024-06-11T13:30:43.863 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_opt=-f 2024-06-11T13:30:43.863 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export which_declare 2024-06-11T13:30:43.863 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export -f which 2024-06-11T13:30:43.863 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset i 2024-06-11T13:30:43.863 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset -f pathmunge 2024-06-11T13:30:43.863 INFO:tasks.workunit.client.0.smithi057.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-11T13:30:43.863 INFO:tasks.workunit.client.0.smithi057.stderr:+ export PATH 2024-06-11T13:30:43.863 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/locale 2024-06-11T13:30:43.863 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-11T13:30:43.863 INFO:tasks.workunit.client.0.smithi057.stderr:+ unset rc 2024-06-11T13:30:43.863 INFO:tasks.workunit.client.0.smithi057.stderr:+ ff=cls 2024-06-11T13:30:43.863 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=78106 2024-06-11T13:30:43.863 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'test cls on pid 78106' 2024-06-11T13:30:43.864 INFO:tasks.workunit.client.0.smithi057.stderr:+ pids[$f]=78106 2024-06-11T13:30:43.864 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-06-11T13:30:43.864 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' 1 -eq 1 ']' 2024-06-11T13:30:43.864 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-11T13:30:43.864 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.864 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-11T13:30:43.864 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.864 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-11T13:30:43.864 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-11T13:30:43.864 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-11T13:30:43.864 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-11T13:30:43.864 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-11T13:30:43.864 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.864 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-11T13:30:43.865 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph_test_rados_api_c_write_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_write_operations.xml 2024-06-11T13:30:43.865 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.865 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-11T13:30:43.865 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-11T13:30:43.865 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.865 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-11T13:30:43.865 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.865 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/gawk.sh 2024-06-11T13:30:43.865 INFO:tasks.workunit.client.0.smithi057.stderr:+ sed 's/^/ api_c_write_operations: /' 2024-06-11T13:30:43.865 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.865 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-11T13:30:43.865 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.865 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/lang.sh 2024-06-11T13:30:43.865 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup 2024-06-11T13:30:43.866 INFO:tasks.workunit.client.0.smithi057.stderr:+ tee ceph_test_rados_api_c_write_operations.log 2024-06-11T13:30:43.866 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-11T13:30:43.866 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/locale 2024-06-11T13:30:43.866 INFO:tasks.workunit.client.0.smithi057.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-11T13:30:43.866 INFO:tasks.workunit.client.0.smithi057.stderr:++++ export LANG=en_US.UTF-8 2024-06-11T13:30:43.866 INFO:tasks.workunit.client.0.smithi057.stderr:++++ LANG=en_US.UTF-8 2024-06-11T13:30:43.866 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.866 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-11T13:30:43.866 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.866 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG=en_US.UTF-8 2024-06-11T13:30:43.866 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup config 2024-06-11T13:30:43.866 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.866 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.867 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' dumb = linux ']' 2024-06-11T13:30:43.867 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.867 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-11T13:30:43.867 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.867 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/less.sh 2024-06-11T13:30:43.867 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-11T13:30:43.867 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.867 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-11T13:30:43.867 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.867 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/which2.sh 2024-06-11T13:30:43.867 INFO:tasks.workunit.client.0.smithi057.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-11T13:30:43.867 INFO:tasks.workunit.client.0.smithi057.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_cls 2>&1 | tee ceph_test_neorados_cls.log | sed "s/^/ cls: /"' 2024-06-11T13:30:43.867 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.867 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-11T13:30:43.868 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.868 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-11T13:30:43.868 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-11T13:30:43.868 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-11T13:30:43.868 INFO:tasks.workunit.client.0.smithi057.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-11T13:30:43.868 INFO:tasks.workunit.client.0.smithi057.stderr:++++ export LANG=en_US.UTF-8 2024-06-11T13:30:43.868 INFO:tasks.workunit.client.0.smithi057.stderr:++++ LANG=en_US.UTF-8 2024-06-11T13:30:43.868 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.868 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-11T13:30:43.868 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.868 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG=en_US.UTF-8 2024-06-11T13:30:43.868 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup config 2024-06-11T13:30:43.868 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.868 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.869 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' dumb = linux ']' 2024-06-11T13:30:43.869 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.869 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-11T13:30:43.869 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.869 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -f /etc/bashrc ']' 2024-06-11T13:30:43.869 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/less.sh 2024-06-11T13:30:43.869 INFO:tasks.workunit.client.0.smithi057.stderr:+ . /etc/bashrc 2024-06-11T13:30:43.869 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-11T13:30:43.869 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.869 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-11T13:30:43.869 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.869 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/which2.sh 2024-06-11T13:30:43.869 INFO:tasks.workunit.client.0.smithi057.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-11T13:30:43.869 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -z '' ']' 2024-06-11T13:30:43.870 INFO:tasks.workunit.client.0.smithi057.stderr:++ BASHRCSOURCED=Y 2024-06-11T13:30:43.870 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.870 INFO:tasks.workunit.client.0.smithi057.stderr:++ shopt -q login_shell 2024-06-11T13:30:43.870 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'grep=grep --color=auto' 2024-06-11T13:30:43.870 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-11T13:30:43.870 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-11T13:30:43.870 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.870 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-11T13:30:43.870 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.870 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorls.sh 2024-06-11T13:30:43.870 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' '!' -t 0 ']' 2024-06-11T13:30:43.870 INFO:tasks.workunit.client.0.smithi057.stderr:+++ return 2024-06-11T13:30:43.870 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.870 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-11T13:30:43.871 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.871 INFO:tasks.workunit.client.0.smithi057.stderr:++ printf %25s cmd 2024-06-11T13:30:43.871 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-11T13:30:43.871 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export S_COLORS=auto 2024-06-11T13:30:43.871 INFO:tasks.workunit.client.0.smithi057.stderr:+++ S_COLORS=auto 2024-06-11T13:30:43.871 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.871 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-11T13:30:43.871 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.871 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-11T13:30:43.871 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.871 INFO:tasks.workunit.client.0.smithi057.stderr:+ r=' cmd' 2024-06-11T13:30:43.871 INFO:tasks.workunit.client.0.smithi057.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-11T13:30:43.871 INFO:tasks.workunit.client.0.smithi057.stderr:++++ export LANG=en_US.UTF-8 2024-06-11T13:30:43.872 INFO:tasks.workunit.client.0.smithi057.stderr:++++ LANG=en_US.UTF-8 2024-06-11T13:30:43.872 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.872 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-11T13:30:43.872 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.872 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG=en_US.UTF-8 2024-06-11T13:30:43.872 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup config 2024-06-11T13:30:43.872 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.872 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.872 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' dumb = linux ']' 2024-06-11T13:30:43.872 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.872 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-11T13:30:43.872 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.872 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/less.sh 2024-06-11T13:30:43.872 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-11T13:30:43.872 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.873 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-11T13:30:43.873 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.873 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/which2.sh 2024-06-11T13:30:43.873 INFO:tasks.workunit.client.0.smithi057.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-11T13:30:43.873 INFO:tasks.workunit.client.0.smithi057.stderr:+++ umask 2024-06-11T13:30:43.873 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' 0022 -eq 0 ']' 2024-06-11T13:30:43.873 INFO:tasks.workunit.client.0.smithi057.stderr:++ SHELL=/bin/bash 2024-06-11T13:30:43.873 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.873 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-11T13:30:43.873 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.873 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-11T13:30:43.873 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-11T13:30:43.873 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.873 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-11T13:30:43.874 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.874 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-11T13:30:43.874 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.874 INFO:tasks.workunit.client.0.smithi057.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.874 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.874 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-11T13:30:43.874 INFO:tasks.workunit.client.0.smithi057.stderr:+++++ readlink /proc/77963/exe 2024-06-11T13:30:43.874 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.874 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-11T13:30:43.874 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.874 INFO:tasks.workunit.client.0.smithi057.stderr:++++ basename /usr/bin/bash 2024-06-11T13:30:43.874 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_declare='declare -f' 2024-06-11T13:30:43.874 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_opt=-f 2024-06-11T13:30:43.874 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export which_declare 2024-06-11T13:30:43.875 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export -f which 2024-06-11T13:30:43.875 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset i 2024-06-11T13:30:43.875 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset -f pathmunge 2024-06-11T13:30:43.875 INFO:tasks.workunit.client.0.smithi057.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-11T13:30:43.875 INFO:tasks.workunit.client.0.smithi057.stderr:+ export PATH 2024-06-11T13:30:43.875 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-11T13:30:43.875 INFO:tasks.workunit.client.0.smithi057.stderr:+ unset rc 2024-06-11T13:30:43.876 INFO:tasks.workunit.client.0.smithi057.stderr:+ tee ceph_test_rados_api_cls_remote_reads.log 2024-06-11T13:30:43.876 INFO:tasks.workunit.client.0.smithi057.stderr:++ awk '{print $1}' 2024-06-11T13:30:43.876 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo cmd 2024-06-11T13:30:43.876 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph_test_rados_api_cls_remote_reads --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cls_remote_reads.xml 2024-06-11T13:30:43.876 INFO:tasks.workunit.client.0.smithi057.stderr:+++++ readlink /proc/77996/exe 2024-06-11T13:30:43.876 INFO:tasks.workunit.client.0.smithi057.stderr:++++ basename /usr/bin/bash 2024-06-11T13:30:43.876 INFO:tasks.workunit.client.0.smithi057.stdout:test cmd on pid 78149 2024-06-11T13:30:43.878 INFO:tasks.workunit.client.0.smithi057.stderr:+ ff=cmd 2024-06-11T13:30:43.879 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=78149 2024-06-11T13:30:43.879 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'test cmd on pid 78149' 2024-06-11T13:30:43.879 INFO:tasks.workunit.client.0.smithi057.stderr:+ pids[$f]=78149 2024-06-11T13:30:43.879 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-06-11T13:30:43.879 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' 1 -eq 1 ']' 2024-06-11T13:30:43.879 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_declare='declare -f' 2024-06-11T13:30:43.879 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_opt=-f 2024-06-11T13:30:43.879 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export which_declare 2024-06-11T13:30:43.879 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export -f which 2024-06-11T13:30:43.879 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset i 2024-06-11T13:30:43.879 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset -f pathmunge 2024-06-11T13:30:43.879 INFO:tasks.workunit.client.0.smithi057.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-11T13:30:43.879 INFO:tasks.workunit.client.0.smithi057.stderr:+ export PATH 2024-06-11T13:30:43.880 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-11T13:30:43.880 INFO:tasks.workunit.client.0.smithi057.stderr:+ unset rc 2024-06-11T13:30:43.880 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'grep=grep --color=auto' 2024-06-11T13:30:43.880 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-11T13:30:43.880 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-11T13:30:43.880 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.880 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-11T13:30:43.880 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.880 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorls.sh 2024-06-11T13:30:43.880 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' '!' -t 0 ']' 2024-06-11T13:30:43.880 INFO:tasks.workunit.client.0.smithi057.stderr:+++ return 2024-06-11T13:30:43.880 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.880 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-11T13:30:43.880 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.880 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-11T13:30:43.881 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export S_COLORS=auto 2024-06-11T13:30:43.881 INFO:tasks.workunit.client.0.smithi057.stderr:+++ S_COLORS=auto 2024-06-11T13:30:43.881 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.881 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-11T13:30:43.881 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.881 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-11T13:30:43.881 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.881 INFO:tasks.workunit.client.0.smithi057.stderr:++ printf %25s handler_error 2024-06-11T13:30:43.881 INFO:tasks.workunit.client.0.smithi057.stderr:+ r=' handler_error' 2024-06-11T13:30:43.881 INFO:tasks.workunit.client.0.smithi057.stderr:+++++ readlink /proc/77977/exe 2024-06-11T13:30:43.881 INFO:tasks.workunit.client.0.smithi057.stderr:+ sed 's/^/ open_pools_parallel: /' 2024-06-11T13:30:43.881 INFO:tasks.workunit.client.0.smithi057.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_cmd 2>&1 | tee ceph_test_neorados_cmd.log | sed "s/^/ cmd: /"' 2024-06-11T13:30:43.881 INFO:tasks.workunit.client.0.smithi057.stderr:++++ basename /usr/bin/bash 2024-06-11T13:30:43.881 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_declare='declare -f' 2024-06-11T13:30:43.882 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_opt=-f 2024-06-11T13:30:43.882 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export which_declare 2024-06-11T13:30:43.882 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export -f which 2024-06-11T13:30:43.882 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset i 2024-06-11T13:30:43.882 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset -f pathmunge 2024-06-11T13:30:43.882 INFO:tasks.workunit.client.0.smithi057.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-11T13:30:43.882 INFO:tasks.workunit.client.0.smithi057.stderr:+ export PATH 2024-06-11T13:30:43.882 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-11T13:30:43.882 INFO:tasks.workunit.client.0.smithi057.stderr:+ unset rc 2024-06-11T13:30:43.882 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph_test_rados_list_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/list_parallel.xml 2024-06-11T13:30:43.882 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-11T13:30:43.882 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-11T13:30:43.882 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-11T13:30:43.882 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.882 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-11T13:30:43.883 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.883 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-11T13:30:43.883 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-11T13:30:43.883 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.883 INFO:tasks.workunit.client.0.smithi057.stderr:+ tee ceph_test_rados_open_pools_parallel.log 2024-06-11T13:30:43.883 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph_test_rados_open_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/open_pools_parallel.xml 2024-06-11T13:30:43.883 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -f /etc/bashrc ']' 2024-06-11T13:30:43.884 INFO:tasks.workunit.client.0.smithi057.stderr:+ tee ceph_test_rados_list_parallel.log 2024-06-11T13:30:43.884 INFO:tasks.workunit.client.0.smithi057.stderr:+ . /etc/bashrc 2024-06-11T13:30:43.884 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -z '' ']' 2024-06-11T13:30:43.884 INFO:tasks.workunit.client.0.smithi057.stderr:++ BASHRCSOURCED=Y 2024-06-11T13:30:43.884 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.884 INFO:tasks.workunit.client.0.smithi057.stderr:++ shopt -q login_shell 2024-06-11T13:30:43.887 INFO:tasks.workunit.client.0.smithi057.stderr:+ sed 's/^/ api_cls_remote_reads: /' 2024-06-11T13:30:43.887 INFO:tasks.workunit.client.0.smithi057.stderr:+ sed 's/^/ list_parallel: /' 2024-06-11T13:30:43.894 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo handler_error 2024-06-11T13:30:43.894 INFO:tasks.workunit.client.0.smithi057.stderr:++ awk '{print $1}' 2024-06-11T13:30:43.897 INFO:tasks.workunit.client.0.smithi057.stderr:+++ umask 2024-06-11T13:30:43.898 INFO:tasks.workunit.client.0.smithi057.stdout:test handler_error on pid 78195 2024-06-11T13:30:43.898 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' 0022 -eq 0 ']' 2024-06-11T13:30:43.898 INFO:tasks.workunit.client.0.smithi057.stderr:++ SHELL=/bin/bash 2024-06-11T13:30:43.898 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.898 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-11T13:30:43.898 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.898 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-11T13:30:43.898 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-11T13:30:43.898 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.899 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-11T13:30:43.899 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.899 INFO:tasks.workunit.client.0.smithi057.stderr:+ ff=handler_error 2024-06-11T13:30:43.899 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-11T13:30:43.899 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.899 INFO:tasks.workunit.client.0.smithi057.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.899 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.899 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-11T13:30:43.899 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.899 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-11T13:30:43.899 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.899 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=78195 2024-06-11T13:30:43.899 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'test handler_error on pid 78195' 2024-06-11T13:30:43.899 INFO:tasks.workunit.client.0.smithi057.stderr:+ pids[$f]=78195 2024-06-11T13:30:43.900 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-06-11T13:30:43.900 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' 1 -eq 1 ']' 2024-06-11T13:30:43.900 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-11T13:30:43.900 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-11T13:30:43.900 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-11T13:30:43.900 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.900 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-11T13:30:43.900 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.900 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-11T13:30:43.900 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-11T13:30:43.901 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.902 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-11T13:30:43.902 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-11T13:30:43.902 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-11T13:30:43.902 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.902 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-11T13:30:43.902 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.902 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-11T13:30:43.902 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-11T13:30:43.902 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.902 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-11T13:30:43.902 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.902 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/gawk.sh 2024-06-11T13:30:43.903 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.903 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-11T13:30:43.903 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.903 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/lang.sh 2024-06-11T13:30:43.903 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup 2024-06-11T13:30:43.905 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-11T13:30:43.905 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-11T13:30:43.905 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-11T13:30:43.905 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.905 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-11T13:30:43.905 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.905 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-11T13:30:43.905 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-11T13:30:43.906 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.906 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-11T13:30:43.906 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.906 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/gawk.sh 2024-06-11T13:30:43.906 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.906 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-11T13:30:43.907 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.907 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/lang.sh 2024-06-11T13:30:43.907 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup 2024-06-11T13:30:43.907 INFO:tasks.workunit.client.0.smithi057.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_handler_error 2>&1 | tee ceph_test_neorados_handler_error.log | sed "s/^/ handler_error: /"' 2024-06-11T13:30:43.907 INFO:tasks.workunit.client.0.smithi057.stderr:++ printf %25s io 2024-06-11T13:30:43.907 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/locale 2024-06-11T13:30:43.908 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'grep=grep --color=auto' 2024-06-11T13:30:43.908 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-11T13:30:43.908 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-11T13:30:43.909 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.909 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-11T13:30:43.909 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.909 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorls.sh 2024-06-11T13:30:43.909 INFO:tasks.workunit.client.0.smithi057.stderr:+ r=' io' 2024-06-11T13:30:43.909 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' '!' -t 0 ']' 2024-06-11T13:30:43.909 INFO:tasks.workunit.client.0.smithi057.stderr:+++ return 2024-06-11T13:30:43.909 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.909 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-11T13:30:43.909 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.909 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-11T13:30:43.909 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export S_COLORS=auto 2024-06-11T13:30:43.909 INFO:tasks.workunit.client.0.smithi057.stderr:+++ S_COLORS=auto 2024-06-11T13:30:43.910 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.910 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-11T13:30:43.910 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.910 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-11T13:30:43.910 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.910 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -f /etc/bashrc ']' 2024-06-11T13:30:43.910 INFO:tasks.workunit.client.0.smithi057.stderr:+ . /etc/bashrc 2024-06-11T13:30:43.910 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -z '' ']' 2024-06-11T13:30:43.910 INFO:tasks.workunit.client.0.smithi057.stderr:++ BASHRCSOURCED=Y 2024-06-11T13:30:43.910 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.910 INFO:tasks.workunit.client.0.smithi057.stderr:++ shopt -q login_shell 2024-06-11T13:30:43.911 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/locale 2024-06-11T13:30:43.914 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.914 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-11T13:30:43.914 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.914 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-11T13:30:43.914 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-11T13:30:43.917 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-11T13:30:43.917 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-11T13:30:43.917 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-11T13:30:43.917 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.917 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-11T13:30:43.917 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.917 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-11T13:30:43.917 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-11T13:30:43.917 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.918 INFO:tasks.workunit.client.0.smithi057.stderr:++ awk '{print $1}' 2024-06-11T13:30:43.919 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.920 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-11T13:30:43.920 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.920 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-11T13:30:43.920 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-11T13:30:43.923 INFO:tasks.workunit.client.0.smithi057.stderr:+++ umask 2024-06-11T13:30:43.923 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' 0022 -eq 0 ']' 2024-06-11T13:30:43.923 INFO:tasks.workunit.client.0.smithi057.stderr:++ SHELL=/bin/bash 2024-06-11T13:30:43.923 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.923 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo io 2024-06-11T13:30:43.923 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-11T13:30:43.923 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.923 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-11T13:30:43.923 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-11T13:30:43.924 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.924 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-11T13:30:43.924 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.924 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-11T13:30:43.924 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.924 INFO:tasks.workunit.client.0.smithi057.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.924 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.924 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-11T13:30:43.924 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.924 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-11T13:30:43.924 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.925 INFO:tasks.workunit.client.0.smithi057.stdout:test io on pid 78271 2024-06-11T13:30:43.927 INFO:tasks.workunit.client.0.smithi057.stderr:+ ff=io 2024-06-11T13:30:43.927 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=78271 2024-06-11T13:30:43.927 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'test io on pid 78271' 2024-06-11T13:30:43.927 INFO:tasks.workunit.client.0.smithi057.stderr:+ pids[$f]=78271 2024-06-11T13:30:43.927 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-06-11T13:30:43.927 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' 1 -eq 1 ']' 2024-06-11T13:30:43.927 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-11T13:30:43.927 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-11T13:30:43.927 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-11T13:30:43.927 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.927 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-11T13:30:43.928 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.928 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-11T13:30:43.928 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-11T13:30:43.928 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.928 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-11T13:30:43.928 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.928 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/gawk.sh 2024-06-11T13:30:43.928 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.928 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-11T13:30:43.928 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.928 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/lang.sh 2024-06-11T13:30:43.928 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup 2024-06-11T13:30:43.928 INFO:tasks.workunit.client.0.smithi057.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_io 2>&1 | tee ceph_test_neorados_io.log | sed "s/^/ io: /"' 2024-06-11T13:30:43.928 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-11T13:30:43.928 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -f /etc/bashrc ']' 2024-06-11T13:30:43.929 INFO:tasks.workunit.client.0.smithi057.stderr:+ . /etc/bashrc 2024-06-11T13:30:43.929 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -z '' ']' 2024-06-11T13:30:43.929 INFO:tasks.workunit.client.0.smithi057.stderr:++ BASHRCSOURCED=Y 2024-06-11T13:30:43.929 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.929 INFO:tasks.workunit.client.0.smithi057.stderr:++ shopt -q login_shell 2024-06-11T13:30:43.929 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-11T13:30:43.930 INFO:tasks.workunit.client.0.smithi057.stderr:++ printf %25s ec_io 2024-06-11T13:30:43.930 INFO:tasks.workunit.client.0.smithi057.stderr:+ r=' ec_io' 2024-06-11T13:30:43.932 INFO:tasks.workunit.client.0.smithi057.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-11T13:30:43.932 INFO:tasks.workunit.client.0.smithi057.stderr:++++ export LANG=en_US.UTF-8 2024-06-11T13:30:43.932 INFO:tasks.workunit.client.0.smithi057.stderr:++++ LANG=en_US.UTF-8 2024-06-11T13:30:43.932 INFO:tasks.workunit.client.0.smithi057.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-11T13:30:43.932 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.932 INFO:tasks.workunit.client.0.smithi057.stderr:++++ export LANG=en_US.UTF-8 2024-06-11T13:30:43.932 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-11T13:30:43.932 INFO:tasks.workunit.client.0.smithi057.stderr:++++ LANG=en_US.UTF-8 2024-06-11T13:30:43.932 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.933 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.933 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG=en_US.UTF-8 2024-06-11T13:30:43.933 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-11T13:30:43.933 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup config 2024-06-11T13:30:43.933 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.933 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG=en_US.UTF-8 2024-06-11T13:30:43.933 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.933 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup config 2024-06-11T13:30:43.933 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.933 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.933 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' dumb = linux ']' 2024-06-11T13:30:43.933 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.933 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.933 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-11T13:30:43.934 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' dumb = linux ']' 2024-06-11T13:30:43.934 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.934 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/less.sh 2024-06-11T13:30:43.934 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.934 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-11T13:30:43.934 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.934 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/less.sh 2024-06-11T13:30:43.934 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-11T13:30:43.934 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.934 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-11T13:30:43.934 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-11T13:30:43.934 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.935 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/which2.sh 2024-06-11T13:30:43.935 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.935 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-11T13:30:43.935 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.935 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/which2.sh 2024-06-11T13:30:43.935 INFO:tasks.workunit.client.0.smithi057.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-11T13:30:43.935 INFO:tasks.workunit.client.0.smithi057.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-11T13:30:43.937 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'grep=grep --color=auto' 2024-06-11T13:30:43.937 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-11T13:30:43.937 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-11T13:30:43.937 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.937 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-11T13:30:43.937 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.937 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorls.sh 2024-06-11T13:30:43.938 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' '!' -t 0 ']' 2024-06-11T13:30:43.938 INFO:tasks.workunit.client.0.smithi057.stderr:+++ return 2024-06-11T13:30:43.938 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.938 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-11T13:30:43.938 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.938 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-11T13:30:43.938 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export S_COLORS=auto 2024-06-11T13:30:43.938 INFO:tasks.workunit.client.0.smithi057.stderr:+++ S_COLORS=auto 2024-06-11T13:30:43.938 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.938 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-11T13:30:43.938 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.938 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-11T13:30:43.939 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.939 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/locale 2024-06-11T13:30:43.939 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo ec_io 2024-06-11T13:30:43.939 INFO:tasks.workunit.client.0.smithi057.stderr:++ awk '{print $1}' 2024-06-11T13:30:43.940 INFO:tasks.workunit.client.0.smithi057.stderr:+++++ readlink /proc/78054/exe 2024-06-11T13:30:43.942 INFO:tasks.workunit.client.0.smithi057.stderr:++++ basename /usr/bin/bash 2024-06-11T13:30:43.942 INFO:tasks.workunit.client.0.smithi057.stderr:+++ umask 2024-06-11T13:30:43.942 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' 0022 -eq 0 ']' 2024-06-11T13:30:43.942 INFO:tasks.workunit.client.0.smithi057.stderr:++ SHELL=/bin/bash 2024-06-11T13:30:43.943 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.943 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-11T13:30:43.943 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.943 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-11T13:30:43.943 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-11T13:30:43.943 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.943 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-11T13:30:43.943 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.943 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-11T13:30:43.943 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.943 INFO:tasks.workunit.client.0.smithi057.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.943 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.943 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-11T13:30:43.944 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.944 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-11T13:30:43.944 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.944 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_declare='declare -f' 2024-06-11T13:30:43.944 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_opt=-f 2024-06-11T13:30:43.944 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export which_declare 2024-06-11T13:30:43.944 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export -f which 2024-06-11T13:30:43.945 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset i 2024-06-11T13:30:43.945 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset -f pathmunge 2024-06-11T13:30:43.945 INFO:tasks.workunit.client.0.smithi057.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-11T13:30:43.945 INFO:tasks.workunit.client.0.smithi057.stderr:+ export PATH 2024-06-11T13:30:43.945 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-11T13:30:43.945 INFO:tasks.workunit.client.0.smithi057.stderr:+ unset rc 2024-06-11T13:30:43.945 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph_test_rados_delete_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/delete_pools_parallel.xml 2024-06-11T13:30:43.945 INFO:tasks.workunit.client.0.smithi057.stderr:+ tee ceph_test_rados_delete_pools_parallel.log 2024-06-11T13:30:43.945 INFO:tasks.workunit.client.0.smithi057.stdout:test ec_io on pid 78321 2024-06-11T13:30:43.946 INFO:tasks.workunit.client.0.smithi057.stderr:+ ff=ec_io 2024-06-11T13:30:43.946 INFO:tasks.workunit.client.0.smithi057.stderr:+ sed 's/^/ delete_pools_parallel: /' 2024-06-11T13:30:43.946 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=78321 2024-06-11T13:30:43.946 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'test ec_io on pid 78321' 2024-06-11T13:30:43.946 INFO:tasks.workunit.client.0.smithi057.stderr:+ pids[$f]=78321 2024-06-11T13:30:43.946 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-06-11T13:30:43.946 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' 1 -eq 1 ']' 2024-06-11T13:30:43.946 INFO:tasks.workunit.client.0.smithi057.stderr:+++++ readlink /proc/78106/exe 2024-06-11T13:30:43.947 INFO:tasks.workunit.client.0.smithi057.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_ec_io 2>&1 | tee ceph_test_neorados_ec_io.log | sed "s/^/ ec_io: /"' 2024-06-11T13:30:43.947 INFO:tasks.workunit.client.0.smithi057.stderr:++ printf %25s list 2024-06-11T13:30:43.948 INFO:tasks.workunit.client.0.smithi057.stderr:+ r=' list' 2024-06-11T13:30:43.949 INFO:tasks.workunit.client.0.smithi057.stderr:++ awk '{print $1}' 2024-06-11T13:30:43.951 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-11T13:30:43.951 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-11T13:30:43.951 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-11T13:30:43.951 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.951 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-11T13:30:43.951 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.951 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-11T13:30:43.951 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-11T13:30:43.951 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.953 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.953 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-11T13:30:43.953 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.954 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-11T13:30:43.954 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-11T13:30:43.954 INFO:tasks.workunit.client.0.smithi057.stderr:++++ basename /usr/bin/bash 2024-06-11T13:30:43.955 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -f /etc/bashrc ']' 2024-06-11T13:30:43.955 INFO:tasks.workunit.client.0.smithi057.stderr:+ . /etc/bashrc 2024-06-11T13:30:43.955 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -z '' ']' 2024-06-11T13:30:43.955 INFO:tasks.workunit.client.0.smithi057.stderr:++ BASHRCSOURCED=Y 2024-06-11T13:30:43.955 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.955 INFO:tasks.workunit.client.0.smithi057.stderr:++ shopt -q login_shell 2024-06-11T13:30:43.956 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'grep=grep --color=auto' 2024-06-11T13:30:43.956 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_declare='declare -f' 2024-06-11T13:30:43.956 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_opt=-f 2024-06-11T13:30:43.956 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-11T13:30:43.956 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-11T13:30:43.956 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export which_declare 2024-06-11T13:30:43.956 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export -f which 2024-06-11T13:30:43.956 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.956 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-11T13:30:43.956 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.956 INFO:tasks.workunit.client.0.smithi057.stderr:+++ umask 2024-06-11T13:30:43.956 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorls.sh 2024-06-11T13:30:43.957 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset i 2024-06-11T13:30:43.957 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset -f pathmunge 2024-06-11T13:30:43.957 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' '!' -t 0 ']' 2024-06-11T13:30:43.957 INFO:tasks.workunit.client.0.smithi057.stderr:+++ return 2024-06-11T13:30:43.957 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.957 INFO:tasks.workunit.client.0.smithi057.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-11T13:30:43.957 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-11T13:30:43.957 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.957 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-11T13:30:43.957 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export S_COLORS=auto 2024-06-11T13:30:43.957 INFO:tasks.workunit.client.0.smithi057.stderr:+++ S_COLORS=auto 2024-06-11T13:30:43.957 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.957 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-11T13:30:43.957 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.957 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-11T13:30:43.958 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' 0022 -eq 0 ']' 2024-06-11T13:30:43.958 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.958 INFO:tasks.workunit.client.0.smithi057.stderr:++ SHELL=/bin/bash 2024-06-11T13:30:43.958 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.958 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-11T13:30:43.958 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.958 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-11T13:30:43.959 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-11T13:30:43.959 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.959 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-11T13:30:43.959 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.959 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo list 2024-06-11T13:30:43.959 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-11T13:30:43.959 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.959 INFO:tasks.workunit.client.0.smithi057.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.959 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.959 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-11T13:30:43.959 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.959 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-11T13:30:43.960 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.960 INFO:tasks.workunit.client.0.smithi057.stderr:+ export PATH 2024-06-11T13:30:43.960 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-11T13:30:43.960 INFO:tasks.workunit.client.0.smithi057.stderr:+ unset rc 2024-06-11T13:30:43.960 INFO:tasks.workunit.client.0.smithi057.stderr:+ ff=list 2024-06-11T13:30:43.960 INFO:tasks.workunit.client.0.smithi057.stdout:test list on pid 78361 2024-06-11T13:30:43.961 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=78361 2024-06-11T13:30:43.961 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'test list on pid 78361' 2024-06-11T13:30:43.961 INFO:tasks.workunit.client.0.smithi057.stderr:+ pids[$f]=78361 2024-06-11T13:30:43.961 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-06-11T13:30:43.962 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' 1 -eq 1 ']' 2024-06-11T13:30:43.962 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-11T13:30:43.962 INFO:tasks.workunit.client.0.smithi057.stderr:+ tee ceph_test_neorados_cls.log 2024-06-11T13:30:43.962 INFO:tasks.workunit.client.0.smithi057.stderr:++ printf %25s ec_list 2024-06-11T13:30:43.962 INFO:tasks.workunit.client.0.smithi057.stderr:+ r=' ec_list' 2024-06-11T13:30:43.962 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-11T13:30:43.962 INFO:tasks.workunit.client.0.smithi057.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-11T13:30:43.962 INFO:tasks.workunit.client.0.smithi057.stderr:++++ export LANG=en_US.UTF-8 2024-06-11T13:30:43.962 INFO:tasks.workunit.client.0.smithi057.stderr:++++ LANG=en_US.UTF-8 2024-06-11T13:30:43.962 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-11T13:30:43.962 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-11T13:30:43.962 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.962 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-11T13:30:43.962 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.963 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-11T13:30:43.963 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.963 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG=en_US.UTF-8 2024-06-11T13:30:43.963 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.963 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-11T13:30:43.963 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup config 2024-06-11T13:30:43.963 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.963 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-11T13:30:43.963 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.963 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-11T13:30:43.963 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.963 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/gawk.sh 2024-06-11T13:30:43.963 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.963 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' dumb = linux ']' 2024-06-11T13:30:43.964 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.964 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-11T13:30:43.964 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.964 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.964 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-11T13:30:43.964 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/lang.sh 2024-06-11T13:30:43.964 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.964 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/less.sh 2024-06-11T13:30:43.964 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup 2024-06-11T13:30:43.964 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph_test_neorados_cls 2024-06-11T13:30:43.964 INFO:tasks.workunit.client.0.smithi057.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_list 2>&1 | tee ceph_test_neorados_list.log | sed "s/^/ list: /"' 2024-06-11T13:30:43.964 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-11T13:30:43.964 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.964 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-11T13:30:43.964 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.965 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/which2.sh 2024-06-11T13:30:43.965 INFO:tasks.workunit.client.0.smithi057.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-11T13:30:43.966 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/locale 2024-06-11T13:30:43.966 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -f /etc/bashrc ']' 2024-06-11T13:30:43.966 INFO:tasks.workunit.client.0.smithi057.stderr:+ . /etc/bashrc 2024-06-11T13:30:43.966 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -z '' ']' 2024-06-11T13:30:43.966 INFO:tasks.workunit.client.0.smithi057.stderr:++ BASHRCSOURCED=Y 2024-06-11T13:30:43.966 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.966 INFO:tasks.workunit.client.0.smithi057.stderr:++ shopt -q login_shell 2024-06-11T13:30:43.966 INFO:tasks.workunit.client.0.smithi057.stderr:+++ umask 2024-06-11T13:30:43.966 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' 0022 -eq 0 ']' 2024-06-11T13:30:43.967 INFO:tasks.workunit.client.0.smithi057.stderr:++ SHELL=/bin/bash 2024-06-11T13:30:43.967 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.967 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-11T13:30:43.967 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.967 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-11T13:30:43.967 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-11T13:30:43.967 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.967 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-11T13:30:43.967 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.967 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-11T13:30:43.967 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.967 INFO:tasks.workunit.client.0.smithi057.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.967 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.968 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-11T13:30:43.968 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.968 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-11T13:30:43.968 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.968 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.968 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-11T13:30:43.968 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.968 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-11T13:30:43.968 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-11T13:30:43.971 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-11T13:30:43.971 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-11T13:30:43.971 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-11T13:30:43.971 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.971 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-11T13:30:43.972 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.972 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-11T13:30:43.972 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-11T13:30:43.972 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.972 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'grep=grep --color=auto' 2024-06-11T13:30:43.972 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-11T13:30:43.972 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-11T13:30:43.972 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.972 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-11T13:30:43.972 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.972 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorls.sh 2024-06-11T13:30:43.972 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' '!' -t 0 ']' 2024-06-11T13:30:43.972 INFO:tasks.workunit.client.0.smithi057.stderr:+++ return 2024-06-11T13:30:43.972 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.973 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-11T13:30:43.973 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.973 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-11T13:30:43.973 INFO:tasks.workunit.client.0.smithi057.stderr:++ awk '{print $1}' 2024-06-11T13:30:43.973 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export S_COLORS=auto 2024-06-11T13:30:43.973 INFO:tasks.workunit.client.0.smithi057.stderr:+++ S_COLORS=auto 2024-06-11T13:30:43.973 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.973 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-11T13:30:43.973 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.973 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-11T13:30:43.973 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.974 INFO:tasks.workunit.client.0.smithi057.stderr:+ sed 's/^/ cls: /' 2024-06-11T13:30:43.974 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'grep=grep --color=auto' 2024-06-11T13:30:43.974 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-11T13:30:43.974 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-11T13:30:43.974 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-11T13:30:43.974 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.974 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-11T13:30:43.974 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.974 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorls.sh 2024-06-11T13:30:43.975 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' '!' -t 0 ']' 2024-06-11T13:30:43.975 INFO:tasks.workunit.client.0.smithi057.stderr:+++ return 2024-06-11T13:30:43.975 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.975 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-11T13:30:43.975 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.975 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-11T13:30:43.975 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export S_COLORS=auto 2024-06-11T13:30:43.975 INFO:tasks.workunit.client.0.smithi057.stderr:+++ S_COLORS=auto 2024-06-11T13:30:43.975 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.975 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-11T13:30:43.975 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.975 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-11T13:30:43.975 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.975 INFO:tasks.workunit.client.0.smithi057.stderr:+++++ readlink /proc/78149/exe 2024-06-11T13:30:43.976 INFO:tasks.workunit.client.0.smithi057.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-11T13:30:43.976 INFO:tasks.workunit.client.0.smithi057.stderr:++++ export LANG=en_US.UTF-8 2024-06-11T13:30:43.976 INFO:tasks.workunit.client.0.smithi057.stderr:++++ LANG=en_US.UTF-8 2024-06-11T13:30:43.976 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.976 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-11T13:30:43.976 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.976 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG=en_US.UTF-8 2024-06-11T13:30:43.977 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup config 2024-06-11T13:30:43.977 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.977 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.977 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' dumb = linux ']' 2024-06-11T13:30:43.977 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.977 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-11T13:30:43.977 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.977 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/less.sh 2024-06-11T13:30:43.977 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-11T13:30:43.977 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.977 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-11T13:30:43.978 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.978 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/which2.sh 2024-06-11T13:30:43.978 INFO:tasks.workunit.client.0.smithi057.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-11T13:30:43.978 INFO:tasks.workunit.client.0.smithi057.stderr:++++ basename /usr/bin/bash 2024-06-11T13:30:43.978 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_declare='declare -f' 2024-06-11T13:30:43.978 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_opt=-f 2024-06-11T13:30:43.979 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export which_declare 2024-06-11T13:30:43.979 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export -f which 2024-06-11T13:30:43.979 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset i 2024-06-11T13:30:43.979 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset -f pathmunge 2024-06-11T13:30:43.979 INFO:tasks.workunit.client.0.smithi057.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-11T13:30:43.979 INFO:tasks.workunit.client.0.smithi057.stderr:+ export PATH 2024-06-11T13:30:43.979 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-11T13:30:43.979 INFO:tasks.workunit.client.0.smithi057.stderr:+ unset rc 2024-06-11T13:30:43.980 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo ec_list 2024-06-11T13:30:43.980 INFO:tasks.workunit.client.0.smithi057.stderr:+ sed 's/^/ cmd: /' 2024-06-11T13:30:43.980 INFO:tasks.workunit.client.0.smithi057.stderr:+ ff=ec_list 2024-06-11T13:30:43.980 INFO:tasks.workunit.client.0.smithi057.stdout:test ec_list on pid 78441 2024-06-11T13:30:43.980 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=78441 2024-06-11T13:30:43.980 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'test ec_list on pid 78441' 2024-06-11T13:30:43.980 INFO:tasks.workunit.client.0.smithi057.stderr:+ pids[$f]=78441 2024-06-11T13:30:43.981 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-06-11T13:30:43.981 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' 1 -eq 1 ']' 2024-06-11T13:30:43.981 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph_test_neorados_cmd 2024-06-11T13:30:43.981 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-11T13:30:43.982 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-11T13:30:43.982 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-11T13:30:43.982 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.982 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-11T13:30:43.982 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.982 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-11T13:30:43.982 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-11T13:30:43.982 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.983 INFO:tasks.workunit.client.0.smithi057.stderr:+ tee ceph_test_neorados_cmd.log 2024-06-11T13:30:43.983 INFO:tasks.workunit.client.0.smithi057.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_ec_list 2>&1 | tee ceph_test_neorados_ec_list.log | sed "s/^/ ec_list: /"' 2024-06-11T13:30:43.984 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-11T13:30:43.984 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-11T13:30:43.984 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-11T13:30:43.984 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.984 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-11T13:30:43.985 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.985 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-11T13:30:43.985 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-11T13:30:43.985 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.985 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-11T13:30:43.986 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.986 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/gawk.sh 2024-06-11T13:30:43.986 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.986 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-11T13:30:43.986 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.986 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/lang.sh 2024-06-11T13:30:43.986 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup 2024-06-11T13:30:43.986 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -f /etc/bashrc ']' 2024-06-11T13:30:43.986 INFO:tasks.workunit.client.0.smithi057.stderr:+ . /etc/bashrc 2024-06-11T13:30:43.986 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/locale 2024-06-11T13:30:43.986 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -z '' ']' 2024-06-11T13:30:43.987 INFO:tasks.workunit.client.0.smithi057.stderr:++ BASHRCSOURCED=Y 2024-06-11T13:30:43.987 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.987 INFO:tasks.workunit.client.0.smithi057.stderr:++ shopt -q login_shell 2024-06-11T13:30:43.988 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.988 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-11T13:30:43.988 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.988 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-11T13:30:43.988 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-11T13:30:43.988 INFO:tasks.workunit.client.0.smithi057.stderr:+++ umask 2024-06-11T13:30:43.988 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' 0022 -eq 0 ']' 2024-06-11T13:30:43.989 INFO:tasks.workunit.client.0.smithi057.stderr:++ SHELL=/bin/bash 2024-06-11T13:30:43.989 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.989 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-11T13:30:43.989 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.989 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-11T13:30:43.989 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-11T13:30:43.989 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.989 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-11T13:30:43.989 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.990 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-11T13:30:43.990 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-11T13:30:43.990 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.990 INFO:tasks.workunit.client.0.smithi057.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:43.990 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.990 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-11T13:30:43.990 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.990 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-11T13:30:43.990 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.990 INFO:tasks.workunit.client.0.smithi057.stderr:+++++ readlink /proc/78195/exe 2024-06-11T13:30:43.992 INFO:tasks.workunit.client.0.smithi057.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-11T13:30:43.992 INFO:tasks.workunit.client.0.smithi057.stderr:++++ export LANG=en_US.UTF-8 2024-06-11T13:30:43.992 INFO:tasks.workunit.client.0.smithi057.stderr:++++ LANG=en_US.UTF-8 2024-06-11T13:30:43.992 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:43.992 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-11T13:30:43.992 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.992 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG=en_US.UTF-8 2024-06-11T13:30:43.992 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup config 2024-06-11T13:30:43.992 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.992 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:43.992 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' dumb = linux ']' 2024-06-11T13:30:43.992 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.992 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-11T13:30:43.993 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.993 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/less.sh 2024-06-11T13:30:43.993 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-11T13:30:43.993 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.993 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-11T13:30:43.993 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.993 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/which2.sh 2024-06-11T13:30:43.993 INFO:tasks.workunit.client.0.smithi057.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-11T13:30:43.993 INFO:tasks.workunit.client.0.smithi057.stderr:++++ basename /usr/bin/bash 2024-06-11T13:30:43.994 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-11T13:30:43.994 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-11T13:30:43.994 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-11T13:30:43.994 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.994 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-11T13:30:43.994 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.994 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-11T13:30:43.994 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-11T13:30:43.994 INFO:tasks.workunit.client.0.smithi057.stderr:++ printf %25s misc 2024-06-11T13:30:43.994 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.994 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-11T13:30:43.994 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.995 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/gawk.sh 2024-06-11T13:30:43.995 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.995 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-11T13:30:43.995 INFO:tasks.workunit.client.0.smithi057.stderr:+++++ readlink /proc/78271/exe 2024-06-11T13:30:43.995 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.995 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/lang.sh 2024-06-11T13:30:43.995 INFO:tasks.workunit.client.0.smithi057.stderr:+ r=' misc' 2024-06-11T13:30:43.995 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup 2024-06-11T13:30:43.995 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_declare='declare -f' 2024-06-11T13:30:43.995 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_opt=-f 2024-06-11T13:30:43.995 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export which_declare 2024-06-11T13:30:43.995 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export -f which 2024-06-11T13:30:43.995 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset i 2024-06-11T13:30:43.995 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset -f pathmunge 2024-06-11T13:30:43.996 INFO:tasks.workunit.client.0.smithi057.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-11T13:30:43.996 INFO:tasks.workunit.client.0.smithi057.stderr:+ export PATH 2024-06-11T13:30:43.996 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-11T13:30:43.996 INFO:tasks.workunit.client.0.smithi057.stderr:+ unset rc 2024-06-11T13:30:43.996 INFO:tasks.workunit.client.0.smithi057.stderr:++++ basename /usr/bin/bash 2024-06-11T13:30:43.996 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/locale 2024-06-11T13:30:43.997 INFO:tasks.workunit.client.0.smithi057.stderr:+ sed 's/^/ handler_error: /' 2024-06-11T13:30:43.997 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_declare='declare -f' 2024-06-11T13:30:43.997 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_opt=-f 2024-06-11T13:30:43.997 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export which_declare 2024-06-11T13:30:43.997 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export -f which 2024-06-11T13:30:43.997 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset i 2024-06-11T13:30:43.997 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset -f pathmunge 2024-06-11T13:30:43.998 INFO:tasks.workunit.client.0.smithi057.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-11T13:30:43.998 INFO:tasks.workunit.client.0.smithi057.stderr:+ export PATH 2024-06-11T13:30:43.998 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-11T13:30:43.998 INFO:tasks.workunit.client.0.smithi057.stderr:+ unset rc 2024-06-11T13:30:43.998 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph_test_neorados_handler_error 2024-06-11T13:30:43.998 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'grep=grep --color=auto' 2024-06-11T13:30:43.998 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-11T13:30:43.998 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-11T13:30:43.998 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.998 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-11T13:30:43.998 INFO:tasks.workunit.client.0.smithi057.stderr:+ tee ceph_test_neorados_handler_error.log 2024-06-11T13:30:43.998 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.998 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorls.sh 2024-06-11T13:30:43.998 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' '!' -t 0 ']' 2024-06-11T13:30:43.999 INFO:tasks.workunit.client.0.smithi057.stderr:+++ return 2024-06-11T13:30:43.999 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.999 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-11T13:30:43.999 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.999 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-11T13:30:43.999 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export S_COLORS=auto 2024-06-11T13:30:43.999 INFO:tasks.workunit.client.0.smithi057.stderr:+++ S_COLORS=auto 2024-06-11T13:30:43.999 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:43.999 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-11T13:30:43.999 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:43.999 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-11T13:30:43.999 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:43.999 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:43.999 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-11T13:30:44.000 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:44.000 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-11T13:30:44.000 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-11T13:30:44.000 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-11T13:30:44.000 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-11T13:30:44.000 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-11T13:30:44.000 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.001 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-11T13:30:44.001 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.001 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-11T13:30:44.001 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-11T13:30:44.001 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:44.001 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-11T13:30:44.001 INFO:tasks.workunit.client.0.smithi057.stderr:+ sed 's/^/ io: /' 2024-06-11T13:30:44.001 INFO:tasks.workunit.client.0.smithi057.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-11T13:30:44.001 INFO:tasks.workunit.client.0.smithi057.stderr:++++ export LANG=en_US.UTF-8 2024-06-11T13:30:44.001 INFO:tasks.workunit.client.0.smithi057.stderr:++++ LANG=en_US.UTF-8 2024-06-11T13:30:44.001 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:44.001 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-11T13:30:44.001 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:44.002 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG=en_US.UTF-8 2024-06-11T13:30:44.002 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup config 2024-06-11T13:30:44.002 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:44.002 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:44.002 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' dumb = linux ']' 2024-06-11T13:30:44.002 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.002 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-11T13:30:44.002 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.002 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/less.sh 2024-06-11T13:30:44.002 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-11T13:30:44.002 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.002 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-11T13:30:44.002 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.002 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/which2.sh 2024-06-11T13:30:44.002 INFO:tasks.workunit.client.0.smithi057.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-11T13:30:44.003 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-11T13:30:44.003 INFO:tasks.workunit.client.0.smithi057.stderr:+++++ readlink /proc/78321/exe 2024-06-11T13:30:44.003 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-11T13:30:44.003 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-11T13:30:44.004 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.004 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-11T13:30:44.004 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.004 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-11T13:30:44.004 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-11T13:30:44.004 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:44.004 INFO:tasks.workunit.client.0.smithi057.stderr:++ awk '{print $1}' 2024-06-11T13:30:44.004 INFO:tasks.workunit.client.0.smithi057.stderr:++++ basename /usr/bin/bash 2024-06-11T13:30:44.004 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph_test_neorados_io 2024-06-11T13:30:44.004 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo misc 2024-06-11T13:30:44.005 INFO:tasks.workunit.client.0.smithi057.stderr:+ tee ceph_test_neorados_io.log 2024-06-11T13:30:44.005 INFO:tasks.workunit.client.0.smithi057.stdout:test misc on pid 78498 2024-06-11T13:30:44.005 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-11T13:30:44.005 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-11T13:30:44.005 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-11T13:30:44.006 INFO:tasks.workunit.client.0.smithi057.stderr:+ ff=misc 2024-06-11T13:30:44.006 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.006 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-11T13:30:44.006 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.006 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-11T13:30:44.006 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-11T13:30:44.006 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.006 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-11T13:30:44.006 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.006 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/gawk.sh 2024-06-11T13:30:44.006 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=78498 2024-06-11T13:30:44.006 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'test misc on pid 78498' 2024-06-11T13:30:44.006 INFO:tasks.workunit.client.0.smithi057.stderr:+ pids[$f]=78498 2024-06-11T13:30:44.006 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-06-11T13:30:44.007 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' 1 -eq 1 ']' 2024-06-11T13:30:44.007 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.007 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-11T13:30:44.007 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.007 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/lang.sh 2024-06-11T13:30:44.007 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup 2024-06-11T13:30:44.007 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_declare='declare -f' 2024-06-11T13:30:44.007 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_opt=-f 2024-06-11T13:30:44.007 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export which_declare 2024-06-11T13:30:44.007 INFO:tasks.workunit.client.0.smithi057.stderr:++ printf %25s pool 2024-06-11T13:30:44.007 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export -f which 2024-06-11T13:30:44.007 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset i 2024-06-11T13:30:44.007 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset -f pathmunge 2024-06-11T13:30:44.008 INFO:tasks.workunit.client.0.smithi057.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-11T13:30:44.008 INFO:tasks.workunit.client.0.smithi057.stderr:+ r=' pool' 2024-06-11T13:30:44.008 INFO:tasks.workunit.client.0.smithi057.stderr:+ export PATH 2024-06-11T13:30:44.008 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-11T13:30:44.008 INFO:tasks.workunit.client.0.smithi057.stderr:+ unset rc 2024-06-11T13:30:44.008 INFO:tasks.workunit.client.0.smithi057.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_misc 2>&1 | tee ceph_test_neorados_misc.log | sed "s/^/ misc: /"' 2024-06-11T13:30:44.009 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/locale 2024-06-11T13:30:44.009 INFO:tasks.workunit.client.0.smithi057.stderr:+ sed 's/^/ ec_io: /' 2024-06-11T13:30:44.009 INFO:tasks.workunit.client.0.smithi057.stderr:+ tee ceph_test_neorados_ec_io.log 2024-06-11T13:30:44.009 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:44.009 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-11T13:30:44.009 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:44.009 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-11T13:30:44.009 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-11T13:30:44.009 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-11T13:30:44.010 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -f /etc/bashrc ']' 2024-06-11T13:30:44.010 INFO:tasks.workunit.client.0.smithi057.stderr:+ . /etc/bashrc 2024-06-11T13:30:44.010 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -z '' ']' 2024-06-11T13:30:44.010 INFO:tasks.workunit.client.0.smithi057.stderr:++ BASHRCSOURCED=Y 2024-06-11T13:30:44.010 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.010 INFO:tasks.workunit.client.0.smithi057.stderr:++ shopt -q login_shell 2024-06-11T13:30:44.010 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph_test_neorados_ec_io 2024-06-11T13:30:44.011 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-11T13:30:44.011 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-11T13:30:44.011 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-11T13:30:44.011 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.011 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-11T13:30:44.011 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.011 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-11T13:30:44.011 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-11T13:30:44.011 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.011 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-11T13:30:44.011 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.012 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/gawk.sh 2024-06-11T13:30:44.012 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.012 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-11T13:30:44.012 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.012 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/lang.sh 2024-06-11T13:30:44.012 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup 2024-06-11T13:30:44.012 INFO:tasks.workunit.client.0.smithi057.stderr:++ awk '{print $1}' 2024-06-11T13:30:44.012 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo pool 2024-06-11T13:30:44.014 INFO:tasks.workunit.client.0.smithi057.stderr:+++ umask 2024-06-11T13:30:44.014 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' 0022 -eq 0 ']' 2024-06-11T13:30:44.014 INFO:tasks.workunit.client.0.smithi057.stderr:++ SHELL=/bin/bash 2024-06-11T13:30:44.014 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.014 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-11T13:30:44.014 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.014 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-11T13:30:44.014 INFO:tasks.workunit.client.0.smithi057.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-11T13:30:44.014 INFO:tasks.workunit.client.0.smithi057.stderr:++++ export LANG=en_US.UTF-8 2024-06-11T13:30:44.014 INFO:tasks.workunit.client.0.smithi057.stderr:++++ LANG=en_US.UTF-8 2024-06-11T13:30:44.014 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-11T13:30:44.014 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:44.015 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.015 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-11T13:30:44.015 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-11T13:30:44.015 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.015 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-11T13:30:44.015 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:44.015 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG=en_US.UTF-8 2024-06-11T13:30:44.015 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:44.015 INFO:tasks.workunit.client.0.smithi057.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:44.015 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup config 2024-06-11T13:30:44.015 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.015 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-11T13:30:44.015 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.015 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-11T13:30:44.016 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:44.016 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:44.016 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:44.016 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' dumb = linux ']' 2024-06-11T13:30:44.016 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.016 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-11T13:30:44.016 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.016 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/less.sh 2024-06-11T13:30:44.016 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-11T13:30:44.016 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.016 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-11T13:30:44.016 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.016 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/which2.sh 2024-06-11T13:30:44.016 INFO:tasks.workunit.client.0.smithi057.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-11T13:30:44.017 INFO:tasks.workunit.client.0.smithi057.stderr:+++++ readlink /proc/78361/exe 2024-06-11T13:30:44.017 INFO:tasks.workunit.client.0.smithi057.stdout:test pool on pid 78533 2024-06-11T13:30:44.017 INFO:tasks.workunit.client.0.smithi057.stderr:+ ff=pool 2024-06-11T13:30:44.018 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=78533 2024-06-11T13:30:44.018 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'test pool on pid 78533' 2024-06-11T13:30:44.018 INFO:tasks.workunit.client.0.smithi057.stderr:+ pids[$f]=78533 2024-06-11T13:30:44.018 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-06-11T13:30:44.018 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' 1 -eq 1 ']' 2024-06-11T13:30:44.018 INFO:tasks.workunit.client.0.smithi057.stderr:++++ basename /usr/bin/bash 2024-06-11T13:30:44.018 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'grep=grep --color=auto' 2024-06-11T13:30:44.018 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-11T13:30:44.018 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-11T13:30:44.018 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.018 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-11T13:30:44.018 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.018 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorls.sh 2024-06-11T13:30:44.019 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' '!' -t 0 ']' 2024-06-11T13:30:44.019 INFO:tasks.workunit.client.0.smithi057.stderr:+++ return 2024-06-11T13:30:44.019 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.019 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-11T13:30:44.019 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.019 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-11T13:30:44.019 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export S_COLORS=auto 2024-06-11T13:30:44.019 INFO:tasks.workunit.client.0.smithi057.stderr:+++ S_COLORS=auto 2024-06-11T13:30:44.019 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.019 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-11T13:30:44.019 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.019 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-11T13:30:44.019 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:44.019 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/locale 2024-06-11T13:30:44.020 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_declare='declare -f' 2024-06-11T13:30:44.020 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_opt=-f 2024-06-11T13:30:44.020 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export which_declare 2024-06-11T13:30:44.020 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export -f which 2024-06-11T13:30:44.020 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset i 2024-06-11T13:30:44.020 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset -f pathmunge 2024-06-11T13:30:44.020 INFO:tasks.workunit.client.0.smithi057.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-11T13:30:44.020 INFO:tasks.workunit.client.0.smithi057.stderr:+ export PATH 2024-06-11T13:30:44.020 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-11T13:30:44.020 INFO:tasks.workunit.client.0.smithi057.stderr:+ unset rc 2024-06-11T13:30:44.021 INFO:tasks.workunit.client.0.smithi057.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_pool 2>&1 | tee ceph_test_neorados_pool.log | sed "s/^/ pool: /"' 2024-06-11T13:30:44.021 INFO:tasks.workunit.client.0.smithi057.stderr:+ sed 's/^/ list: /' 2024-06-11T13:30:44.021 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph_test_neorados_list 2024-06-11T13:30:44.021 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -f /etc/bashrc ']' 2024-06-11T13:30:44.021 INFO:tasks.workunit.client.0.smithi057.stderr:+ . /etc/bashrc 2024-06-11T13:30:44.021 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:44.021 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-11T13:30:44.022 INFO:tasks.workunit.client.0.smithi057.stderr:++ printf %25s read_operations 2024-06-11T13:30:44.022 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:44.022 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-11T13:30:44.022 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-11T13:30:44.022 INFO:tasks.workunit.client.0.smithi057.stderr:+ tee ceph_test_neorados_list.log 2024-06-11T13:30:44.022 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-11T13:30:44.022 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-11T13:30:44.022 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-11T13:30:44.022 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-11T13:30:44.022 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.023 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-11T13:30:44.023 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.023 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-11T13:30:44.023 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-11T13:30:44.023 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:44.026 INFO:tasks.workunit.client.0.smithi057.stderr:+ r=' read_operations' 2024-06-11T13:30:44.026 INFO:tasks.workunit.client.0.smithi057.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-11T13:30:44.026 INFO:tasks.workunit.client.0.smithi057.stderr:++++ export LANG=en_US.UTF-8 2024-06-11T13:30:44.026 INFO:tasks.workunit.client.0.smithi057.stderr:++++ LANG=en_US.UTF-8 2024-06-11T13:30:44.026 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:44.026 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-11T13:30:44.026 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:44.026 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG=en_US.UTF-8 2024-06-11T13:30:44.026 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup config 2024-06-11T13:30:44.026 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:44.026 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:44.027 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' dumb = linux ']' 2024-06-11T13:30:44.027 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.027 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-11T13:30:44.027 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.027 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/less.sh 2024-06-11T13:30:44.027 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-11T13:30:44.027 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.027 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-11T13:30:44.027 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.027 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/which2.sh 2024-06-11T13:30:44.027 INFO:tasks.workunit.client.0.smithi057.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-11T13:30:44.027 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -z '' ']' 2024-06-11T13:30:44.027 INFO:tasks.workunit.client.0.smithi057.stderr:++ BASHRCSOURCED=Y 2024-06-11T13:30:44.028 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-11T13:30:44.028 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-11T13:30:44.028 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-11T13:30:44.028 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.028 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-11T13:30:44.029 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.029 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-11T13:30:44.029 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-11T13:30:44.029 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.029 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-11T13:30:44.029 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.029 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/gawk.sh 2024-06-11T13:30:44.029 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.029 INFO:tasks.workunit.client.0.smithi057.stderr:++ shopt -q login_shell 2024-06-11T13:30:44.029 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.029 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-11T13:30:44.029 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.029 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/lang.sh 2024-06-11T13:30:44.029 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup 2024-06-11T13:30:44.029 INFO:tasks.workunit.client.0.smithi057.stderr:++ awk '{print $1}' 2024-06-11T13:30:44.030 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/locale 2024-06-11T13:30:44.030 INFO:tasks.workunit.client.0.smithi057.stderr:+++++ readlink /proc/78441/exe 2024-06-11T13:30:44.030 INFO:tasks.workunit.client.0.smithi057.stderr:+++ umask 2024-06-11T13:30:44.031 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' 0022 -eq 0 ']' 2024-06-11T13:30:44.031 INFO:tasks.workunit.client.0.smithi057.stderr:++ SHELL=/bin/bash 2024-06-11T13:30:44.031 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.031 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-11T13:30:44.031 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.031 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-11T13:30:44.031 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-11T13:30:44.031 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.031 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-11T13:30:44.031 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.031 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-11T13:30:44.031 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:44.031 INFO:tasks.workunit.client.0.smithi057.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:44.031 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.031 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-11T13:30:44.032 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.032 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-11T13:30:44.032 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:44.032 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:44.032 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-11T13:30:44.032 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:44.032 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-11T13:30:44.032 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-11T13:30:44.032 INFO:tasks.workunit.client.0.smithi057.stderr:++++ basename /usr/bin/bash 2024-06-11T13:30:44.032 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo read_operations 2024-06-11T13:30:44.032 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-11T13:30:44.033 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_declare='declare -f' 2024-06-11T13:30:44.033 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_opt=-f 2024-06-11T13:30:44.033 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export which_declare 2024-06-11T13:30:44.033 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export -f which 2024-06-11T13:30:44.033 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset i 2024-06-11T13:30:44.033 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset -f pathmunge 2024-06-11T13:30:44.033 INFO:tasks.workunit.client.0.smithi057.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-11T13:30:44.033 INFO:tasks.workunit.client.0.smithi057.stderr:+ export PATH 2024-06-11T13:30:44.033 INFO:tasks.workunit.client.0.smithi057.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-11T13:30:44.033 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-11T13:30:44.033 INFO:tasks.workunit.client.0.smithi057.stderr:++++ export LANG=en_US.UTF-8 2024-06-11T13:30:44.034 INFO:tasks.workunit.client.0.smithi057.stderr:++++ LANG=en_US.UTF-8 2024-06-11T13:30:44.034 INFO:tasks.workunit.client.0.smithi057.stderr:+ unset rc 2024-06-11T13:30:44.034 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:44.034 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-11T13:30:44.034 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:44.034 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG=en_US.UTF-8 2024-06-11T13:30:44.034 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup config 2024-06-11T13:30:44.034 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:44.034 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:44.034 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' dumb = linux ']' 2024-06-11T13:30:44.034 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.034 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-11T13:30:44.034 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.035 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/less.sh 2024-06-11T13:30:44.035 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-11T13:30:44.035 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.035 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-11T13:30:44.035 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.035 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/which2.sh 2024-06-11T13:30:44.035 INFO:tasks.workunit.client.0.smithi057.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-11T13:30:44.035 INFO:tasks.workunit.client.0.smithi057.stderr:+ ff=read_operations 2024-06-11T13:30:44.035 INFO:tasks.workunit.client.0.smithi057.stdout:test read_operations on pid 78574 2024-06-11T13:30:44.036 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=78574 2024-06-11T13:30:44.036 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'test read_operations on pid 78574' 2024-06-11T13:30:44.036 INFO:tasks.workunit.client.0.smithi057.stderr:+ pids[$f]=78574 2024-06-11T13:30:44.036 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-06-11T13:30:44.036 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' 1 -eq 1 ']' 2024-06-11T13:30:44.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:30:43 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:30:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:30:44.037 INFO:tasks.workunit.client.0.smithi057.stderr:++ printf %25s snapshots 2024-06-11T13:30:44.037 INFO:tasks.workunit.client.0.smithi057.stderr:+ tee ceph_test_neorados_ec_list.log 2024-06-11T13:30:44.038 INFO:tasks.workunit.client.0.smithi057.stderr:+ r=' snapshots' 2024-06-11T13:30:44.038 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph_test_neorados_ec_list 2024-06-11T13:30:44.038 INFO:tasks.workunit.client.0.smithi057.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_read_operations 2>&1 | tee ceph_test_neorados_read_operations.log | sed "s/^/ read_operations: /"' 2024-06-11T13:30:44.038 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo snapshots 2024-06-11T13:30:44.038 INFO:tasks.workunit.client.0.smithi057.stderr:++ awk '{print $1}' 2024-06-11T13:30:44.038 INFO:tasks.workunit.client.0.smithi057.stderr:+ sed 's/^/ ec_list: /' 2024-06-11T13:30:44.038 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -f /etc/bashrc ']' 2024-06-11T13:30:44.038 INFO:tasks.workunit.client.0.smithi057.stderr:+ . /etc/bashrc 2024-06-11T13:30:44.038 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -z '' ']' 2024-06-11T13:30:44.038 INFO:tasks.workunit.client.0.smithi057.stderr:++ BASHRCSOURCED=Y 2024-06-11T13:30:44.038 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.038 INFO:tasks.workunit.client.0.smithi057.stderr:++ shopt -q login_shell 2024-06-11T13:30:44.039 INFO:tasks.workunit.client.0.smithi057.stderr:+++++ readlink /proc/78498/exe 2024-06-11T13:30:44.040 INFO:tasks.workunit.client.0.smithi057.stdout:test snapshots on pid 78598 2024-06-11T13:30:44.040 INFO:tasks.workunit.client.0.smithi057.stderr:++++ basename /usr/bin/bash 2024-06-11T13:30:44.040 INFO:tasks.workunit.client.0.smithi057.stderr:+ ff=snapshots 2024-06-11T13:30:44.040 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=78598 2024-06-11T13:30:44.040 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'test snapshots on pid 78598' 2024-06-11T13:30:44.040 INFO:tasks.workunit.client.0.smithi057.stderr:+ pids[$f]=78598 2024-06-11T13:30:44.040 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-06-11T13:30:44.040 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' 1 -eq 1 ']' 2024-06-11T13:30:44.041 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_declare='declare -f' 2024-06-11T13:30:44.041 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_opt=-f 2024-06-11T13:30:44.041 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export which_declare 2024-06-11T13:30:44.041 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export -f which 2024-06-11T13:30:44.041 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset i 2024-06-11T13:30:44.041 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset -f pathmunge 2024-06-11T13:30:44.041 INFO:tasks.workunit.client.0.smithi057.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-11T13:30:44.041 INFO:tasks.workunit.client.0.smithi057.stderr:+ export PATH 2024-06-11T13:30:44.041 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-11T13:30:44.041 INFO:tasks.workunit.client.0.smithi057.stderr:+ unset rc 2024-06-11T13:30:44.042 INFO:tasks.workunit.client.0.smithi057.stderr:+ sed 's/^/ misc: /' 2024-06-11T13:30:44.042 INFO:tasks.workunit.client.0.smithi057.stderr:+++ umask 2024-06-11T13:30:44.042 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'grep=grep --color=auto' 2024-06-11T13:30:44.042 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-11T13:30:44.042 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-11T13:30:44.043 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.043 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-11T13:30:44.043 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.043 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorls.sh 2024-06-11T13:30:44.043 INFO:tasks.workunit.client.0.smithi057.stderr:+ tee ceph_test_neorados_misc.log 2024-06-11T13:30:44.043 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' '!' -t 0 ']' 2024-06-11T13:30:44.043 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' 0022 -eq 0 ']' 2024-06-11T13:30:44.043 INFO:tasks.workunit.client.0.smithi057.stderr:+++ return 2024-06-11T13:30:44.043 INFO:tasks.workunit.client.0.smithi057.stderr:++ SHELL=/bin/bash 2024-06-11T13:30:44.043 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.043 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-11T13:30:44.043 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.043 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-11T13:30:44.043 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.043 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-11T13:30:44.044 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.044 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-11T13:30:44.044 INFO:tasks.workunit.client.0.smithi057.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_snapshots 2>&1 | tee ceph_test_neorados_snapshots.log | sed "s/^/ snapshots: /"' 2024-06-11T13:30:44.044 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-11T13:30:44.044 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.044 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-11T13:30:44.044 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.044 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-11T13:30:44.044 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:44.045 INFO:tasks.workunit.client.0.smithi057.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:44.045 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.045 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-11T13:30:44.045 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.045 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-11T13:30:44.045 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph_test_neorados_misc 2024-06-11T13:30:44.045 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:44.046 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export S_COLORS=auto 2024-06-11T13:30:44.046 INFO:tasks.workunit.client.0.smithi057.stderr:+++ S_COLORS=auto 2024-06-11T13:30:44.046 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.046 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-11T13:30:44.046 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.046 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-11T13:30:44.046 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:44.046 INFO:tasks.workunit.client.0.smithi057.stderr:++ printf %25s watch_notify 2024-06-11T13:30:44.046 INFO:tasks.workunit.client.0.smithi057.stderr:+ r=' watch_notify' 2024-06-11T13:30:44.047 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -f /etc/bashrc ']' 2024-06-11T13:30:44.047 INFO:tasks.workunit.client.0.smithi057.stderr:+ . /etc/bashrc 2024-06-11T13:30:44.047 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -z '' ']' 2024-06-11T13:30:44.048 INFO:tasks.workunit.client.0.smithi057.stderr:++ BASHRCSOURCED=Y 2024-06-11T13:30:44.048 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.048 INFO:tasks.workunit.client.0.smithi057.stderr:++ shopt -q login_shell 2024-06-11T13:30:44.049 INFO:tasks.workunit.client.0.smithi057.stderr:+++ umask 2024-06-11T13:30:44.050 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' 0022 -eq 0 ']' 2024-06-11T13:30:44.050 INFO:tasks.workunit.client.0.smithi057.stderr:++ SHELL=/bin/bash 2024-06-11T13:30:44.050 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.050 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-11T13:30:44.050 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.050 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-11T13:30:44.050 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-11T13:30:44.051 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.051 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-11T13:30:44.051 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.051 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-11T13:30:44.051 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:44.051 INFO:tasks.workunit.client.0.smithi057.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:44.051 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-11T13:30:44.051 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.051 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-11T13:30:44.051 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.051 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-11T13:30:44.051 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-11T13:30:44.051 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-11T13:30:44.051 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:44.051 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.052 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-11T13:30:44.052 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.052 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-11T13:30:44.052 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-11T13:30:44.052 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:44.052 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo watch_notify 2024-06-11T13:30:44.052 INFO:tasks.workunit.client.0.smithi057.stderr:++ awk '{print $1}' 2024-06-11T13:30:44.056 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'grep=grep --color=auto' 2024-06-11T13:30:44.056 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-11T13:30:44.056 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-11T13:30:44.056 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.056 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-11T13:30:44.056 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.056 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorls.sh 2024-06-11T13:30:44.056 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' '!' -t 0 ']' 2024-06-11T13:30:44.056 INFO:tasks.workunit.client.0.smithi057.stderr:+++ return 2024-06-11T13:30:44.057 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.057 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-11T13:30:44.057 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.057 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-11T13:30:44.057 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export S_COLORS=auto 2024-06-11T13:30:44.057 INFO:tasks.workunit.client.0.smithi057.stderr:+++ S_COLORS=auto 2024-06-11T13:30:44.057 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.057 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-11T13:30:44.057 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.057 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-11T13:30:44.057 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:44.058 INFO:tasks.workunit.client.0.smithi057.stdout:test watch_notify on pid 78655 2024-06-11T13:30:44.060 INFO:tasks.workunit.client.0.smithi057.stderr:+ ff=watch_notify 2024-06-11T13:30:44.060 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=78655 2024-06-11T13:30:44.060 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'test watch_notify on pid 78655' 2024-06-11T13:30:44.060 INFO:tasks.workunit.client.0.smithi057.stderr:+ pids[$f]=78655 2024-06-11T13:30:44.060 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-06-11T13:30:44.061 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' 1 -eq 1 ']' 2024-06-11T13:30:44.061 INFO:tasks.workunit.client.0.smithi057.stderr:++ printf %25s write_operations 2024-06-11T13:30:44.061 INFO:tasks.workunit.client.0.smithi057.stderr:+ r=' write_operations' 2024-06-11T13:30:44.061 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-11T13:30:44.061 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-11T13:30:44.061 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-11T13:30:44.061 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.061 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'grep=grep --color=auto' 2024-06-11T13:30:44.061 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-11T13:30:44.061 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.061 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-11T13:30:44.061 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-11T13:30:44.061 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-11T13:30:44.061 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.062 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-11T13:30:44.062 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-11T13:30:44.062 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.062 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-11T13:30:44.062 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.062 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.062 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/gawk.sh 2024-06-11T13:30:44.062 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorls.sh 2024-06-11T13:30:44.062 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' '!' -t 0 ']' 2024-06-11T13:30:44.062 INFO:tasks.workunit.client.0.smithi057.stderr:+++ return 2024-06-11T13:30:44.062 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.062 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.062 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-11T13:30:44.063 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-11T13:30:44.063 INFO:tasks.workunit.client.0.smithi057.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_watch_notify 2>&1 | tee ceph_test_neorados_watch_notify.log | sed "s/^/ watch_notify: /"' 2024-06-11T13:30:44.063 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.063 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.063 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/lang.sh 2024-06-11T13:30:44.063 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup 2024-06-11T13:30:44.063 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-11T13:30:44.063 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export S_COLORS=auto 2024-06-11T13:30:44.063 INFO:tasks.workunit.client.0.smithi057.stderr:+++ S_COLORS=auto 2024-06-11T13:30:44.063 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.063 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-11T13:30:44.063 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.063 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-11T13:30:44.063 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:44.064 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/locale 2024-06-11T13:30:44.064 INFO:tasks.workunit.client.0.smithi057.stderr:++ awk '{print $1}' 2024-06-11T13:30:44.064 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -f /etc/bashrc ']' 2024-06-11T13:30:44.064 INFO:tasks.workunit.client.0.smithi057.stderr:+ . /etc/bashrc 2024-06-11T13:30:44.064 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -z '' ']' 2024-06-11T13:30:44.064 INFO:tasks.workunit.client.0.smithi057.stderr:++ BASHRCSOURCED=Y 2024-06-11T13:30:44.064 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.064 INFO:tasks.workunit.client.0.smithi057.stderr:++ shopt -q login_shell 2024-06-11T13:30:44.064 INFO:tasks.workunit.client.0.smithi057.stderr:+++ umask 2024-06-11T13:30:44.065 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:44.065 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-11T13:30:44.065 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:44.065 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-11T13:30:44.065 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-11T13:30:44.065 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-11T13:30:44.065 INFO:tasks.workunit.client.0.smithi057.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-11T13:30:44.066 INFO:tasks.workunit.client.0.smithi057.stderr:++++ export LANG=en_US.UTF-8 2024-06-11T13:30:44.066 INFO:tasks.workunit.client.0.smithi057.stderr:++++ LANG=en_US.UTF-8 2024-06-11T13:30:44.066 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:44.066 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-11T13:30:44.066 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:44.066 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG=en_US.UTF-8 2024-06-11T13:30:44.066 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup config 2024-06-11T13:30:44.066 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:44.066 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:44.066 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' dumb = linux ']' 2024-06-11T13:30:44.066 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' 0022 -eq 0 ']' 2024-06-11T13:30:44.066 INFO:tasks.workunit.client.0.smithi057.stderr:++ SHELL=/bin/bash 2024-06-11T13:30:44.066 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.067 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.067 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-11T13:30:44.067 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-11T13:30:44.067 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.067 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.067 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/less.sh 2024-06-11T13:30:44.067 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-11T13:30:44.067 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-11T13:30:44.067 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.067 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-11T13:30:44.067 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.067 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/which2.sh 2024-06-11T13:30:44.067 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-11T13:30:44.067 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.067 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-11T13:30:44.068 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.068 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-11T13:30:44.068 INFO:tasks.workunit.client.0.smithi057.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-11T13:30:44.068 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:44.068 INFO:tasks.workunit.client.0.smithi057.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:44.068 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.068 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-11T13:30:44.068 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.068 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-11T13:30:44.068 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:44.068 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo write_operations 2024-06-11T13:30:44.069 INFO:tasks.workunit.client.0.smithi057.stdout:test write_operations on pid 78675 2024-06-11T13:30:44.073 INFO:tasks.workunit.client.0.smithi057.stderr:+ ff=write_operations 2024-06-11T13:30:44.073 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=78675 2024-06-11T13:30:44.074 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'test write_operations on pid 78675' 2024-06-11T13:30:44.074 INFO:tasks.workunit.client.0.smithi057.stderr:+ pids[$f]=78675 2024-06-11T13:30:44.074 INFO:tasks.workunit.client.0.smithi057.stderr:+ ret=0 2024-06-11T13:30:44.074 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' 1 -eq 1 ']' 2024-06-11T13:30:44.074 INFO:tasks.workunit.client.0.smithi057.stderr:+ for t in "${!pids[@]}" 2024-06-11T13:30:44.074 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77482 2024-06-11T13:30:44.074 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 77482 2024-06-11T13:30:44.074 INFO:tasks.workunit.client.0.smithi057.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_write_operations 2>&1 | tee ceph_test_neorados_write_operations.log | sed "s/^/ write_operations: /"' 2024-06-11T13:30:44.074 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-11T13:30:44.074 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-11T13:30:44.074 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-11T13:30:44.074 INFO:tasks.workunit.client.0.smithi057.stderr:+++++ readlink /proc/78533/exe 2024-06-11T13:30:44.074 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.075 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-11T13:30:44.075 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.075 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-11T13:30:44.075 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-11T13:30:44.075 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:44.075 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-11T13:30:44.075 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-11T13:30:44.075 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-11T13:30:44.075 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.075 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-11T13:30:44.075 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.075 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-11T13:30:44.075 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-11T13:30:44.075 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:44.076 INFO:tasks.workunit.client.0.smithi057.stderr:++++ basename /usr/bin/bash 2024-06-11T13:30:44.076 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -f /etc/bashrc ']' 2024-06-11T13:30:44.076 INFO:tasks.workunit.client.0.smithi057.stderr:+ . /etc/bashrc 2024-06-11T13:30:44.076 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -z '' ']' 2024-06-11T13:30:44.076 INFO:tasks.workunit.client.0.smithi057.stderr:++ BASHRCSOURCED=Y 2024-06-11T13:30:44.076 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.076 INFO:tasks.workunit.client.0.smithi057.stderr:++ shopt -q login_shell 2024-06-11T13:30:44.076 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'grep=grep --color=auto' 2024-06-11T13:30:44.076 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-11T13:30:44.076 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-11T13:30:44.076 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.076 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_declare='declare -f' 2024-06-11T13:30:44.076 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-11T13:30:44.076 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_opt=-f 2024-06-11T13:30:44.076 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.077 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorls.sh 2024-06-11T13:30:44.077 INFO:tasks.workunit.client.0.smithi057.stderr:+++ umask 2024-06-11T13:30:44.077 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export which_declare 2024-06-11T13:30:44.077 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' '!' -t 0 ']' 2024-06-11T13:30:44.077 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export -f which 2024-06-11T13:30:44.077 INFO:tasks.workunit.client.0.smithi057.stderr:+++ return 2024-06-11T13:30:44.077 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.077 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-11T13:30:44.077 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset i 2024-06-11T13:30:44.077 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.077 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-11T13:30:44.077 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset -f pathmunge 2024-06-11T13:30:44.077 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export S_COLORS=auto 2024-06-11T13:30:44.077 INFO:tasks.workunit.client.0.smithi057.stderr:+++ S_COLORS=auto 2024-06-11T13:30:44.078 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.078 INFO:tasks.workunit.client.0.smithi057.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-11T13:30:44.078 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-11T13:30:44.078 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.078 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-11T13:30:44.078 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:44.078 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' 0022 -eq 0 ']' 2024-06-11T13:30:44.078 INFO:tasks.workunit.client.0.smithi057.stderr:++ SHELL=/bin/bash 2024-06-11T13:30:44.078 INFO:tasks.workunit.client.0.smithi057.stderr:+ export PATH 2024-06-11T13:30:44.078 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-11T13:30:44.078 INFO:tasks.workunit.client.0.smithi057.stderr:+ unset rc 2024-06-11T13:30:44.078 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.078 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-06-11T13:30:44.079 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.079 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/bash_completion.sh 2024-06-11T13:30:44.079 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-06-11T13:30:44.079 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.079 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-06-11T13:30:44.079 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.079 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/cobbler.sh 2024-06-11T13:30:44.079 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:44.079 INFO:tasks.workunit.client.0.smithi057.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-06-11T13:30:44.079 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.079 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-06-11T13:30:44.079 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.079 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorgrep.sh 2024-06-11T13:30:44.079 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:44.080 INFO:tasks.workunit.client.0.smithi057.stderr:+ tee ceph_test_neorados_pool.log 2024-06-11T13:30:44.080 INFO:tasks.workunit.client.0.smithi057.stderr:+ sed 's/^/ pool: /' 2024-06-11T13:30:44.080 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-11T13:30:44.080 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-11T13:30:44.080 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-11T13:30:44.080 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.080 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-11T13:30:44.080 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.080 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-11T13:30:44.080 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-11T13:30:44.080 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-11T13:30:44.080 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-11T13:30:44.080 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-11T13:30:44.080 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.080 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-11T13:30:44.081 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.081 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.081 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/gawk.sh 2024-06-11T13:30:44.081 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-11T13:30:44.081 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.081 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-11T13:30:44.081 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-11T13:30:44.081 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.081 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.081 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-11T13:30:44.081 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-11T13:30:44.081 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.081 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.081 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/gawk.sh 2024-06-11T13:30:44.082 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/lang.sh 2024-06-11T13:30:44.082 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup 2024-06-11T13:30:44.082 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.082 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-11T13:30:44.082 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.082 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/lang.sh 2024-06-11T13:30:44.082 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup 2024-06-11T13:30:44.082 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph_test_neorados_pool 2024-06-11T13:30:44.082 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/locale 2024-06-11T13:30:44.082 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/locale 2024-06-11T13:30:44.082 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-11T13:30:44.082 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-11T13:30:44.082 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-11T13:30:44.083 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.083 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-11T13:30:44.083 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.083 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-11T13:30:44.083 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-11T13:30:44.083 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:44.083 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:44.083 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-11T13:30:44.083 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:44.083 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-11T13:30:44.083 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-11T13:30:44.083 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:44.083 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-11T13:30:44.083 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:44.084 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-11T13:30:44.084 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-11T13:30:44.084 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-11T13:30:44.084 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-11T13:30:44.084 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'grep=grep --color=auto' 2024-06-11T13:30:44.084 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'egrep=egrep --color=auto' 2024-06-11T13:30:44.084 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-06-11T13:30:44.084 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.084 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-06-11T13:30:44.084 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.084 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorls.sh 2024-06-11T13:30:44.084 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' '!' -t 0 ']' 2024-06-11T13:30:44.084 INFO:tasks.workunit.client.0.smithi057.stderr:+++ return 2024-06-11T13:30:44.084 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.084 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-06-11T13:30:44.085 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.085 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-06-11T13:30:44.085 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export S_COLORS=auto 2024-06-11T13:30:44.085 INFO:tasks.workunit.client.0.smithi057.stderr:+++ S_COLORS=auto 2024-06-11T13:30:44.085 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.085 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-06-11T13:30:44.085 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.085 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-06-11T13:30:44.085 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:44.090 INFO:tasks.workunit.client.0.smithi057.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-11T13:30:44.090 INFO:tasks.workunit.client.0.smithi057.stderr:++++ export LANG=en_US.UTF-8 2024-06-11T13:30:44.090 INFO:tasks.workunit.client.0.smithi057.stderr:++++ LANG=en_US.UTF-8 2024-06-11T13:30:44.090 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:44.090 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-11T13:30:44.090 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:44.090 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG=en_US.UTF-8 2024-06-11T13:30:44.090 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup config 2024-06-11T13:30:44.090 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:44.090 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:44.090 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' dumb = linux ']' 2024-06-11T13:30:44.090 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.090 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-11T13:30:44.091 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-11T13:30:44.091 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.091 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/less.sh 2024-06-11T13:30:44.091 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-11T13:30:44.091 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-11T13:30:44.091 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-11T13:30:44.091 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.091 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-11T13:30:44.091 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.091 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.091 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/which2.sh 2024-06-11T13:30:44.091 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-11T13:30:44.091 INFO:tasks.workunit.client.0.smithi057.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-11T13:30:44.091 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.092 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-11T13:30:44.092 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-11T13:30:44.092 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.092 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-11T13:30:44.092 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.092 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/gawk.sh 2024-06-11T13:30:44.092 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.092 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-11T13:30:44.092 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.092 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/lang.sh 2024-06-11T13:30:44.092 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup 2024-06-11T13:30:44.092 INFO:tasks.workunit.client.0.smithi057.stderr:+++++ readlink /proc/78574/exe 2024-06-11T13:30:44.092 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/locale 2024-06-11T13:30:44.092 INFO:tasks.workunit.client.0.smithi057.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-11T13:30:44.092 INFO:tasks.workunit.client.0.smithi057.stderr:++++ export LANG=en_US.UTF-8 2024-06-11T13:30:44.093 INFO:tasks.workunit.client.0.smithi057.stderr:++++ LANG=en_US.UTF-8 2024-06-11T13:30:44.093 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:44.093 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-11T13:30:44.093 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:44.093 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG=en_US.UTF-8 2024-06-11T13:30:44.093 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup config 2024-06-11T13:30:44.093 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:44.093 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:44.093 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' dumb = linux ']' 2024-06-11T13:30:44.093 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.093 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-11T13:30:44.093 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.093 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/less.sh 2024-06-11T13:30:44.093 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-11T13:30:44.094 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.094 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-06-11T13:30:44.094 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-11T13:30:44.094 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.094 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/which2.sh 2024-06-11T13:30:44.094 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-06-11T13:30:44.094 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-06-11T13:30:44.094 INFO:tasks.workunit.client.0.smithi057.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-11T13:30:44.094 INFO:tasks.workunit.client.0.smithi057.stderr:++++ basename /usr/bin/bash 2024-06-11T13:30:44.094 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.094 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-06-11T13:30:44.094 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.094 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-06-11T13:30:44.094 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-06-11T13:30:44.095 INFO:tasks.workunit.client.0.smithi057.stderr:+++ /usr/libexec/grepconf.sh -c 2024-06-11T13:30:44.095 INFO:tasks.workunit.client.0.smithi057.stderr:+++++ readlink /proc/78598/exe 2024-06-11T13:30:44.095 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_declare='declare -f' 2024-06-11T13:30:44.095 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_opt=-f 2024-06-11T13:30:44.095 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export which_declare 2024-06-11T13:30:44.095 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export -f which 2024-06-11T13:30:44.095 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset i 2024-06-11T13:30:44.095 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset -f pathmunge 2024-06-11T13:30:44.095 INFO:tasks.workunit.client.0.smithi057.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-11T13:30:44.095 INFO:tasks.workunit.client.0.smithi057.stderr:+ export PATH 2024-06-11T13:30:44.095 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-11T13:30:44.095 INFO:tasks.workunit.client.0.smithi057.stderr:+ unset rc 2024-06-11T13:30:44.095 INFO:tasks.workunit.client.0.smithi057.stderr:++++ basename /usr/bin/bash 2024-06-11T13:30:44.095 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:44.096 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-11T13:30:44.096 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:44.096 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph_test_neorados_read_operations 2024-06-11T13:30:44.096 INFO:tasks.workunit.client.0.smithi057.stderr:+ sed 's/^/ read_operations: /' 2024-06-11T13:30:44.096 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-11T13:30:44.096 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-11T13:30:44.096 INFO:tasks.workunit.client.0.smithi057.stderr:+ tee ceph_test_neorados_read_operations.log 2024-06-11T13:30:44.096 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_declare='declare -f' 2024-06-11T13:30:44.096 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_opt=-f 2024-06-11T13:30:44.096 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export which_declare 2024-06-11T13:30:44.096 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export -f which 2024-06-11T13:30:44.096 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset i 2024-06-11T13:30:44.096 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset -f pathmunge 2024-06-11T13:30:44.096 INFO:tasks.workunit.client.0.smithi057.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-11T13:30:44.097 INFO:tasks.workunit.client.0.smithi057.stderr:+ export PATH 2024-06-11T13:30:44.097 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-11T13:30:44.097 INFO:tasks.workunit.client.0.smithi057.stderr:+ unset rc 2024-06-11T13:30:44.097 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph_test_neorados_snapshots 2024-06-11T13:30:44.097 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-06-11T13:30:44.097 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-06-11T13:30:44.097 INFO:tasks.workunit.client.0.smithi057.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-06-11T13:30:44.097 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-11T13:30:44.097 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.097 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-06-11T13:30:44.097 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.097 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/debuginfod.sh 2024-06-11T13:30:44.097 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-06-11T13:30:44.097 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.098 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-06-11T13:30:44.098 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.098 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/gawk.sh 2024-06-11T13:30:44.098 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.098 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-06-11T13:30:44.098 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.098 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/lang.sh 2024-06-11T13:30:44.098 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup 2024-06-11T13:30:44.098 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/locale 2024-06-11T13:30:44.098 INFO:tasks.workunit.client.0.smithi057.stderr:+ tee ceph_test_neorados_snapshots.log 2024-06-11T13:30:44.098 INFO:tasks.workunit.client.0.smithi057.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-11T13:30:44.098 INFO:tasks.workunit.client.0.smithi057.stderr:++++ export LANG=en_US.UTF-8 2024-06-11T13:30:44.098 INFO:tasks.workunit.client.0.smithi057.stderr:++++ LANG=en_US.UTF-8 2024-06-11T13:30:44.098 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:44.099 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-11T13:30:44.099 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:44.099 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG=en_US.UTF-8 2024-06-11T13:30:44.099 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup config 2024-06-11T13:30:44.099 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:44.099 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:44.099 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' dumb = linux ']' 2024-06-11T13:30:44.099 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.099 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-11T13:30:44.099 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.099 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/less.sh 2024-06-11T13:30:44.099 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-11T13:30:44.099 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.099 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-11T13:30:44.099 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.100 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/which2.sh 2024-06-11T13:30:44.100 INFO:tasks.workunit.client.0.smithi057.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-11T13:30:44.100 INFO:tasks.workunit.client.0.smithi057.stderr:+ sed 's/^/ snapshots: /' 2024-06-11T13:30:44.100 INFO:tasks.workunit.client.0.smithi057.stderr:+++++ readlink /proc/78655/exe 2024-06-11T13:30:44.100 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:44.100 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG_backup=en_US.UTF-8 2024-06-11T13:30:44.100 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:44.100 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /etc/locale.conf ']' 2024-06-11T13:30:44.100 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -x /usr/bin/sed ']' 2024-06-11T13:30:44.100 INFO:tasks.workunit.client.0.smithi057.stderr:++++ basename /usr/bin/bash 2024-06-11T13:30:44.100 INFO:tasks.workunit.client.0.smithi057.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-06-11T13:30:44.100 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_declare='declare -f' 2024-06-11T13:30:44.100 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_opt=-f 2024-06-11T13:30:44.100 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export which_declare 2024-06-11T13:30:44.101 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export -f which 2024-06-11T13:30:44.101 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset i 2024-06-11T13:30:44.101 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset -f pathmunge 2024-06-11T13:30:44.101 INFO:tasks.workunit.client.0.smithi057.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-11T13:30:44.101 INFO:tasks.workunit.client.0.smithi057.stderr:+ export PATH 2024-06-11T13:30:44.101 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-11T13:30:44.101 INFO:tasks.workunit.client.0.smithi057.stderr:+ unset rc 2024-06-11T13:30:44.101 INFO:tasks.workunit.client.0.smithi057.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-06-11T13:30:44.101 INFO:tasks.workunit.client.0.smithi057.stderr:++++ export LANG=en_US.UTF-8 2024-06-11T13:30:44.101 INFO:tasks.workunit.client.0.smithi057.stderr:++++ LANG=en_US.UTF-8 2024-06-11T13:30:44.101 INFO:tasks.workunit.client.0.smithi057.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-06-11T13:30:44.101 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-06-11T13:30:44.101 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:44.101 INFO:tasks.workunit.client.0.smithi057.stderr:+++ LANG=en_US.UTF-8 2024-06-11T13:30:44.101 INFO:tasks.workunit.client.0.smithi057.stderr:+++ unset LANG_backup config 2024-06-11T13:30:44.102 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n '' ']' 2024-06-11T13:30:44.102 INFO:tasks.workunit.client.0.smithi057.stderr:+ tee ceph_test_neorados_watch_notify.log 2024-06-11T13:30:44.102 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-06-11T13:30:44.102 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' dumb = linux ']' 2024-06-11T13:30:44.102 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.102 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-06-11T13:30:44.102 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.102 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/less.sh 2024-06-11T13:30:44.102 INFO:tasks.workunit.client.0.smithi057.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-06-11T13:30:44.102 INFO:tasks.workunit.client.0.smithi057.stderr:++ for i in /etc/profile.d/*.sh 2024-06-11T13:30:44.102 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-06-11T13:30:44.102 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' ']' 2024-06-11T13:30:44.102 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph_test_neorados_watch_notify 2024-06-11T13:30:44.102 INFO:tasks.workunit.client.0.smithi057.stderr:++ . /etc/profile.d/which2.sh 2024-06-11T13:30:44.103 INFO:tasks.workunit.client.0.smithi057.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-06-11T13:30:44.103 INFO:tasks.workunit.client.0.smithi057.stderr:+ sed 's/^/ watch_notify: /' 2024-06-11T13:30:44.103 INFO:tasks.workunit.client.0.smithi057.stderr:+++++ readlink /proc/78675/exe 2024-06-11T13:30:44.104 INFO:tasks.workunit.client.0.smithi057.stderr:++++ basename /usr/bin/bash 2024-06-11T13:30:44.104 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_declare='declare -f' 2024-06-11T13:30:44.104 INFO:tasks.workunit.client.0.smithi057.stderr:+++ which_opt=-f 2024-06-11T13:30:44.104 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export which_declare 2024-06-11T13:30:44.104 INFO:tasks.workunit.client.0.smithi057.stderr:+++ export -f which 2024-06-11T13:30:44.104 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset i 2024-06-11T13:30:44.104 INFO:tasks.workunit.client.0.smithi057.stderr:++ unset -f pathmunge 2024-06-11T13:30:44.105 INFO:tasks.workunit.client.0.smithi057.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-06-11T13:30:44.105 INFO:tasks.workunit.client.0.smithi057.stderr:+ export PATH 2024-06-11T13:30:44.105 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-06-11T13:30:44.105 INFO:tasks.workunit.client.0.smithi057.stderr:+ unset rc 2024-06-11T13:30:44.105 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph_test_neorados_write_operations 2024-06-11T13:30:44.105 INFO:tasks.workunit.client.0.smithi057.stderr:+ tee ceph_test_neorados_write_operations.log 2024-06-11T13:30:44.105 INFO:tasks.workunit.client.0.smithi057.stderr:+ sed 's/^/ write_operations: /' 2024-06-11T13:30:45.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:44 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/289483390' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi057-77882-1"}]: dispatch 2024-06-11T13:30:45.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:44 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3021699619' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-06-11T13:30:45.033 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:44 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/4147419557' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi057-78504-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:30:45.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:44 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2305832357' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi057-78571-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:30:45.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:44 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/289483390' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi057-77882-1"}]: dispatch 2024-06-11T13:30:45.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:44 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3021699619' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-06-11T13:30:45.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:44 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/4147419557' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi057-78504-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:30:45.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:44 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2305832357' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi057-78571-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:30:45.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:44 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/289483390' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi057-77882-1"}]: dispatch 2024-06-11T13:30:45.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:44 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3021699619' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-06-11T13:30:45.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:44 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/4147419557' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi057-78504-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:30:45.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:44 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2305832357' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi057-78571-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:30:50.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:49 smithi057 ceph-mon[29805]: from='client.34727 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi057-77882-1"}]: dispatch 2024-06-11T13:30:50.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:49 smithi057 ceph-mon[29805]: from='client.34859 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi057-78504-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:30:50.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:49 smithi057 ceph-mon[40588]: from='client.34727 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi057-77882-1"}]: dispatch 2024-06-11T13:30:50.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:49 smithi057 ceph-mon[40588]: from='client.34859 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi057-78504-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:30:50.296 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:49 smithi107 ceph-mon[33178]: from='client.34727 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi057-77882-1"}]: dispatch 2024-06-11T13:30:50.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:49 smithi107 ceph-mon[33178]: from='client.34859 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi057-78504-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:30:51.547 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:30:51 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:30:54.035 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:30:53 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:30:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:30:55.232 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: [==========] Running 4 tests from 1 test suite. 2024-06-11T13:30:55.232 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: [----------] Global test environment set-up. 2024-06-11T13:30:55.233 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: [----------] 4 tests from LibRadosService 2024-06-11T13:30:55.233 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: [ RUN ] LibRadosService.RegisterEarly 2024-06-11T13:30:55.233 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: [ OK ] LibRadosService.RegisterEarly (5056 ms) 2024-06-11T13:30:55.233 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: [ RUN ] LibRadosService.RegisterLate 2024-06-11T13:30:55.233 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: [ OK ] LibRadosService.RegisterLate (16 ms) 2024-06-11T13:30:55.233 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: [ RUN ] LibRadosService.StatusFormat 2024-06-11T13:30:55.233 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: cluster: 2024-06-11T13:30:55.233 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: id: 98a6635e-27f5-11ef-bc9b-c7b262605968 2024-06-11T13:30:55.233 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: health: HEALTH_OK 2024-06-11T13:30:55.233 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 2024-06-11T13:30:55.233 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: services: 2024-06-11T13:30:55.233 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-06-11T13:30:55.233 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: mgr: y(active, since 119s), standbys: x 2024-06-11T13:30:55.234 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: osd: 8 osds: 8 up (since 2m), 8 in (since 3m) 2024-06-11T13:30:55.234 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-06-11T13:30:55.234 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 2024-06-11T13:30:55.234 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: data: 2024-06-11T13:30:55.234 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: pools: 6 pools, 132 pgs 2024-06-11T13:30:55.234 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: objects: 198 objects, 583 KiB 2024-06-11T13:30:55.234 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: usage: 247 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:30:55.234 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: pgs: 132 active+clean 2024-06-11T13:30:55.234 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 2024-06-11T13:30:55.234 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: io: 2024-06-11T13:30:55.234 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: client: 1.2 KiB/s rd, 1 op/s rd, 0 op/s wr 2024-06-11T13:30:55.234 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 2024-06-11T13:30:55.234 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 2024-06-11T13:30:55.235 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: cluster: 2024-06-11T13:30:55.235 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: id: 98a6635e-27f5-11ef-bc9b-c7b262605968 2024-06-11T13:30:55.235 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: health: HEALTH_OK 2024-06-11T13:30:55.235 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 2024-06-11T13:30:55.235 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: services: 2024-06-11T13:30:55.235 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-06-11T13:30:55.235 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: mgr: y(active, since 2m), standbys: x 2024-06-11T13:30:55.235 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: osd: 8 osds: 8 up (since 2m), 8 in (since 3m) 2024-06-11T13:30:55.235 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: laundry: 2 daemons active (1 hosts) 2024-06-11T13:30:55.235 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-06-11T13:30:55.235 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 2024-06-11T13:30:55.235 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: data: 2024-06-11T13:30:55.235 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: pools: 6 pools, 132 pgs 2024-06-11T13:30:55.235 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: objects: 198 objects, 583 KiB 2024-06-11T13:30:55.236 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: usage: 247 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:30:55.236 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: pgs: 132 active+clean 2024-06-11T13:30:55.236 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 2024-06-11T13:30:55.236 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: io: 2024-06-11T13:30:55.236 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: client: 853 B/s rd, 0 op/s rd, 0 op/s wr 2024-06-11T13:30:55.236 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 2024-06-11T13:30:55.236 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 2024-06-11T13:30:55.236 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: cluster: 2024-06-11T13:30:55.236 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: id: 98a6635e-27f5-11ef-bc9b-c7b262605968 2024-06-11T13:30:55.236 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: health: HEALTH_OK 2024-06-11T13:30:55.236 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 2024-06-11T13:30:55.236 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: services: 2024-06-11T13:30:55.236 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-06-11T13:30:55.236 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: mgr: y(active, since 2m), standbys: x 2024-06-11T13:30:55.237 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: osd: 8 osds: 8 up (since 2m), 8 in (since 3m) 2024-06-11T13:30:55.237 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: laundry: 2 daemons active (1 hosts) 2024-06-11T13:30:55.237 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-06-11T13:30:55.237 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 2024-06-11T13:30:55.237 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: data: 2024-06-11T13:30:55.237 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: pools: 6 pools, 132 pgs 2024-06-11T13:30:55.237 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: objects: 198 objects, 583 KiB 2024-06-11T13:30:55.237 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: usage: 247 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:30:55.237 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: pgs: 132 active+clean 2024-06-11T13:30:55.237 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 2024-06-11T13:30:55.237 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: io: 2024-06-11T13:30:55.237 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: client: 853 B/s rd, 0 op/s rd, 0 op/s wr 2024-06-11T13:30:55.237 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 2024-06-11T13:30:55.237 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 2024-06-11T13:30:55.238 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: cluster: 2024-06-11T13:30:55.238 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: id: 98a6635e-27f5-11ef-bc9b-c7b262605968 2024-06-11T13:30:55.238 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: health: HEALTH_WARN 2024-06-11T13:30:55.238 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 2 pool(s) do not have an application enabled 2024-06-11T13:30:55.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:55 smithi057 ceph-mon[29805]: pgmap v62: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:30:55.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:55 smithi057 ceph-mon[29805]: from='client.24901 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi057-78571-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:30:55.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:55 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/289483390' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi057-77882-1"}]: dispatch 2024-06-11T13:30:55.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:55 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/177650484' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi057-77351-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:30:55.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:55 smithi057 ceph-mon[29805]: pgmap v64: 196 pgs: 33 creating+peering, 31 unknown, 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T13:30:55.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:55 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3502500416' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-06-11T13:30:55.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:55 smithi057 ceph-mon[29805]: osdmap e73: 8 total, 8 up, 8 in 2024-06-11T13:30:55.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:55 smithi057 ceph-mon[29805]: from='client.34727 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi057-77882-1"}]: dispatch 2024-06-11T13:30:55.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:55 smithi057 ceph-mon[29805]: from='client.24667 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi057-77351-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:30:55.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:55 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1488776209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi057-77362-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:30:55.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:55 smithi057 ceph-mon[40588]: pgmap v62: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:30:55.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:55 smithi057 ceph-mon[40588]: from='client.24901 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi057-78571-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:30:55.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:55 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/289483390' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi057-77882-1"}]: dispatch 2024-06-11T13:30:55.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:55 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/177650484' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi057-77351-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:30:55.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:55 smithi057 ceph-mon[40588]: pgmap v64: 196 pgs: 33 creating+peering, 31 unknown, 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T13:30:55.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:55 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3502500416' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-06-11T13:30:55.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:55 smithi057 ceph-mon[40588]: osdmap e73: 8 total, 8 up, 8 in 2024-06-11T13:30:55.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:55 smithi057 ceph-mon[40588]: from='client.34727 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi057-77882-1"}]: dispatch 2024-06-11T13:30:55.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:55 smithi057 ceph-mon[40588]: from='client.24667 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi057-77351-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:30:55.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:55 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1488776209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi057-77362-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:30:55.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:55 smithi107 ceph-mon[33178]: pgmap v62: 132 pgs: 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:30:55.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:55 smithi107 ceph-mon[33178]: from='client.24901 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi057-78571-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:30:55.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:55 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/289483390' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi057-77882-1"}]: dispatch 2024-06-11T13:30:55.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:55 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/177650484' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi057-77351-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:30:55.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:55 smithi107 ceph-mon[33178]: pgmap v64: 196 pgs: 33 creating+peering, 31 unknown, 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T13:30:55.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:55 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3502500416' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-06-11T13:30:55.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:55 smithi107 ceph-mon[33178]: osdmap e73: 8 total, 8 up, 8 in 2024-06-11T13:30:55.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:55 smithi107 ceph-mon[33178]: from='client.34727 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi057-77882-1"}]: dispatch 2024-06-11T13:30:55.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:55 smithi107 ceph-mon[33178]: from='client.24667 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi057-77351-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:30:55.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:55 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1488776209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi057-77362-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:30:59.880 INFO:tasks.workunit.client.0.smithi057.stdout: a api_asio: [==========] Running 12 tests from 1 test suite. 2024-06-11T13:30:59.881 INFO:tasks.workunit.client.0.smithi057.stdout: api_asio: [----------] Global test environment set-up. 2024-06-11T13:30:59.881 INFO:tasks.workunit.client.0.smithi057.stdout: api_asio: [----------] 12 tests from AsioRados 2024-06-11T13:30:59.881 INFO:tasks.workunit.client.0.smithi057.stdout: api_asio: [ RUN ] AsioRados.AsyncReadCallback 2024-06-11T13:30:59.881 INFO:tasks.workunit.client.0.smithi057.stdout: api_asio: [ OK ] AsioRados.AsyncReadCallback (1 ms) 2024-06-11T13:30:59.881 INFO:tasks.workunit.client.0.smithi057.stdout: api_asio: [ RUN ] AsioRados.AsyncReadFuture 2024-06-11T13:30:59.881 INFO:tasks.workunit.client.0.smithi057.stdout: api_asio: [ OK ] AsioRados.AsyncReadFuture (1 ms) 2024-06-11T13:30:59.881 INFO:tasks.workunit.client.0.smithi057.stdout: api_asio: [ RUN ] AsioRados.AsyncReadYield 2024-06-11T13:30:59.881 INFO:tasks.workunit.client.0.smithi057.stdout: api_asio: [ OK ] AsioRados.AsyncReadYield (1 ms) 2024-06-11T13:30:59.881 INFO:tasks.workunit.client.0.smithi057.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteCallback 2024-06-11T13:30:59.881 INFO:tasks.workunit.client.0.smithi057.stdout: api_asio: [ OK ] AsioRados.AsyncWriteCallback (4 ms) 2024-06-11T13:30:59.881 INFO:tasks.workunit.client.0.smithi057.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteFuture 2024-06-11T13:30:59.881 INFO:tasks.workunit.client.0.smithi057.stdout: api_asio: [ OK ] AsioRados.AsyncWriteFuture (9 ms) 2024-06-11T13:30:59.881 INFO:tasks.workunit.client.0.smithi057.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteYield 2024-06-11T13:30:59.882 INFO:tasks.workunit.client.0.smithi057.stdout: api_asio: [ OK ] AsioRados.AsyncWriteYield (2 ms) 2024-06-11T13:30:59.882 INFO:tasks.workunit.client.0.smithi057.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationCallback 2024-06-11T13:30:59.882 INFO:tasks.workunit.client.0.smithi057.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationCallback (1 ms) 2024-06-11T13:30:59.882 INFO:tasks.workunit.client.0.smithi057.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationFuture 2024-06-11T13:30:59.882 INFO:tasks.workunit.client.0.smithi057.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationFuture (1 ms) 2024-06-11T13:30:59.882 INFO:tasks.workunit.client.0.smithi057.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationYield 2024-06-11T13:30:59.882 INFO:tasks.workunit.client.0.smithi057.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationYield (0 ms) 2024-06-11T13:30:59.882 INFO:tasks.workunit.client.0.smithi057.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationCallback 2024-06-11T13:30:59.882 INFO:tasks.workunit.client.0.smithi057.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationCallback (2 ms) 2024-06-11T13:30:59.882 INFO:tasks.workunit.client.0.smithi057.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationFuture 2024-06-11T13:30:59.882 INFO:tasks.workunit.client.0.smithi057.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationFuture (2 ms) 2024-06-11T13:30:59.882 INFO:tasks.workunit.client.0.smithi057.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationYield 2024-06-11T13:30:59.882 INFO:tasks.workunit.client.0.smithi057.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationYield (2 ms) 2024-06-11T13:30:59.883 INFO:tasks.workunit.client.0.smithi057.stdout: api_asio: [----------] 12 tests from AsioRados (26 ms total) 2024-06-11T13:30:59.883 INFO:tasks.workunit.client.0.smithi057.stdout: api_asio: 2024-06-11T13:30:59.883 INFO:tasks.workunit.client.0.smithi057.stdout: api_asio: [----------] Global test environment tear-down 2024-06-11T13:30:59.883 INFO:tasks.workunit.client.0.smithi057.stdout: api_asio: [==========] 12 tests from 1 test suite ran. (16278 ms total) 2024-06-11T13:30:59.883 INFO:tasks.workunit.client.0.smithi057.stdout: api_asio: [ PASSED ] 12 tests. 2024-06-11T13:31:00.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[40588]: pgmap v65: 196 pgs: 33 creating+peering, 31 unknown, 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T13:31:00.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[40588]: from='client.34859 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi057-78504-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:00.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/4147419557' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi057-78504-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi057-78504-1"}]: dispatch 2024-06-11T13:31:00.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1855176019' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi057-77374-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:00.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi057-77430-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:00.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi057-77576-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:00.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3684754234' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi057-77396-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:00.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2733316561' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi057-77506-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:00.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1241524721' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi057-77494-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:00.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[40588]: pgmap v67: 580 pgs: 33 creating+peering, 415 unknown, 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-06-11T13:31:00.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:31:00.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/289483390' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi057-77882-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:00.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:31:00.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3786528187' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-06-11T13:31:00.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[40588]: pgmap v68: 580 pgs: 298 creating+peering, 150 unknown, 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:31:00.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/934772732' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-06-11T13:31:00.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[40588]: pgmap v69: 580 pgs: 298 creating+peering, 150 unknown, 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-06-11T13:31:00.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[40588]: osdmap e74: 8 total, 8 up, 8 in 2024-06-11T13:31:00.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[40588]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:31:00.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[40588]: from='client.24685 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi057-77374-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:00.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[40588]: from='client.34859 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi057-78504-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi057-78504-1"}]: dispatch 2024-06-11T13:31:00.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[40588]: from='client.24709 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi057-77430-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:00.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[40588]: from='client.24721 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi057-77576-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:00.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[40588]: from='client.24682 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi057-77396-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:00.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[40588]: from='client.24700 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi057-77506-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:00.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[40588]: from='client.24694 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi057-77494-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:00.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[40588]: from='client.34727 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi057-77882-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:00.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1248647670' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-06-11T13:31:00.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[29805]: pgmap v65: 196 pgs: 33 creating+peering, 31 unknown, 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T13:31:00.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[29805]: from='client.34859 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi057-78504-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:00.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/4147419557' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi057-78504-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi057-78504-1"}]: dispatch 2024-06-11T13:31:00.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1855176019' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi057-77374-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:00.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi057-77430-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:00.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi057-77576-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:00.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3684754234' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi057-77396-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:00.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2733316561' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi057-77506-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:00.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1241524721' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi057-77494-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:00.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[29805]: pgmap v67: 580 pgs: 33 creating+peering, 415 unknown, 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-06-11T13:31:00.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:31:00.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/289483390' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi057-77882-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:00.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:31:00.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3786528187' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-06-11T13:31:00.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[29805]: pgmap v68: 580 pgs: 298 creating+peering, 150 unknown, 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:31:00.038 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/934772732' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-06-11T13:31:00.038 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[29805]: pgmap v69: 580 pgs: 298 creating+peering, 150 unknown, 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-06-11T13:31:00.038 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[29805]: osdmap e74: 8 total, 8 up, 8 in 2024-06-11T13:31:00.038 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[29805]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:31:00.038 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[29805]: from='client.24685 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi057-77374-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:00.038 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[29805]: from='client.34859 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi057-78504-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi057-78504-1"}]: dispatch 2024-06-11T13:31:00.038 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[29805]: from='client.24709 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi057-77430-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:00.038 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[29805]: from='client.24721 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi057-77576-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:00.038 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[29805]: from='client.24682 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi057-77396-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:00.038 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[29805]: from='client.24700 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi057-77506-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:00.038 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[29805]: from='client.24694 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi057-77494-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:00.038 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[29805]: from='client.34727 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi057-77882-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:00.038 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:30:59 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1248647670' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-06-11T13:31:00.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:59 smithi107 ceph-mon[33178]: pgmap v65: 196 pgs: 33 creating+peering, 31 unknown, 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T13:31:00.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:59 smithi107 ceph-mon[33178]: from='client.34859 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi057-78504-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:00.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:59 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/4147419557' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi057-78504-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi057-78504-1"}]: dispatch 2024-06-11T13:31:00.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:59 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1855176019' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi057-77374-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:00.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:59 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi057-77430-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:00.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:59 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi057-77576-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:00.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:59 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3684754234' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi057-77396-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:00.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:59 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2733316561' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi057-77506-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:00.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:59 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1241524721' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi057-77494-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:00.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:59 smithi107 ceph-mon[33178]: pgmap v67: 580 pgs: 33 creating+peering, 415 unknown, 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-06-11T13:31:00.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:59 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:31:00.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:59 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/289483390' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi057-77882-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:00.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:59 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:31:00.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:59 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3786528187' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-06-11T13:31:00.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:59 smithi107 ceph-mon[33178]: pgmap v68: 580 pgs: 298 creating+peering, 150 unknown, 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:31:00.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:59 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/934772732' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-06-11T13:31:00.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:59 smithi107 ceph-mon[33178]: pgmap v69: 580 pgs: 298 creating+peering, 150 unknown, 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-06-11T13:31:00.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:59 smithi107 ceph-mon[33178]: osdmap e74: 8 total, 8 up, 8 in 2024-06-11T13:31:00.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:59 smithi107 ceph-mon[33178]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:31:00.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:59 smithi107 ceph-mon[33178]: from='client.24685 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi057-77374-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:00.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:59 smithi107 ceph-mon[33178]: from='client.34859 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi057-78504-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi057-78504-1"}]: dispatch 2024-06-11T13:31:00.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:59 smithi107 ceph-mon[33178]: from='client.24709 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi057-77430-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:00.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:59 smithi107 ceph-mon[33178]: from='client.24721 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi057-77576-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:00.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:59 smithi107 ceph-mon[33178]: from='client.24682 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi057-77396-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:00.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:59 smithi107 ceph-mon[33178]: from='client.24700 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi057-77506-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:00.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:59 smithi107 ceph-mon[33178]: from='client.24694 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi057-77494-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:00.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:59 smithi107 ceph-mon[33178]: from='client.34727 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi057-77882-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:00.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:30:59 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1248647670' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-06-11T13:31:00.125 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: [==========] Running 11 tests from 3 test suites. 2024-06-11T13:31:00.125 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: [----------] Global test environment set-up. 2024-06-11T13:31:00.125 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: [----------] 7 tests from LibRadosList 2024-06-11T13:31:00.125 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: [ RUN ] LibRadosList.ListObjects 2024-06-11T13:31:00.126 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: [ OK ] LibRadosList.ListObjects (196 ms) 2024-06-11T13:31:00.126 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: [ RUN ] LibRadosList.ListObjectsZeroInName 2024-06-11T13:31:00.126 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: [ OK ] LibRadosList.ListObjectsZeroInName (22 ms) 2024-06-11T13:31:00.126 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: [ RUN ] LibRadosList.ListObjectsNS 2024-06-11T13:31:00.126 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: myset foo1,foo2,foo3 2024-06-11T13:31:00.126 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: foo1 2024-06-11T13:31:00.126 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: foo2 2024-06-11T13:31:00.126 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: foo3 2024-06-11T13:31:00.126 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: myset foo1,foo4,foo5 2024-06-11T13:31:00.126 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: foo4 2024-06-11T13:31:00.126 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: foo5 2024-06-11T13:31:00.126 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: foo1 2024-06-11T13:31:00.127 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: myset foo6,foo7 2024-06-11T13:31:00.127 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: foo7 2024-06-11T13:31:00.127 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: foo6 2024-06-11T13:31:00.127 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-06-11T13:31:00.127 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: ns1:foo4 2024-06-11T13:31:00.127 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: ns1:foo5 2024-06-11T13:31:00.127 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: ns2:foo7 2024-06-11T13:31:00.127 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: ns2:foo6 2024-06-11T13:31:00.127 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: ns1:foo1 2024-06-11T13:31:00.127 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: :foo1 2024-06-11T13:31:00.127 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: :foo2 2024-06-11T13:31:00.127 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: :foo3 2024-06-11T13:31:00.128 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: [ OK ] LibRadosList.ListObjectsNS (72 ms) 2024-06-11T13:31:00.128 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: [ RUN ] LibRadosList.ListObjectsStart 2024-06-11T13:31:00.128 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: 1 0 2024-06-11T13:31:00.128 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: 10 0 2024-06-11T13:31:00.128 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: 13 0 2024-06-11T13:31:00.128 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: 7 0 2024-06-11T13:31:00.128 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: 14 0 2024-06-11T13:31:00.128 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: 0 0 2024-06-11T13:31:00.128 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: 15 0 2024-06-11T13:31:00.128 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: 11 0 2024-06-11T13:31:00.128 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: 5 0 2024-06-11T13:31:00.128 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: 8 0 2024-06-11T13:31:00.128 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: 6 0 2024-06-11T13:31:00.128 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: 3 0 2024-06-11T13:31:00.128 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: 4 0 2024-06-11T13:31:00.129 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: 12 0 2024-06-11T13:31:00.129 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: 9 0 2024-06-11T13:31:00.129 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: 2 0 2024-06-11T13:31:00.129 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-06-11T13:31:00.129 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: [ OK ] LibRadosList.ListObjectsStart (61 ms) 2024-06-11T13:31:00.129 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: [ RUN ] LibRadosList.ListObjectsCursor 2024-06-11T13:31:00.129 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: x cursor=MIN 2024-06-11T13:31:00.129 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > oid=1 cursor=17:02547ec2:::1:head 2024-06-11T13:31:00.129 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > oid=10 cursor=17:52ea6a34:::10:head 2024-06-11T13:31:00.129 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > oid=13 cursor=17:566253c9:::13:head 2024-06-11T13:31:00.129 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > oid=7 cursor=17:5c6b0b28:::7:head 2024-06-11T13:31:00.130 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > oid=14 cursor=17:62a1935d:::14:head 2024-06-11T13:31:00.130 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > oid=0 cursor=17:6cac518f:::0:head 2024-06-11T13:31:00.130 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > oid=15 cursor=17:863748b0:::15:head 2024-06-11T13:31:00.130 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > oid=11 cursor=17:89d3ae78:::11:head 2024-06-11T13:31:00.130 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > oid=5 cursor=17:b29083e3:::5:head 2024-06-11T13:31:00.130 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > oid=8 cursor=17:bd63b0f1:::8:head 2024-06-11T13:31:00.130 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > oid=6 cursor=17:c4fdafeb:::6:head 2024-06-11T13:31:00.130 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > oid=3 cursor=17:cfc208b3:::3:head 2024-06-11T13:31:00.130 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > oid=4 cursor=17:d83876eb:::4:head 2024-06-11T13:31:00.130 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > oid=12 cursor=17:de5d7c5f:::12:head 2024-06-11T13:31:00.130 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > oid=9 cursor=17:e960b815:::9:head 2024-06-11T13:31:00.131 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > oid=2 cursor=17:f905c69b:::2:head 2024-06-11T13:31:00.131 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: FIRST> seek to MIN oid=1 2024-06-11T13:31:00.131 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : oid=1 cursor=17:02547ec2:::1:head 2024-06-11T13:31:00.131 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : seek to 17:02547ec2:::1:head 2024-06-11T13:31:00.131 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > 17:02547ec2:::1:head -> 1 2024-06-11T13:31:00.131 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : oid=10 cursor=17:52ea6a34:::10:head 2024-06-11T13:31:00.131 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : seek to 17:52ea6a34:::10:head 2024-06-11T13:31:00.131 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > 17:52ea6a34:::10:head -> 10 2024-06-11T13:31:00.131 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : oid=13 cursor=17:566253c9:::13:head 2024-06-11T13:31:00.131 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : seek to 17:566253c9:::13:head 2024-06-11T13:31:00.131 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > 17:566253c9:::13:head -> 13 2024-06-11T13:31:00.131 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : oid=7 cursor=17:5c6b0b28:::7:head 2024-06-11T13:31:00.131 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : seek to 17:5c6b0b28:::7:head 2024-06-11T13:31:00.131 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > 17:5c6b0b28:::7:head -> 7 2024-06-11T13:31:00.178 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : oid=14 cursor=17:62a1935d:::14:head 2024-06-11T13:31:00.178 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : seek to 17:62a1935d:::14:head 2024-06-11T13:31:00.179 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > 17:62a1935d:::14:head -> 14 2024-06-11T13:31:00.179 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : oid=0 cursor=17:6cac518f:::0:head 2024-06-11T13:31:00.179 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : seek to 17:6cac518f:::0:head 2024-06-11T13:31:00.179 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > 17:6cac518f:::0:head -> 0 2024-06-11T13:31:00.179 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : oid=15 cursor=17:863748b0:::15:head 2024-06-11T13:31:00.179 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : seek to 17:863748b0:::15:head 2024-06-11T13:31:00.179 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > 17:863748b0:::15:head -> 15 2024-06-11T13:31:00.179 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : oid=11 cursor=17:89d3ae78:::11:head 2024-06-11T13:31:00.179 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : seek to 17:89d3ae78:::11:head 2024-06-11T13:31:00.179 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > 17:89d3ae78:::11:head -> 11 2024-06-11T13:31:00.179 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : oid=5 cursor=17:b29083e3:::5:head 2024-06-11T13:31:00.179 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : seek to 17:b29083e3:::5:head 2024-06-11T13:31:00.180 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > 17:b29083e3:::5:head -> 5 2024-06-11T13:31:00.180 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : oid=8 cursor=17:bd63b0f1:::8:head 2024-06-11T13:31:00.180 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : seek to 17:bd63b0f1:::8:head 2024-06-11T13:31:00.180 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > 17:bd63b0f1:::8:head -> 8 2024-06-11T13:31:00.180 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : oid=6 cursor=17:c4fdafeb:::6:head 2024-06-11T13:31:00.180 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : seek to 17:c4fdafeb:::6:head 2024-06-11T13:31:00.180 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > 17:c4fdafeb:::6:head -> 6 2024-06-11T13:31:00.180 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : oid=3 cursor=17:cfc208b3:::3:head 2024-06-11T13:31:00.180 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : seek to 17:cfc208b3:::3:head 2024-06-11T13:31:00.180 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > 17:cfc208b3:::3:head -> 3 2024-06-11T13:31:00.180 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : oid=4 cursor=17:d83876eb:::4:head 2024-06-11T13:31:00.180 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : seek to 17:d83876eb:::4:head 2024-06-11T13:31:00.180 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > 17:d83876eb:::4:head -> 4 2024-06-11T13:31:00.181 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : oid=12 cursor=17:de5d7c5f:::12:head 2024-06-11T13:31:00.181 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : seek to 17:de5d7c5f:::12:head 2024-06-11T13:31:00.181 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > 17:de5d7c5f:::12:head -> 12 2024-06-11T13:31:00.181 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : oid=9 cursor=17:e960b815:::9:head 2024-06-11T13:31:00.181 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : seek to 17:e960b815:::9:head 2024-06-11T13:31:00.181 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > 17:e960b815:::9:head -> 9 2024-06-11T13:31:00.181 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : oid=2 cursor=17:f905c69b:::2:head 2024-06-11T13:31:00.181 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : seek to 17:f905c69b:::2:head 2024-06-11T13:31:00.181 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > 17:f905c69b:::2:head -> 2 2024-06-11T13:31:00.181 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : seek to 17:52ea6a34:::10:head 2024-06-11T13:31:00.181 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : cursor()=17:52ea6a34:::10:head expected=17:52ea6a34:::10:head 2024-06-11T13:31:00.181 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > 17:52ea6a34:::10:head -> 10 2024-06-11T13:31:00.181 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : entry=10 expected=10 2024-06-11T13:31:00.181 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : seek to 17:c4fdafeb:::6:head 2024-06-11T13:31:00.181 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : cursor()=17:c4fdafeb:::6:head expected=17:c4fdafeb:::6:head 2024-06-11T13:31:00.181 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > 17:c4fdafeb:::6:head -> 6 2024-06-11T13:31:00.182 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : entry=6 expected=6 2024-06-11T13:31:00.182 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : seek to 17:bd63b0f1:::8:head 2024-06-11T13:31:00.182 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : cursor()=17:bd63b0f1:::8:head expected=17:bd63b0f1:::8:head 2024-06-11T13:31:00.182 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > 17:bd63b0f1:::8:head -> 8 2024-06-11T13:31:00.182 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : entry=8 expected=8 2024-06-11T13:31:00.182 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : seek to 17:de5d7c5f:::12:head 2024-06-11T13:31:00.182 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : cursor()=17:de5d7c5f:::12:head expected=17:de5d7c5f:::12:head 2024-06-11T13:31:00.182 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > 17:de5d7c5f:::12:head -> 12 2024-06-11T13:31:00.182 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : entry=12 expected=12 2024-06-11T13:31:00.182 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : seek to 17:d83876eb:::4:head 2024-06-11T13:31:00.182 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : cursor()=17:d83876eb:::4:head expected=17:d83876eb:::4:head 2024-06-11T13:31:00.182 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > 17:d83876eb:::4:head -> 4 2024-06-11T13:31:00.183 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : entry=4 expected=4 2024-06-11T13:31:00.183 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : seek to 17:cfc208b3:::3:head 2024-06-11T13:31:00.183 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : cursor()=17:cfc208b3:::3:head expected=17:cfc208b3:::3:head 2024-06-11T13:31:00.183 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > 17:cfc208b3:::3:head -> 3 2024-06-11T13:31:00.183 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : entry=3 expected=3 2024-06-11T13:31:00.183 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : seek to 17:b29083e3:::5:head 2024-06-11T13:31:00.183 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : cursor()=17:b29083e3:::5:head expected=17:b29083e3:::5:head 2024-06-11T13:31:00.183 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > 17:b29083e3:::5:head -> 5 2024-06-11T13:31:00.183 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : entry=5 expected=5 2024-06-11T13:31:00.183 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : seek to 17:89d3ae78:::11:head 2024-06-11T13:31:00.183 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : cursor()=17:89d3ae78:::11:head expected=17:89d3ae78:::11:head 2024-06-11T13:31:00.183 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > 17:89d3ae78:::11:head -> 11 2024-06-11T13:31:01.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2075751813' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3764695880' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi057-77714-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2557045370' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi057-77785-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1311563201' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi057-77831-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/729777987' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi057-77874-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1227277278' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi057-78096-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/100913204' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2914256649' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi057-77799-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2932398727' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi057-77599-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[29805]: pgmap v71: 964 pgs: 669 creating+peering, 163 unknown, 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:31:01.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3530622050' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-06-11T13:31:01.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[29805]: pgmap v72: 964 pgs: 669 creating+peering, 163 unknown, 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:31:01.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1177552964' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-06-11T13:31:01.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[29805]: osdmap e75: 8 total, 8 up, 8 in 2024-06-11T13:31:01.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[29805]: from='client.24724 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[29805]: from='client.24733 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi057-77714-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[29805]: from='client.24739 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi057-77785-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[29805]: from='client.24754 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi057-77831-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[29805]: from='client.24763 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi057-77874-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[29805]: from='client.24787 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi057-78096-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[29805]: from='client.24802 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[29805]: from='client.24745 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi057-77799-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[29805]: from='client.24727 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi057-77599-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[29805]: from='client.24685 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi057-77374-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:01.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[29805]: from='client.24709 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi057-77430-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:01.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[29805]: from='client.24721 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi057-77576-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:01.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[29805]: from='client.24682 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi057-77396-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:01.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[29805]: from='client.24700 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi057-77506-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:01.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[29805]: from='client.24694 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi057-77494-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:01.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[29805]: from='client.34727 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi057-77882-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:01.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[29805]: from='client.24901 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi057-78571-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:01.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[29805]: from='client.24667 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi057-77351-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:01.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1488776209' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi057-77362-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:01.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2305832357' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi057-78571-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi057-78571-1"}]: dispatch 2024-06-11T13:31:01.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/289483390' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi057-77882-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi057-77882-1"}]: dispatch 2024-06-11T13:31:01.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2132387160' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78142-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2038330211' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77975-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.038 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[29805]: osdmap e76: 8 total, 8 up, 8 in 2024-06-11T13:31:01.038 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/912185714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77928-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.038 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[29805]: from='client.24901 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi057-78571-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi057-78571-1"}]: dispatch 2024-06-11T13:31:01.038 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[29805]: from='client.34727 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi057-77882-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi057-77882-1"}]: dispatch 2024-06-11T13:31:01.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2075751813' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3764695880' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi057-77714-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2557045370' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi057-77785-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1311563201' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi057-77831-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/729777987' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi057-77874-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1227277278' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi057-78096-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.039 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/100913204' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.039 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2914256649' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi057-77799-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.039 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2932398727' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi057-77599-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.039 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[40588]: pgmap v71: 964 pgs: 669 creating+peering, 163 unknown, 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:31:01.039 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3530622050' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-06-11T13:31:01.039 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[40588]: pgmap v72: 964 pgs: 669 creating+peering, 163 unknown, 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:31:01.039 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1177552964' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-06-11T13:31:01.039 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[40588]: osdmap e75: 8 total, 8 up, 8 in 2024-06-11T13:31:01.039 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[40588]: from='client.24724 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.039 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[40588]: from='client.24733 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi057-77714-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.039 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[40588]: from='client.24739 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi057-77785-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.039 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[40588]: from='client.24754 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi057-77831-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.039 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[40588]: from='client.24763 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi057-77874-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.039 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[40588]: from='client.24787 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi057-78096-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.040 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[40588]: from='client.24802 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.040 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[40588]: from='client.24745 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi057-77799-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.040 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[40588]: from='client.24727 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi057-77599-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.040 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[40588]: from='client.24685 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi057-77374-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:01.040 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[40588]: from='client.24709 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi057-77430-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:01.040 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[40588]: from='client.24721 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi057-77576-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:01.040 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[40588]: from='client.24682 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi057-77396-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:01.040 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[40588]: from='client.24700 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi057-77506-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:01.040 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[40588]: from='client.24694 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi057-77494-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:01.040 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[40588]: from='client.34727 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi057-77882-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:01.040 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[40588]: from='client.24901 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi057-78571-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:01.040 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[40588]: from='client.24667 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi057-77351-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:01.040 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1488776209' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi057-77362-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:01.040 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2305832357' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi057-78571-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi057-78571-1"}]: dispatch 2024-06-11T13:31:01.040 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/289483390' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi057-77882-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi057-77882-1"}]: dispatch 2024-06-11T13:31:01.041 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2132387160' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78142-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.041 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2038330211' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77975-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.041 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[40588]: osdmap e76: 8 total, 8 up, 8 in 2024-06-11T13:31:01.041 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/912185714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77928-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.041 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[40588]: from='client.24901 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi057-78571-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi057-78571-1"}]: dispatch 2024-06-11T13:31:01.041 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:00 smithi057 ceph-mon[40588]: from='client.34727 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi057-77882-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi057-77882-1"}]: dispatch 2024-06-11T13:31:01.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:00 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2075751813' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:00 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3764695880' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi057-77714-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:00 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2557045370' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi057-77785-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:00 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1311563201' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi057-77831-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:00 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/729777987' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi057-77874-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:00 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1227277278' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi057-78096-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:00 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/100913204' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:00 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2914256649' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi057-77799-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:00 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2932398727' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi057-77599-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:00 smithi107 ceph-mon[33178]: pgmap v71: 964 pgs: 669 creating+peering, 163 unknown, 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:31:01.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:00 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3530622050' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-06-11T13:31:01.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:00 smithi107 ceph-mon[33178]: pgmap v72: 964 pgs: 669 creating+peering, 163 unknown, 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:31:01.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:00 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1177552964' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-06-11T13:31:01.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:00 smithi107 ceph-mon[33178]: osdmap e75: 8 total, 8 up, 8 in 2024-06-11T13:31:01.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:00 smithi107 ceph-mon[33178]: from='client.24724 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:00 smithi107 ceph-mon[33178]: from='client.24733 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi057-77714-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:00 smithi107 ceph-mon[33178]: from='client.24739 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi057-77785-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:00 smithi107 ceph-mon[33178]: from='client.24754 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi057-77831-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:00 smithi107 ceph-mon[33178]: from='client.24763 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi057-77874-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:00 smithi107 ceph-mon[33178]: from='client.24787 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi057-78096-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:00 smithi107 ceph-mon[33178]: from='client.24802 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:00 smithi107 ceph-mon[33178]: from='client.24745 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi057-77799-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:00 smithi107 ceph-mon[33178]: from='client.24727 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi057-77599-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:00 smithi107 ceph-mon[33178]: from='client.24685 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi057-77374-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:01.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:00 smithi107 ceph-mon[33178]: from='client.24709 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi057-77430-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:01.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:00 smithi107 ceph-mon[33178]: from='client.24721 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi057-77576-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:01.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:00 smithi107 ceph-mon[33178]: from='client.24682 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi057-77396-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:01.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:00 smithi107 ceph-mon[33178]: from='client.24700 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi057-77506-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:01.050 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:00 smithi107 ceph-mon[33178]: from='client.24694 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi057-77494-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:01.050 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:00 smithi107 ceph-mon[33178]: from='client.34727 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi057-77882-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:01.050 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:00 smithi107 ceph-mon[33178]: from='client.24901 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi057-78571-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:01.050 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:00 smithi107 ceph-mon[33178]: from='client.24667 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi057-77351-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:01.050 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:00 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1488776209' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi057-77362-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:01.050 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:00 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2305832357' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi057-78571-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi057-78571-1"}]: dispatch 2024-06-11T13:31:01.050 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:00 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/289483390' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi057-77882-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi057-77882-1"}]: dispatch 2024-06-11T13:31:01.050 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:00 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2132387160' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78142-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.050 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:00 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2038330211' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77975-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.050 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:00 smithi107 ceph-mon[33178]: osdmap e76: 8 total, 8 up, 8 in 2024-06-11T13:31:01.050 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:00 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/912185714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77928-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:01.050 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:00 smithi107 ceph-mon[33178]: from='client.24901 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi057-78571-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi057-78571-1"}]: dispatch 2024-06-11T13:31:01.050 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:00 smithi107 ceph-mon[33178]: from='client.34727 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi057-77882-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi057-77882-1"}]: dispatch 2024-06-11T13:31:01.288 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : entpi_service: 2024-06-11T13:31:01.288 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: services: 2024-06-11T13:31:01.288 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-06-11T13:31:01.288 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: mgr: y(active, since 2m), standbys: x 2024-06-11T13:31:01.288 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: osd: 8 osds: 8 up (since 2m), 8 in (since 3m) 2024-06-11T13:31:01.288 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: laundry: 2 daemons active (1 hosts) 2024-06-11T13:31:01.288 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-06-11T13:31:01.288 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 2024-06-11T13:31:01.288 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: data: 2024-06-11T13:31:01.288 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: pools: 8 pools, 196 pgs 2024-06-11T13:31:01.288 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: objects: 198 objects, 583 KiB 2024-06-11T13:31:01.289 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: usage: 247 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:31:01.289 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: pgs: 15.816% pgs unknown 2024-06-11T13:31:01.289 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 16.837% pgs not active 2024-06-11T13:31:01.289 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 132 active+clean 2024-06-11T13:31:01.289 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 33 creating+peering 2024-06-11T13:31:01.289 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 31 unknown 2024-06-11T13:31:01.289 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 2024-06-11T13:31:01.289 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: io: 2024-06-11T13:31:01.289 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: client: 1023 B/s rd, 0 op/s rd, 0 op/s wr 2024-06-11T13:31:01.289 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 2024-06-11T13:31:01.289 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 2024-06-11T13:31:01.289 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: cluster: 2024-06-11T13:31:01.289 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: id: 98a6635e-27f5-11ef-bc9b-c7b262605968 2024-06-11T13:31:01.289 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: health: HEALTH_WARN 2024-06-11T13:31:01.290 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 2 pool(s) do not have an application enabled 2024-06-11T13:31:01.290 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 2024-06-11T13:31:01.290 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: services: 2024-06-11T13:31:01.290 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-06-11T13:31:01.290 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: mgr: y(active, since 2m), standbys: x 2024-06-11T13:31:01.290 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: osd: 8 osds: 8 up (since 3m), 8 in (since 3m) 2024-06-11T13:31:01.290 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: laundry: 2 daemons active (1 hosts) 2024-06-11T13:31:01.290 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-06-11T13:31:01.290 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 2024-06-11T13:31:01.290 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: data: 2024-06-11T13:31:01.290 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: pools: 8 pools, 196 pgs 2024-06-11T13:31:01.290 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: objects: 198 objects, 583 KiB 2024-06-11T13:31:01.290 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: usage: 247 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:31:01.290 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: pgs: 15.816% pgs unknown 2024-06-11T13:31:01.291 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 16.837% pgs not active 2024-06-11T13:31:01.291 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 132 active+clean 2024-06-11T13:31:01.291 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 33 creating+peering 2024-06-11T13:31:01.291 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 31 unknown 2024-06-11T13:31:01.291 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 2024-06-11T13:31:01.291 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: io: 2024-06-11T13:31:01.291 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: client: 1023 B/s rd, 0 op/s rd, 0 op/s wr 2024-06-11T13:31:01.291 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 2024-06-11T13:31:01.291 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 2024-06-11T13:31:01.291 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: cluster: 2024-06-11T13:31:01.291 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: id: 98a6635e-27f5-11ef-bc9b-c7b262605968 2024-06-11T13:31:01.291 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: health: HEALTH_WARN 2024-06-11T13:31:01.291 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 2 pool(s) do not have an application enabled 2024-06-11T13:31:01.291 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 2024-06-11T13:31:01.291 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: services: 2024-06-11T13:31:01.292 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-06-11T13:31:01.292 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: mgr: y(active, since 2m), standbys: x 2024-06-11T13:31:01.292 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: osd: 8 osds: 8 up (since 3m), 8 in (since 3m) 2024-06-11T13:31:01.292 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: laundry: 2 daemons active (1 hosts) 2024-06-11T13:31:01.292 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-06-11T13:31:01.292 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 2024-06-11T13:31:01.292 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: data: 2024-06-11T13:31:01.292 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: pools: 8 pools, 196 pgs 2024-06-11T13:31:01.292 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: objects: 198 objects, 583 KiB 2024-06-11T13:31:01.292 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: usage: 247 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:31:01.292 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: pgs: 15.816% pgs unknown 2024-06-11T13:31:01.292 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 16.837% pgs not active 2024-06-11T13:31:01.292 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 132 active+clean 2024-06-11T13:31:01.293 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 33 creating+peering 2024-06-11T13:31:01.293 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 31 unknown 2024-06-11T13:31:01.293 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 2024-06-11T13:31:01.293 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: io: 2024-06-11T13:31:01.293 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: client: 1023 B/s rd, 0 op/s rd, 0 op/s wr 2024-06-11T13:31:01.293 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 2024-06-11T13:31:01.293 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 2024-06-11T13:31:01.293 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: cluster: 2024-06-11T13:31:01.337 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:31:01 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:31:03.200 INFO:tasks.workunit.client.0.smithi057.stdout: api_servi list_parallel: process_1_[78190]: starting. 2024-06-11T13:31:03.200 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_1_[78190]: creating pool ceph_test_rados_list_parallel.smithi057-78162 2024-06-11T13:31:03.200 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_1_[78190]: created object 0... 2024-06-11T13:31:03.200 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_1_[78190]: created object 25... 2024-06-11T13:31:03.200 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_1_[78190]: created object 49... 2024-06-11T13:31:03.200 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_1_[78190]: finishing. 2024-06-11T13:31:03.200 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_1_[78190]: shutting down. 2024-06-11T13:31:03.200 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_2_[78191]: starting. 2024-06-11T13:31:03.201 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_2_[78191]: listing objects. 2024-06-11T13:31:03.201 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_2_[78191]: listed object 0... 2024-06-11T13:31:03.201 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_2_[78191]: listed object 25... 2024-06-11T13:31:03.201 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_2_[78191]: saw 50 objects 2024-06-11T13:31:03.201 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_2_[78191]: shutting down. 2024-06-11T13:31:03.201 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: ******************************* 2024-06-11T13:31:03.201 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_3_[79506]: starting. 2024-06-11T13:31:03.201 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_3_[79506]: creating pool ceph_test_rados_list_parallel.smithi057-78162 2024-06-11T13:31:03.201 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_3_[79506]: created object 0... 2024-06-11T13:31:03.201 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_3_[79506]: created object 25... 2024-06-11T13:31:03.201 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_3_[79506]: created object 49... 2024-06-11T13:31:03.201 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_3_[79506]: finishing. 2024-06-11T13:31:03.201 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_3_[79506]: shutting down. 2024-06-11T13:31:03.202 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: ******************************* 2024-06-11T13:31:03.202 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_4_[79507]: starting. 2024-06-11T13:31:03.202 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_4_[79507]: listing objects. 2024-06-11T13:31:03.202 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_4_[79507]: listed object 0... 2024-06-11T13:31:03.202 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_4_[79507]: listed object 25... 2024-06-11T13:31:03.202 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_4_[79507]: saw 43 objects 2024-06-11T13:31:03.202 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_4_[79507]: shutting down. 2024-06-11T13:31:03.202 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: ******************************* 2024-06-11T13:31:03.202 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_5_[79508]: starting. 2024-06-11T13:31:03.202 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_5_[79508]: removed 25 objects... 2024-06-11T13:31:03.202 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_5_[79508]: removed half of the objects 2024-06-11T13:31:03.202 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_5_[79508]: removed 50 objects... 2024-06-11T13:31:03.202 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_5_[79508]: removed 50 objects 2024-06-11T13:31:03.203 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_5_[79508]: shutting down. 2024-06-11T13:31:03.203 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: ******************************* 2024-06-11T13:31:03.203 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: ******************************* 2024-06-11T13:31:03.203 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_6_[79554]: starting. 2024-06-11T13:31:03.203 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_6_[79554]: creating pool ceph_test_rados_list_parallel.smithi057-78162 2024-06-11T13:31:03.203 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_6_[79554]: created object 0... 2024-06-11T13:31:03.203 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_6_[79554]: created object 25... 2024-06-11T13:31:03.203 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_6_[79554]: created object 49... 2024-06-11T13:31:03.203 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_6_[79554]: finishing. 2024-06-11T13:31:03.203 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_6_[79554]: shutting down. 2024-06-11T13:31:03.203 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: ******************************* 2024-06-11T13:31:03.203 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: ******************************* 2024-06-11T13:31:03.203 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_7_[79555]: starting. 2024-06-11T13:31:03.203 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_7_[79555]: listing objects. 2024-06-11T13:31:03.203 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_7_[79555]: listed object 0... 2024-06-11T13:31:03.204 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_7_[79555]: listed object 25... 2024-06-11T13:31:03.204 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_7_[79555]: listed object 50... 2024-06-11T13:31:03.204 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_7_[79555]: saw 56 objects 2024-06-11T13:31:03.204 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_7_[79555]: shutting down. 2024-06-11T13:31:03.204 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: ******************************* 2024-06-11T13:31:03.204 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: ******************************* 2024-06-11T13:31:03.204 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_8_[79556]: starting. 2024-06-11T13:31:03.204 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_8_[79556]: added 25 objects... 2024-06-11T13:31:03.204 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_8_[79556]: added half of the objects 2024-06-11T13:31:03.204 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_8_[79556]: added 50 objects... 2024-06-11T13:31:03.204 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_8_[79556]: added 50 objects 2024-06-11T13:31:03.204 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_8_[79556]: shutting down. 2024-06-11T13:31:03.204 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: ******************************* 2024-06-11T13:31:03.204 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: ******************************* 2024-06-11T13:31:03.818 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: ******************************* 2024-06-11T13:31:03.818 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_9_[79602]: starting. 2024-06-11T13:31:03.818 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_9_[79602]: creating pool ceph_test_rados_list_parallel.smithi057-78162 2024-06-11T13:31:03.818 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_9_[79602]: created object 0... 2024-06-11T13:31:03.818 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_9_[79602]: created object 25... 2024-06-11T13:31:03.818 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_9_[79602]: created object 49... 2024-06-11T13:31:03.818 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_9_[79602]: finishing. 2024-06-11T13:31:03.819 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_9_[79602]: shutting down. 2024-06-11T13:31:03.819 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: ******************************* 2024-06-11T13:31:03.819 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: ******************************* 2024-06-11T13:31:03.819 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: ******************************* 2024-06-11T13:31:03.819 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_10_[79603]: starting. 2024-06-11T13:31:03.819 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_10_[79603]: listing objects. 2024-06-11T13:31:03.819 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_10_[79603]: listed object 0... 2024-06-11T13:31:03.819 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_10_[79603]: listed object 25... 2024-06-11T13:31:03.819 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_10_[79603]: listed object 50... 2024-06-11T13:31:03.819 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_10_[79603]: listed object 75... 2024-06-11T13:31:03.819 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_10_[79603]: saw 97 objects 2024-06-11T13:31:03.819 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_10_[79603]: shutting down. 2024-06-11T13:31:03.819 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: ******************************* 2024-06-11T13:31:03.819 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: ******************************* 2024-06-11T13:31:03.820 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: ******************************* 2024-06-11T13:31:03.820 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_13_[79606]: starting. 2024-06-11T13:31:03.820 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_13_[79606]: removed 25 objects... 2024-06-11T13:31:03.820 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_13_[79606]: removed half of the objects 2024-06-11T13:31:03.820 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_13_[79606]: removed 50 objects... 2024-06-11T13:31:03.820 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_13_[79606]: removed 50 objects 2024-06-11T13:31:03.820 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_13_[79606]: shutting down. 2024-06-11T13:31:03.820 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: ******************************* 2024-06-11T13:31:03.820 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: ******************************* 2024-06-11T13:31:03.820 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: ******************************* 2024-06-11T13:31:03.820 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_11_[79604]: starting. 2024-06-11T13:31:03.820 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_11_[79604]: added 25 objects... 2024-06-11T13:31:03.820 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_11_[79604]: added half of the objects 2024-06-11T13:31:03.820 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_11_[79604]: added 50 objects... 2024-06-11T13:31:03.821 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_11_[79604]: added 50 objects 2024-06-11T13:31:03.821 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_11_[79604]: shutting down. 2024-06-11T13:31:03.821 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: ******************************* 2024-06-11T13:31:03.821 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: ******************************* 2024-06-11T13:31:03.821 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: ******************************* 2024-06-11T13:31:03.821 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_12_[79605]: starting. 2024-06-11T13:31:03.821 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_12_[79605]: added 25 objects... 2024-06-11T13:31:03.821 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_12_[79605]: added half of the objects 2024-06-11T13:31:03.821 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_12_[79605]: added 50 objects... 2024-06-11T13:31:03.821 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_12_[79605]: added 50 objects 2024-06-11T13:31:03.821 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_12_[79605]: shutting down. 2024-06-11T13:31:03.821 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: ******************************* 2024-06-11T13:31:03.821 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: ******************************* 2024-06-11T13:31:03.821 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: ******************************* 2024-06-11T13:31:03.822 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: ******************************* 2024-06-11T13:31:03.822 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_14_[79684]: starting. 2024-06-11T13:31:03.822 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_14_[79684]: creating pool ceph_test_rados_list_parallel.smithi057-78162 2024-06-11T13:31:03.822 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_14_[79684]: created object 0... 2024-06-11T13:31:03.822 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_14_[79684]: created object 25... 2024-06-11T13:31:03.822 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_14_[79684]: created object 49... 2024-06-11T13:31:03.822 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_14_[79684]: finishing. 2024-06-11T13:31:03.822 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_14_[79684]: shutting down. 2024-06-11T13:31:03.822 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: ******************************* 2024-06-11T13:31:03.822 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: ******************************* 2024-06-11T13:31:03.822 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: ******************************* 2024-06-11T13:31:03.822 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: ******************************* 2024-06-11T13:31:03.822 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_15_[79685]: starting. 2024-06-11T13:31:03.822 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_15_[79685]: listing objects. 2024-06-11T13:31:03.823 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_15_[79685]: listed object 0... 2024-06-11T13:31:04.033 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:31:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:31:04.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[40588]: pgmap v74: 1220 pgs: 605 creating+peering, 483 unknown, 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-06-11T13:31:04.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:04.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[40588]: Health check update: 26 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:31:04.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[40588]: from='client.34859 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi057-78504-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi057-78504-1"}]': finished 2024-06-11T13:31:04.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[40588]: from='client.24724 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:04.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[40588]: from='client.24733 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi057-77714-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:04.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[40588]: from='client.24739 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi057-77785-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:04.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[40588]: from='client.24754 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi057-77831-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:04.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[40588]: from='client.24763 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi057-77874-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:04.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[40588]: from='client.24787 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi057-78096-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:04.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[40588]: from='client.24802 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:04.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[40588]: from='client.24745 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi057-77799-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:04.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[40588]: from='client.24727 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi057-77599-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:04.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2574389455' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-06-11T13:31:04.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:04.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.0"}]: dispatch 2024-06-11T13:31:04.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.1"}]: dispatch 2024-06-11T13:31:04.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.2"}]: dispatch 2024-06-11T13:31:04.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.3"}]: dispatch 2024-06-11T13:31:04.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.4"}]: dispatch 2024-06-11T13:31:04.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[29805]: pgmap v74: 1220 pgs: 605 creating+peering, 483 unknown, 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-06-11T13:31:04.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:04.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[29805]: Health check update: 26 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:31:04.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[29805]: from='client.34859 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi057-78504-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi057-78504-1"}]': finished 2024-06-11T13:31:04.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[29805]: from='client.24724 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:04.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[29805]: from='client.24733 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi057-77714-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:04.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[29805]: from='client.24739 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi057-77785-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:04.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[29805]: from='client.24754 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi057-77831-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:04.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[29805]: from='client.24763 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi057-77874-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:04.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[29805]: from='client.24787 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi057-78096-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:04.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[29805]: from='client.24802 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:04.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[29805]: from='client.24745 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi057-77799-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:04.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[29805]: from='client.24727 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi057-77599-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:04.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2574389455' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-06-11T13:31:04.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:04.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.0"}]: dispatch 2024-06-11T13:31:04.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.1"}]: dispatch 2024-06-11T13:31:04.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.2"}]: dispatch 2024-06-11T13:31:04.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.3"}]: dispatch 2024-06-11T13:31:04.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.4"}]: dispatch 2024-06-11T13:31:04.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.5"}]: dispatch 2024-06-11T13:31:04.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.6"}]: dispatch 2024-06-11T13:31:04.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.7"}]: dispatch 2024-06-11T13:31:04.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.8"}]: dispatch 2024-06-11T13:31:04.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.9"}]: dispatch 2024-06-11T13:31:04.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:04.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[40588]: osdmap e77: 8 total, 8 up, 8 in 2024-06-11T13:31:04.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[40588]: from='client.24901 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi057-78571-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi057-78571-1"}]': finished 2024-06-11T13:31:04.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[40588]: from='client.34727 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi057-77882-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi057-77882-1"}]': finished 2024-06-11T13:31:04.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[40588]: osdmap e78: 8 total, 8 up, 8 in 2024-06-11T13:31:04.038 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.5"}]: dispatch 2024-06-11T13:31:04.038 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.6"}]: dispatch 2024-06-11T13:31:04.038 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.7"}]: dispatch 2024-06-11T13:31:04.038 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.8"}]: dispatch 2024-06-11T13:31:04.038 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.9"}]: dispatch 2024-06-11T13:31:04.038 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:04.039 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[29805]: osdmap e77: 8 total, 8 up, 8 in 2024-06-11T13:31:04.039 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[29805]: from='client.24901 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi057-78571-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi057-78571-1"}]': finished 2024-06-11T13:31:04.039 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[29805]: from='client.34727 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi057-77882-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi057-77882-1"}]': finished 2024-06-11T13:31:04.039 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:03 smithi057 ceph-mon[29805]: osdmap e78: 8 total, 8 up, 8 in 2024-06-11T13:31:04.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:03 smithi107 ceph-mon[33178]: pgmap v74: 1220 pgs: 605 creating+peering, 483 unknown, 132 active+clean; 583 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-06-11T13:31:04.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:03 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:04.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:03 smithi107 ceph-mon[33178]: Health check update: 26 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:31:04.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:03 smithi107 ceph-mon[33178]: from='client.34859 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi057-78504-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi057-78504-1"}]': finished 2024-06-11T13:31:04.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:03 smithi107 ceph-mon[33178]: from='client.24724 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:04.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:03 smithi107 ceph-mon[33178]: from='client.24733 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi057-77714-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:04.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:03 smithi107 ceph-mon[33178]: from='client.24739 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi057-77785-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:04.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:03 smithi107 ceph-mon[33178]: from='client.24754 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi057-77831-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:04.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:03 smithi107 ceph-mon[33178]: from='client.24763 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi057-77874-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:04.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:03 smithi107 ceph-mon[33178]: from='client.24787 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi057-78096-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:04.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:03 smithi107 ceph-mon[33178]: from='client.24802 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:04.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:03 smithi107 ceph-mon[33178]: from='client.24745 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi057-77799-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:04.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:03 smithi107 ceph-mon[33178]: from='client.24727 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi057-77599-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:04.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:03 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2574389455' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-06-11T13:31:04.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:03 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:04.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:03 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.0"}]: dispatch 2024-06-11T13:31:04.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:03 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.1"}]: dispatch 2024-06-11T13:31:04.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:03 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.2"}]: dispatch 2024-06-11T13:31:04.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:03 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.3"}]: dispatch 2024-06-11T13:31:04.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:03 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.4"}]: dispatch 2024-06-11T13:31:04.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:03 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.5"}]: dispatch 2024-06-11T13:31:04.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:03 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.6"}]: dispatch 2024-06-11T13:31:04.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:03 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.7"}]: dispatch 2024-06-11T13:31:04.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:03 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.8"}]: dispatch 2024-06-11T13:31:04.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:03 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.9"}]: dispatch 2024-06-11T13:31:04.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:03 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:04.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:03 smithi107 ceph-mon[33178]: osdmap e77: 8 total, 8 up, 8 in 2024-06-11T13:31:04.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:03 smithi107 ceph-mon[33178]: from='client.24901 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi057-78571-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi057-78571-1"}]': finished 2024-06-11T13:31:04.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:03 smithi107 ceph-mon[33178]: from='client.34727 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi057-77882-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi057-77882-1"}]': finished 2024-06-11T13:31:04.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:03 smithi107 ceph-mon[33178]: osdmap e78: 8 total, 8 up, 8 in 2024-06-11T13:31:05.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:05 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:31:05.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:05 smithi057 ceph-mon[29805]: pgmap v76: 1100 pgs: 205 creating+peering, 33 unknown, 862 active+clean; 590 KiB data, 252 MiB used, 715 GiB / 715 GiB avail; 9.3 KiB/s rd, 20 KiB/s wr, 212 op/s 2024-06-11T13:31:05.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:05 smithi057 ceph-mon[29805]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.0"}]: dispatch 2024-06-11T13:31:05.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:05 smithi057 ceph-mon[29805]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.1"}]: dispatch 2024-06-11T13:31:05.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:05 smithi057 ceph-mon[29805]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.2"}]: dispatch 2024-06-11T13:31:05.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:05 smithi057 ceph-mon[29805]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.3"}]: dispatch 2024-06-11T13:31:05.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:05 smithi057 ceph-mon[29805]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.4"}]: dispatch 2024-06-11T13:31:05.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:05 smithi057 ceph-mon[29805]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.5"}]: dispatch 2024-06-11T13:31:05.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:05 smithi057 ceph-mon[29805]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.6"}]: dispatch 2024-06-11T13:31:05.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:05 smithi057 ceph-mon[29805]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.7"}]: dispatch 2024-06-11T13:31:05.288 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:05 smithi057 ceph-mon[29805]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.8"}]: dispatch 2024-06-11T13:31:05.288 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:05 smithi057 ceph-mon[29805]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.9"}]: dispatch 2024-06-11T13:31:05.288 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:05 smithi057 ceph-mon[29805]: 19.7 deep-scrub starts 2024-06-11T13:31:05.288 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:05 smithi057 ceph-mon[29805]: 19.7 deep-scrub ok 2024-06-11T13:31:05.288 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:05 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:05.289 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:05 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:31:05.289 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:05 smithi057 ceph-mon[40588]: pgmap v76: 1100 pgs: 205 creating+peering, 33 unknown, 862 active+clean; 590 KiB data, 252 MiB used, 715 GiB / 715 GiB avail; 9.3 KiB/s rd, 20 KiB/s wr, 212 op/s 2024-06-11T13:31:05.289 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:05 smithi057 ceph-mon[40588]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.0"}]: dispatch 2024-06-11T13:31:05.289 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:05 smithi057 ceph-mon[40588]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.1"}]: dispatch 2024-06-11T13:31:05.289 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:05 smithi057 ceph-mon[40588]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.2"}]: dispatch 2024-06-11T13:31:05.289 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:05 smithi057 ceph-mon[40588]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.3"}]: dispatch 2024-06-11T13:31:05.289 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:05 smithi057 ceph-mon[40588]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.4"}]: dispatch 2024-06-11T13:31:05.290 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:05 smithi057 ceph-mon[40588]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.5"}]: dispatch 2024-06-11T13:31:05.290 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:05 smithi057 ceph-mon[40588]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.6"}]: dispatch 2024-06-11T13:31:05.290 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:05 smithi057 ceph-mon[40588]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.7"}]: dispatch 2024-06-11T13:31:05.290 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:05 smithi057 ceph-mon[40588]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.8"}]: dispatch 2024-06-11T13:31:05.290 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:05 smithi057 ceph-mon[40588]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.9"}]: dispatch 2024-06-11T13:31:05.290 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:05 smithi057 ceph-mon[40588]: 19.7 deep-scrub starts 2024-06-11T13:31:05.291 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:05 smithi057 ceph-mon[40588]: 19.7 deep-scrub ok 2024-06-11T13:31:05.291 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:05 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:05.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:05 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:31:05.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:05 smithi107 ceph-mon[33178]: pgmap v76: 1100 pgs: 205 creating+peering, 33 unknown, 862 active+clean; 590 KiB data, 252 MiB used, 715 GiB / 715 GiB avail; 9.3 KiB/s rd, 20 KiB/s wr, 212 op/s 2024-06-11T13:31:05.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:05 smithi107 ceph-mon[33178]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.0"}]: dispatch 2024-06-11T13:31:05.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:05 smithi107 ceph-mon[33178]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.1"}]: dispatch 2024-06-11T13:31:05.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:05 smithi107 ceph-mon[33178]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.2"}]: dispatch 2024-06-11T13:31:05.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:05 smithi107 ceph-mon[33178]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.3"}]: dispatch 2024-06-11T13:31:05.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:05 smithi107 ceph-mon[33178]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.4"}]: dispatch 2024-06-11T13:31:05.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:05 smithi107 ceph-mon[33178]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.5"}]: dispatch 2024-06-11T13:31:05.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:05 smithi107 ceph-mon[33178]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.6"}]: dispatch 2024-06-11T13:31:05.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:05 smithi107 ceph-mon[33178]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.7"}]: dispatch 2024-06-11T13:31:05.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:05 smithi107 ceph-mon[33178]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.8"}]: dispatch 2024-06-11T13:31:05.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:05 smithi107 ceph-mon[33178]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.9"}]: dispatch 2024-06-11T13:31:05.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:05 smithi107 ceph-mon[33178]: 19.7 deep-scrub starts 2024-06-11T13:31:05.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:05 smithi107 ceph-mon[33178]: 19.7 deep-scrub ok 2024-06-11T13:31:05.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:05 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:06.471 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_15_[79685]: listed object 25... 2024-06-11T13:31:06.471 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_15_[79685]: listed object 50... 2024-06-11T13:31:06.471 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_15_[79685]: listed object 75... 2024-06-11T13:31:06.471 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_15_[79685]: listed object 100... 2024-06-11T13:31:06.471 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_15_[79685]: listed object 125... 2024-06-11T13:31:06.472 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_15_[79685]: saw 150 objects 2024-06-11T13:31:06.472 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_15_[79685]: shutting down. 2024-06-11T13:31:06.472 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: ******************************* 2024-06-11T13:31:06.472 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: ******************************* 2024-06-11T13:31:06.472 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: ******************************* 2024-06-11T13:31:06.472 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: ******************************* 2024-06-11T13:31:06.472 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_16_[79686]: starting. 2024-06-11T13:31:06.472 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_16_[79686]: added 25 objects... 2024-06-11T13:31:06.472 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_16_[79686]: added half of the objects 2024-06-11T13:31:06.472 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_16_[79686]: added 50 objects... 2024-06-11T13:31:06.472 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_16_[79686]: added 50 objects 2024-06-11T13:31:06.472 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: process_16_[79686]: shutting down. 2024-06-11T13:31:06.472 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: ******************************* 2024-06-11T13:31:06.472 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: ******************************* 2024-06-11T13:31:06.473 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: ******************************* 2024-06-11T13:31:06.473 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: ******************************* 2024-06-11T13:31:06.473 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: ******************************* 2024-06-11T13:31:06.473 INFO:tasks.workunit.client.0.smithi057.stdout: list_parallel: ******* SUCCESS ********** 2024-06-11T13:31:06.499 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_read_operations: Running main() from gmock_main.cc 2024-06-11T13:31:06.499 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_read_operations: [==========] Running 17 tests from 1 test suite. 2024-06-11T13:31:06.499 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_read_operations: [----------] Global test environment set-up. 2024-06-11T13:31:06.499 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest 2024-06-11T13:31:06.499 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.NewDelete 2024-06-11T13:31:06.500 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_read_operations: [ OK ] CReadOpsTest.NewDelete (0 ms) 2024-06-11T13:31:06.500 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.SetOpFlags 2024-06-11T13:31:06.500 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_read_operations: [ OK ] CReadOpsTest.SetOpFlags (216 ms) 2024-06-11T13:31:06.500 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertExists 2024-06-11T13:31:06.500 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertExists (9 ms) 2024-06-11T13:31:06.500 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertVersion 2024-06-11T13:31:06.500 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertVersion (14 ms) 2024-06-11T13:31:06.500 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpXattr 2024-06-11T13:31:06.500 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpXattr (27 ms) 2024-06-11T13:31:06.500 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Read 2024-06-11T13:31:06.500 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Read (10 ms) 2024-06-11T13:31:06.500 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Checksum 2024-06-11T13:31:06.501 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Checksum (9 ms) 2024-06-11T13:31:06.501 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.RWOrderedRead 2024-06-11T13:31:06.501 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_read_operations: [ OK ] CReadOpsTest.RWOrderedRead (7 ms) 2024-06-11T13:31:06.501 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ShortRead 2024-06-11T13:31:06.501 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ShortRead (9 ms) 2024-06-11T13:31:06.501 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Exec 2024-06-11T13:31:06.501 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Exec (4 ms) 2024-06-11T13:31:06.501 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ExecUserBuf 2024-06-11T13:31:06.501 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ExecUserBuf (3 ms) 2024-06-11T13:31:06.501 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat 2024-06-11T13:31:06.501 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat (4 ms) 2024-06-11T13:31:06.501 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat2 2024-06-11T13:31:06.501 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat2 (4 ms) 2024-06-11T13:31:06.501 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Omap 2024-06-11T13:31:06.502 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Omap (10 ms) 2024-06-11T13:31:06.502 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.OmapNuls 2024-06-11T13:31:06.502 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_read_operations: [ OK ] CReadOpsTest.OmapNuls (8 ms) 2024-06-11T13:31:06.502 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.GetXattrs 2024-06-11T13:31:06.502 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_read_operations: [ OK ] CReadOpsTest.GetXattrs (6 ms) 2024-06-11T13:31:06.502 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpExt 2024-06-11T13:31:06.502 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpExt (7 ms) 2024-06-11T13:31:06.502 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest (347 ms total) 2024-06-11T13:31:06.502 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_read_operations: 2024-06-11T13:31:06.502 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_read_operations: [----------] Global test environment tear-down 2024-06-11T13:31:06.502 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_read_operations: [==========] 17 tests from 1 test suite ran. (22631 ms total) 2024-06-11T13:31:06.502 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_read_operations: [ PASSED ] 17 tests. 2024-06-11T13:31:06.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[29805]: 19.2 deep-scrub starts 2024-06-11T13:31:06.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[29805]: 19.2 deep-scrub ok 2024-06-11T13:31:06.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[29805]: 19.5 deep-scrub starts 2024-06-11T13:31:06.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[29805]: 19.5 deep-scrub ok 2024-06-11T13:31:06.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[29805]: 19.3 deep-scrub starts 2024-06-11T13:31:06.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[29805]: 19.3 deep-scrub ok 2024-06-11T13:31:06.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[29805]: 19.8 deep-scrub starts 2024-06-11T13:31:06.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[29805]: 19.8 deep-scrub ok 2024-06-11T13:31:06.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[29805]: 19.0 deep-scrub starts 2024-06-11T13:31:06.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[29805]: 19.0 deep-scrub ok 2024-06-11T13:31:06.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[29805]: 19.6 deep-scrub starts 2024-06-11T13:31:06.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[29805]: 19.6 deep-scrub ok 2024-06-11T13:31:06.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[29805]: 19.9 deep-scrub starts 2024-06-11T13:31:06.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[29805]: 19.9 deep-scrub ok 2024-06-11T13:31:06.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/4147419557' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi057-78504-1"}]: dispatch 2024-06-11T13:31:06.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[29805]: 19.1 deep-scrub starts 2024-06-11T13:31:06.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[29805]: 19.1 deep-scrub ok 2024-06-11T13:31:06.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[29805]: pgmap v79: 1204 pgs: 199 creating+peering, 175 unknown, 830 active+clean; 590 KiB data, 252 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 32 KiB/s wr, 333 op/s 2024-06-11T13:31:06.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:06.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[29805]: 19.4 deep-scrub starts 2024-06-11T13:31:06.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[29805]: 19.4 deep-scrub ok 2024-06-11T13:31:06.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[29805]: osdmap e79: 8 total, 8 up, 8 in 2024-06-11T13:31:06.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[29805]: from='client.34859 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi057-78504-1"}]: dispatch 2024-06-11T13:31:06.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[29805]: from='client.24820 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78142-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:06.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[29805]: from='client.24844 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77975-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:06.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[29805]: from='client.24838 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77928-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:06.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[29805]: osdmap e80: 8 total, 8 up, 8 in 2024-06-11T13:31:06.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/71948999' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi057-77362-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:06.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[29805]: from='client.35114 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi057-77362-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:06.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2422052846' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi057-77351-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:06.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[29805]: from='client.35111 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi057-77351-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:06.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:06.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:31:06.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:31:06.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:06.787 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[40588]: 19.2 deep-scrub starts 2024-06-11T13:31:06.787 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[40588]: 19.2 deep-scrub ok 2024-06-11T13:31:06.787 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[40588]: 19.5 deep-scrub starts 2024-06-11T13:31:06.788 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[40588]: 19.5 deep-scrub ok 2024-06-11T13:31:06.788 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[40588]: 19.3 deep-scrub starts 2024-06-11T13:31:06.788 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[40588]: 19.3 deep-scrub ok 2024-06-11T13:31:06.788 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[40588]: 19.8 deep-scrub starts 2024-06-11T13:31:06.788 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[40588]: 19.8 deep-scrub ok 2024-06-11T13:31:06.788 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[40588]: 19.0 deep-scrub starts 2024-06-11T13:31:06.788 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[40588]: 19.0 deep-scrub ok 2024-06-11T13:31:06.788 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[40588]: 19.6 deep-scrub starts 2024-06-11T13:31:06.788 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[40588]: 19.6 deep-scrub ok 2024-06-11T13:31:06.788 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[40588]: 19.9 deep-scrub starts 2024-06-11T13:31:06.788 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[40588]: 19.9 deep-scrub ok 2024-06-11T13:31:06.788 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/4147419557' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi057-78504-1"}]: dispatch 2024-06-11T13:31:06.788 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[40588]: 19.1 deep-scrub starts 2024-06-11T13:31:06.789 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[40588]: 19.1 deep-scrub ok 2024-06-11T13:31:06.789 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[40588]: pgmap v79: 1204 pgs: 199 creating+peering, 175 unknown, 830 active+clean; 590 KiB data, 252 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 32 KiB/s wr, 333 op/s 2024-06-11T13:31:06.789 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:06.789 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[40588]: 19.4 deep-scrub starts 2024-06-11T13:31:06.789 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[40588]: 19.4 deep-scrub ok 2024-06-11T13:31:06.789 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[40588]: osdmap e79: 8 total, 8 up, 8 in 2024-06-11T13:31:06.789 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[40588]: from='client.34859 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi057-78504-1"}]: dispatch 2024-06-11T13:31:06.789 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[40588]: from='client.24820 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78142-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:06.789 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[40588]: from='client.24844 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77975-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:06.789 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[40588]: from='client.24838 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77928-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:06.790 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[40588]: osdmap e80: 8 total, 8 up, 8 in 2024-06-11T13:31:06.790 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/71948999' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi057-77362-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:06.790 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[40588]: from='client.35114 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi057-77362-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:06.790 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2422052846' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi057-77351-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:06.790 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[40588]: from='client.35111 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi057-77351-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:06.790 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:06.790 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:31:06.790 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:31:06.790 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:06 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:06.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:06 smithi107 ceph-mon[33178]: 19.2 deep-scrub starts 2024-06-11T13:31:06.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:06 smithi107 ceph-mon[33178]: 19.2 deep-scrub ok 2024-06-11T13:31:06.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:06 smithi107 ceph-mon[33178]: 19.5 deep-scrub starts 2024-06-11T13:31:06.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:06 smithi107 ceph-mon[33178]: 19.5 deep-scrub ok 2024-06-11T13:31:06.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:06 smithi107 ceph-mon[33178]: 19.3 deep-scrub starts 2024-06-11T13:31:06.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:06 smithi107 ceph-mon[33178]: 19.3 deep-scrub ok 2024-06-11T13:31:06.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:06 smithi107 ceph-mon[33178]: 19.8 deep-scrub starts 2024-06-11T13:31:06.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:06 smithi107 ceph-mon[33178]: 19.8 deep-scrub ok 2024-06-11T13:31:06.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:06 smithi107 ceph-mon[33178]: 19.0 deep-scrub starts 2024-06-11T13:31:06.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:06 smithi107 ceph-mon[33178]: 19.0 deep-scrub ok 2024-06-11T13:31:06.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:06 smithi107 ceph-mon[33178]: 19.6 deep-scrub starts 2024-06-11T13:31:06.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:06 smithi107 ceph-mon[33178]: 19.6 deep-scrub ok 2024-06-11T13:31:06.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:06 smithi107 ceph-mon[33178]: 19.9 deep-scrub starts 2024-06-11T13:31:06.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:06 smithi107 ceph-mon[33178]: 19.9 deep-scrub ok 2024-06-11T13:31:06.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:06 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/4147419557' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi057-78504-1"}]: dispatch 2024-06-11T13:31:06.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:06 smithi107 ceph-mon[33178]: 19.1 deep-scrub starts 2024-06-11T13:31:06.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:06 smithi107 ceph-mon[33178]: 19.1 deep-scrub ok 2024-06-11T13:31:06.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:06 smithi107 ceph-mon[33178]: pgmap v79: 1204 pgs: 199 creating+peering, 175 unknown, 830 active+clean; 590 KiB data, 252 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 32 KiB/s wr, 333 op/s 2024-06-11T13:31:06.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:06 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:06.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:06 smithi107 ceph-mon[33178]: 19.4 deep-scrub starts 2024-06-11T13:31:06.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:06 smithi107 ceph-mon[33178]: 19.4 deep-scrub ok 2024-06-11T13:31:06.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:06 smithi107 ceph-mon[33178]: osdmap e79: 8 total, 8 up, 8 in 2024-06-11T13:31:06.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:06 smithi107 ceph-mon[33178]: from='client.34859 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi057-78504-1"}]: dispatch 2024-06-11T13:31:06.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:06 smithi107 ceph-mon[33178]: from='client.24820 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78142-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:06.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:06 smithi107 ceph-mon[33178]: from='client.24844 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77975-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:06.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:06 smithi107 ceph-mon[33178]: from='client.24838 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77928-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:06.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:06 smithi107 ceph-mon[33178]: osdmap e80: 8 total, 8 up, 8 in 2024-06-11T13:31:06.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:06 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/71948999' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi057-77362-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:06.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:06 smithi107 ceph-mon[33178]: from='client.35114 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi057-77362-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:06.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:06 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2422052846' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi057-77351-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:06.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:06 smithi107 ceph-mon[33178]: from='client.35111 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi057-77351-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:06.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:06 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:06.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:06 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:31:06.800 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:06 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:31:06.800 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:06 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:08.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:07 smithi057 ceph-mon[40588]: Health check update: 20 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:31:08.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:07 smithi057 ceph-mon[40588]: from='client.34859 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi057-78504-1"}]': finished 2024-06-11T13:31:08.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:07 smithi057 ceph-mon[40588]: from='client.24820 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78142-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:08.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:07 smithi057 ceph-mon[40588]: from='client.24844 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77975-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:08.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:07 smithi057 ceph-mon[40588]: from='client.24838 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77928-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:08.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:07 smithi057 ceph-mon[40588]: from='client.35114 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi057-77362-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:08.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:07 smithi057 ceph-mon[40588]: from='client.35111 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi057-77351-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:08.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:07 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/4147419557' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi057-78504-1"}]: dispatch 2024-06-11T13:31:08.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:07 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2245769429' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:08.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:07 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/232412548' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi057-77506-10"}]: dispatch 2024-06-11T13:31:08.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:07 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2607108900' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi057-77831-7"}]: dispatch 2024-06-11T13:31:08.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:07 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3748941041' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi057-77494-10"}]: dispatch 2024-06-11T13:31:08.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:07 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3479760694' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi057-77785-7"}]: dispatch 2024-06-11T13:31:08.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:07 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1129714156' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi057-77396-22"}]: dispatch 2024-06-11T13:31:08.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:07 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1420997228' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi057-77374-16"}]: dispatch 2024-06-11T13:31:08.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:07 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:08.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:07 smithi057 ceph-mon[40588]: osdmap e81: 8 total, 8 up, 8 in 2024-06-11T13:31:08.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:07 smithi057 ceph-mon[40588]: from='client.34859 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi057-78504-1"}]: dispatch 2024-06-11T13:31:08.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:07 smithi057 ceph-mon[29805]: Health check update: 20 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:31:08.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:07 smithi057 ceph-mon[29805]: from='client.34859 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi057-78504-1"}]': finished 2024-06-11T13:31:08.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:07 smithi057 ceph-mon[29805]: from='client.24820 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78142-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:08.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:07 smithi057 ceph-mon[29805]: from='client.24844 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77975-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:08.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:07 smithi057 ceph-mon[29805]: from='client.24838 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77928-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:08.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:07 smithi057 ceph-mon[29805]: from='client.35114 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi057-77362-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:08.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:07 smithi057 ceph-mon[29805]: from='client.35111 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi057-77351-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:08.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:07 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/4147419557' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi057-78504-1"}]: dispatch 2024-06-11T13:31:08.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:07 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2245769429' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:08.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:07 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/232412548' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi057-77506-10"}]: dispatch 2024-06-11T13:31:08.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:07 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2607108900' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi057-77831-7"}]: dispatch 2024-06-11T13:31:08.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:07 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3748941041' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi057-77494-10"}]: dispatch 2024-06-11T13:31:08.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:07 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3479760694' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi057-77785-7"}]: dispatch 2024-06-11T13:31:08.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:07 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1129714156' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi057-77396-22"}]: dispatch 2024-06-11T13:31:08.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:07 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1420997228' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi057-77374-16"}]: dispatch 2024-06-11T13:31:08.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:07 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:08.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:07 smithi057 ceph-mon[29805]: osdmap e81: 8 total, 8 up, 8 in 2024-06-11T13:31:08.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:07 smithi057 ceph-mon[29805]: from='client.34859 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi057-78504-1"}]: dispatch 2024-06-11T13:31:08.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:07 smithi107 ceph-mon[33178]: Health check update: 20 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:31:08.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:07 smithi107 ceph-mon[33178]: from='client.34859 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi057-78504-1"}]': finished 2024-06-11T13:31:08.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:07 smithi107 ceph-mon[33178]: from='client.24820 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78142-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:08.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:07 smithi107 ceph-mon[33178]: from='client.24844 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77975-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:08.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:07 smithi107 ceph-mon[33178]: from='client.24838 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77928-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:08.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:07 smithi107 ceph-mon[33178]: from='client.35114 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi057-77362-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:08.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:07 smithi107 ceph-mon[33178]: from='client.35111 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi057-77351-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:08.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:07 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/4147419557' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi057-78504-1"}]: dispatch 2024-06-11T13:31:08.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:07 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2245769429' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:08.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:07 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/232412548' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi057-77506-10"}]: dispatch 2024-06-11T13:31:08.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:07 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2607108900' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi057-77831-7"}]: dispatch 2024-06-11T13:31:08.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:07 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3748941041' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi057-77494-10"}]: dispatch 2024-06-11T13:31:08.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:07 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3479760694' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi057-77785-7"}]: dispatch 2024-06-11T13:31:08.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:07 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1129714156' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi057-77396-22"}]: dispatch 2024-06-11T13:31:08.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:07 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1420997228' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi057-77374-16"}]: dispatch 2024-06-11T13:31:08.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:07 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:08.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:07 smithi107 ceph-mon[33178]: osdmap e81: 8 total, 8 up, 8 in 2024-06-11T13:31:08.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:07 smithi107 ceph-mon[33178]: from='client.34859 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi057-78504-1"}]: dispatch 2024-06-11T13:31:09.533 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:09 smithi057 ceph-mon[40588]: from='client.25108 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:09.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:09 smithi057 ceph-mon[40588]: from='client.35297 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi057-77494-10"}]: dispatch 2024-06-11T13:31:09.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:09 smithi057 ceph-mon[40588]: from='client.35306 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi057-77396-22"}]: dispatch 2024-06-11T13:31:09.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:09 smithi057 ceph-mon[40588]: from='client.34859 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi057-78504-1"}]': finished 2024-06-11T13:31:09.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:09 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/71948999' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi057-77362-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:09.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:09 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2571911560' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi057-78504-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:09.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:09 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:09.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:09 smithi057 ceph-mon[40588]: osdmap e82: 8 total, 8 up, 8 in 2024-06-11T13:31:09.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:09 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3748941041' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi057-77494-10"}]: dispatch 2024-06-11T13:31:09.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:09 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1129714156' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi057-77396-22"}]: dispatch 2024-06-11T13:31:09.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:09 smithi057 ceph-mon[40588]: from='client.35312 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi057-77374-16"}]: dispatch 2024-06-11T13:31:09.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:09 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1420997228' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi057-77374-16"}]: dispatch 2024-06-11T13:31:09.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:09 smithi057 ceph-mon[40588]: from='client.25279 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi057-77506-10"}]: dispatch 2024-06-11T13:31:09.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:09 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/232412548' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi057-77506-10"}]: dispatch 2024-06-11T13:31:09.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:09 smithi057 ceph-mon[29805]: from='client.25108 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:09.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:09 smithi057 ceph-mon[29805]: from='client.35297 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi057-77494-10"}]: dispatch 2024-06-11T13:31:09.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:09 smithi057 ceph-mon[29805]: from='client.35306 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi057-77396-22"}]: dispatch 2024-06-11T13:31:09.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:09 smithi057 ceph-mon[29805]: from='client.34859 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi057-78504-1"}]': finished 2024-06-11T13:31:09.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:09 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/71948999' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi057-77362-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:09.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:09 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2571911560' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi057-78504-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:09.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:09 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:09.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:09 smithi057 ceph-mon[29805]: osdmap e82: 8 total, 8 up, 8 in 2024-06-11T13:31:09.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:09 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3748941041' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi057-77494-10"}]: dispatch 2024-06-11T13:31:09.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:09 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1129714156' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi057-77396-22"}]: dispatch 2024-06-11T13:31:09.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:09 smithi057 ceph-mon[29805]: from='client.35312 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi057-77374-16"}]: dispatch 2024-06-11T13:31:09.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:09 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1420997228' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi057-77374-16"}]: dispatch 2024-06-11T13:31:09.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:09 smithi057 ceph-mon[29805]: from='client.25279 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi057-77506-10"}]: dispatch 2024-06-11T13:31:09.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:09 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/232412548' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi057-77506-10"}]: dispatch 2024-06-11T13:31:09.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:09 smithi107 ceph-mon[33178]: from='client.25108 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:09.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:09 smithi107 ceph-mon[33178]: from='client.35297 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi057-77494-10"}]: dispatch 2024-06-11T13:31:09.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:09 smithi107 ceph-mon[33178]: from='client.35306 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi057-77396-22"}]: dispatch 2024-06-11T13:31:09.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:09 smithi107 ceph-mon[33178]: from='client.34859 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi057-78504-1"}]': finished 2024-06-11T13:31:09.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:09 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/71948999' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi057-77362-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:09.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:09 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2571911560' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi057-78504-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:09.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:09 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:09.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:09 smithi107 ceph-mon[33178]: osdmap e82: 8 total, 8 up, 8 in 2024-06-11T13:31:09.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:09 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3748941041' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi057-77494-10"}]: dispatch 2024-06-11T13:31:09.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:09 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1129714156' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi057-77396-22"}]: dispatch 2024-06-11T13:31:09.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:09 smithi107 ceph-mon[33178]: from='client.35312 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi057-77374-16"}]: dispatch 2024-06-11T13:31:09.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:09 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1420997228' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi057-77374-16"}]: dispatch 2024-06-11T13:31:09.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:09 smithi107 ceph-mon[33178]: from='client.25279 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi057-77506-10"}]: dispatch 2024-06-11T13:31:09.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:09 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/232412548' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi057-77506-10"}]: dispatch 2024-06-11T13:31:10.667 INFO:tasks.workunit.client.0.smithi057.stdout: api_cls_remote_reads: [==========] Running 1 test from 1 test suite. 2024-06-11T13:31:10.668 INFO:tasks.workunit.client.0.smithi057.stdout: api_cls_remote_reads: [----------] Global test environment set-up. 2024-06-11T13:31:10.668 INFO:tasks.workunit.client.0.smithi057.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads 2024-06-11T13:31:10.668 INFO:tasks.workunit.client.0.smithi057.stdout: api_cls_remote_reads: [ RUN ] ClsTestRemoteReads.TestGather 2024-06-11T13:31:10.668 INFO:tasks.workunit.client.0.smithi057.stdout: api_cls_remote_reads: [ OK ] ClsTestRemoteReads.TestGather (26756 ms) 2024-06-11T13:31:10.668 INFO:tasks.workunit.client.0.smithi057.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads (26756 ms total) 2024-06-11T13:31:10.668 INFO:tasks.workunit.client.0.smithi057.stdout: api_cls_remote_reads: 2024-06-11T13:31:10.668 INFO:tasks.workunit.client.0.smithi057.stdout: api_cls_remote_reads: [----------] Global test environment tear-down 2024-06-11T13:31:10.668 INFO:tasks.workunit.client.0.smithi057.stdout: api_cls_remote_reads: [==========] 1 test from 1 test suite ran. (26756 ms total) 2024-06-11T13:31:10.668 INFO:tasks.workunit.client.0.smithi057.stdout: api_cls_remote_reads: [ PASSED ] 1 test. 2024-06-11T13:31:10.668 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd_pp: Running main() from gmock_main.cc 2024-06-11T13:31:10.668 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd_pp: [==========] Running 3 tests from 1 test suite. 2024-06-11T13:31:10.668 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd_pp: [----------] Global test environment set-up. 2024-06-11T13:31:10.669 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd 2024-06-11T13:31:10.669 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.MonDescribePP 2024-06-11T13:31:10.669 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd_pp: [ OK ] LibRadosCmd.MonDescribePP (59 ms) 2024-06-11T13:31:10.669 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.OSDCmdPP 2024-06-11T13:31:10.669 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd_pp: [ OK ] LibRadosCmd.OSDCmdPP (55 ms) 2024-06-11T13:31:10.669 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.PGCmdPP 2024-06-11T13:31:10.669 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd_pp: [ OK ] LibRadosCmd.PGCmdPP (26728 ms) 2024-06-11T13:31:10.669 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd (26842 ms total) 2024-06-11T13:31:10.669 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd_pp: 2024-06-11T13:31:10.669 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd_pp: [----------] Global test environment tear-down 2024-06-11T13:31:10.669 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd_pp: [==========] 3 tests from 1 test suite ran. (26845 ms total) 2024-06-11T13:31:10.669 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd_pp: [ PASSED ] 3 tests. 2024-06-11T13:31:11.033 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:10 smithi057 ceph-mon[40588]: pgmap v81: 1204 pgs: 220 creating+peering, 984 active+clean; 145 MiB data, 717 MiB used, 715 GiB / 715 GiB avail; 9.1 MiB/s rd, 28 MiB/s wr, 590 op/s 2024-06-11T13:31:11.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:10 smithi057 ceph-mon[40588]: pgmap v84: 948 pgs: 192 unknown, 220 creating+peering, 536 active+clean; 145 MiB data, 717 MiB used, 715 GiB / 715 GiB avail; 11 MiB/s rd, 35 MiB/s wr, 177 op/s 2024-06-11T13:31:11.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:10 smithi057 ceph-mon[40588]: from='client.25297 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi057-77831-7"}]: dispatch 2024-06-11T13:31:11.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:10 smithi057 ceph-mon[40588]: from='client.35114 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi057-77362-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:11.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:10 smithi057 ceph-mon[40588]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi057-78504-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:11.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:10 smithi057 ceph-mon[40588]: from='client.35297 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi057-77494-10"}]: dispatch 2024-06-11T13:31:11.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:10 smithi057 ceph-mon[40588]: from='client.35306 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi057-77396-22"}]: dispatch 2024-06-11T13:31:11.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:10 smithi057 ceph-mon[40588]: from='client.35312 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi057-77374-16"}]: dispatch 2024-06-11T13:31:11.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:10 smithi057 ceph-mon[40588]: from='client.25303 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi057-77785-7"}]: dispatch 2024-06-11T13:31:11.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:10 smithi057 ceph-mon[29805]: pgmap v81: 1204 pgs: 220 creating+peering, 984 active+clean; 145 MiB data, 717 MiB used, 715 GiB / 715 GiB avail; 9.1 MiB/s rd, 28 MiB/s wr, 590 op/s 2024-06-11T13:31:11.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:10 smithi057 ceph-mon[29805]: pgmap v84: 948 pgs: 192 unknown, 220 creating+peering, 536 active+clean; 145 MiB data, 717 MiB used, 715 GiB / 715 GiB avail; 11 MiB/s rd, 35 MiB/s wr, 177 op/s 2024-06-11T13:31:11.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:10 smithi057 ceph-mon[29805]: from='client.25297 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi057-77831-7"}]: dispatch 2024-06-11T13:31:11.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:10 smithi057 ceph-mon[29805]: from='client.35114 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi057-77362-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:11.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:10 smithi057 ceph-mon[29805]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi057-78504-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:11.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:10 smithi057 ceph-mon[29805]: from='client.35297 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi057-77494-10"}]: dispatch 2024-06-11T13:31:11.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:10 smithi057 ceph-mon[29805]: from='client.35306 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi057-77396-22"}]: dispatch 2024-06-11T13:31:11.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:10 smithi057 ceph-mon[29805]: from='client.35312 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi057-77374-16"}]: dispatch 2024-06-11T13:31:11.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:10 smithi057 ceph-mon[29805]: from='client.25303 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi057-77785-7"}]: dispatch 2024-06-11T13:31:11.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:10 smithi107 ceph-mon[33178]: pgmap v81: 1204 pgs: 220 creating+peering, 984 active+clean; 145 MiB data, 717 MiB used, 715 GiB / 715 GiB avail; 9.1 MiB/s rd, 28 MiB/s wr, 590 op/s 2024-06-11T13:31:11.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:10 smithi107 ceph-mon[33178]: pgmap v84: 948 pgs: 192 unknown, 220 creating+peering, 536 active+clean; 145 MiB data, 717 MiB used, 715 GiB / 715 GiB avail; 11 MiB/s rd, 35 MiB/s wr, 177 op/s 2024-06-11T13:31:11.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:10 smithi107 ceph-mon[33178]: from='client.25297 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi057-77831-7"}]: dispatch 2024-06-11T13:31:11.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:10 smithi107 ceph-mon[33178]: from='client.35114 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi057-77362-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:11.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:10 smithi107 ceph-mon[33178]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi057-78504-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:11.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:10 smithi107 ceph-mon[33178]: from='client.35297 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi057-77494-10"}]: dispatch 2024-06-11T13:31:11.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:10 smithi107 ceph-mon[33178]: from='client.35306 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi057-77396-22"}]: dispatch 2024-06-11T13:31:11.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:10 smithi107 ceph-mon[33178]: from='client.35312 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi057-77374-16"}]: dispatch 2024-06-11T13:31:11.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:10 smithi107 ceph-mon[33178]: from='client.25303 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi057-77785-7"}]: dispatch 2024-06-11T13:31:11.547 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:31:11 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:31:12.053 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: Running main() from gmock_main.cc 2024-06-11T13:31:12.053 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: [==========] Running 4 tests from 1 test suite. 2024-06-11T13:31:12.053 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: [----------] Global test environment set-up. 2024-06-11T13:31:12.053 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: [----------] 4 tests from LibRadosCmd 2024-06-11T13:31:12.053 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: [ RUN ] LibRadosCmd.MonDescribe 2024-06-11T13:31:12.053 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: [ OK ] LibRadosCmd.MonDescribe (54 ms) 2024-06-11T13:31:12.054 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: [ RUN ] LibRadosCmd.OSDCmd 2024-06-11T13:31:12.054 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: [ OK ] LibRadosCmd.OSDCmd (66 ms) 2024-06-11T13:31:12.054 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: [ RUN ] LibRadosCmd.PGCmd 2024-06-11T13:31:12.054 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: [ OK ] LibRadosCmd.PGCmd (26745 ms) 2024-06-11T13:31:12.054 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: [ RUN ] LibRadosCmd.WatchLog 2024-06-11T13:31:12.054 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:09.269838+0000 mon.a [INF] from='client.25297 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi057-77831-7"}]: dispatch 2024-06-11T13:31:12.054 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:09.270430+0000 mon.a [INF] from='client.35114 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi057-77362-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:12.054 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:09.270685+0000 mon.a [INF] from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi057-78504-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:12.054 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:09.283106+0000 mon.a [INF] from='client.35297 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi057-77494-10"}]: dispatch 2024-06-11T13:31:12.054 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:09.283515+0000 mon.a [INF] from='client.35306 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi057-77396-22"}]: dispatch 2024-06-11T13:31:12.054 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:09.283782+0000 mon.a [INF] from='client.35312 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi057-77374-16"}]: dispatch 2024-06-11T13:31:12.054 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:09.284132+0000 mon.a [INF] from='client.25303 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi057-77785-7"}]: dispatch 2024-06-11T13:31:12.055 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:09.288443+0000 mon.a [INF] from='client.25108 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:12.055 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:09.293746+0000 mon.b [INF] from='client.? v1:172.21.15.57:0/2870457208' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:12.055 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:09.749432+0000 mon.b [INF] from='client.? v1:172.21.15.57:0/2607108900' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi057-77831-7"}]: dispatch 2024-06-11T13:31:12.055 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:10.193443+0000 mon.c [INF] from='client.? v1:172.21.15.57:0/3748941041' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi057-77494-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:12.055 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:10.193680+0000 mon.c [INF] from='client.? v1:172.21.15.57:0/1129714156' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi057-77396-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:12.055 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:10.193846+0000 mon.c [INF] from='client.? v1:172.21.15.57:0/1420997228' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi057-77374-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:12.055 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:10.194049+0000 mon.b [INF] from='client.? v1:172.21.15.57:0/3479760694' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi057-77785-7"}]: dispatch 2024-06-11T13:31:12.055 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:10.647364+0000 mon.a [INF] from='client.25279 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi057-77506-10"}]: dispatch 2024-06-11T13:31:12.055 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:10.647544+0000 mon.a [INF] from='client.35297 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi057-77494-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:12.055 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:10.647840+0000 mon.a [INF] from='client.35306 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi057-77396-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:12.055 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:10.648070+0000 mon.a [INF] from='client.35312 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi057-77374-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:12.055 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:10.648314+0000 mon.a [INF] from='client.25285 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:12.056 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:10.654838+0000 mon.a [INF] from='client.35114 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi057-77362-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:12.056 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:10.654909+0000 mon.a [INF] from='client.35318 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi057-78504-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:12.056 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:10.663969+0000 mon.c [INF] from='client.? v1:172.21.15.57:0/2571911560' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi057-78504-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi057-78504-2"}]: dispatch 2024-06-11T13:31:12.056 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:10.665924+0000 mon.b [INF] from='client.? v1:172.21.15.57:0/410451071' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:12.056 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:10.667036+0000 mon.b [INF] from='client.? v1:172.21.15.57:0/2305832357' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi057-78571-1"}]: dispatch 2024-06-11T13:31:12.056 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:10.668356+0000 mon.c [INF] from='client.? v1:172.21.15.57:0/71948999' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi057-77362-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-06-11T13:31:12.056 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:10.668706+0000 mon.c [INF] from='client.? v1:172.21.15.57:0/289483390' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi057-77882-1"}]: dispatch 2024-06-11T13:31:12.056 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:10.733887+0000 mon.c [INF] from='client.? v1:172.21.15.57:0/2747983990' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-06-11T13:31:12.056 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:11.148935+0000 mon.b [INF] from='client.? v1:172.21.15.57:0/232412548' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi057-77506-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:12.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:12 smithi107 ceph-mon[33178]: from='client.25108 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:12.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:12 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2870457208' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:12.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:12 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:12.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:12 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2607108900' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi057-77831-7"}]: dispatch 2024-06-11T13:31:12.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:12 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3748941041' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi057-77494-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:12.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:12 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1129714156' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi057-77396-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:12.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:12 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1420997228' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi057-77374-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:12.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:12 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3479760694' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi057-77785-7"}]: dispatch 2024-06-11T13:31:12.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:12 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:12.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:12 smithi107 ceph-mon[33178]: osdmap e83: 8 total, 8 up, 8 in 2024-06-11T13:31:12.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:12 smithi107 ceph-mon[33178]: from='client.25279 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi057-77506-10"}]: dispatch 2024-06-11T13:31:12.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:12 smithi107 ceph-mon[33178]: from='client.35297 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi057-77494-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:12.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:12 smithi107 ceph-mon[33178]: from='client.35306 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi057-77396-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:12.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:12 smithi107 ceph-mon[33178]: from='client.35312 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi057-77374-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:12.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:12 smithi107 ceph-mon[33178]: from='client.25285 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:12.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:12 smithi107 ceph-mon[33178]: from='client.35114 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi057-77362-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:12.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:12 smithi107 ceph-mon[33178]: from='client.35318 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi057-78504-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:12.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:12 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2571911560' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi057-78504-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi057-78504-2"}]: dispatch 2024-06-11T13:31:12.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:12 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/410451071' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:12.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:12 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2305832357' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi057-78571-1"}]: dispatch 2024-06-11T13:31:12.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:12 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/71948999' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi057-77362-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-06-11T13:31:12.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:12 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/289483390' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi057-77882-1"}]: dispatch 2024-06-11T13:31:12.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:12 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2747983990' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-06-11T13:31:12.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:12 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/232412548' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi057-77506-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:12.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:12 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:12.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:12 smithi107 ceph-mon[33178]: osdmap e84: 8 total, 8 up, 8 in 2024-06-11T13:31:12.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:12 smithi107 ceph-mon[33178]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi057-78504-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi057-78504-2"}]: dispatch 2024-06-11T13:31:12.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:12 smithi107 ceph-mon[33178]: from='client.35114 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi057-77362-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-06-11T13:31:12.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:12 smithi107 ceph-mon[33178]: from='client.34727 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi057-77882-1"}]: dispatch 2024-06-11T13:31:12.512 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[29805]: from='client.25108 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:12.512 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2870457208' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:12.513 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:12.513 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2607108900' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi057-77831-7"}]: dispatch 2024-06-11T13:31:12.513 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3748941041' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi057-77494-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:12.513 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1129714156' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi057-77396-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:12.513 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1420997228' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi057-77374-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:12.513 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3479760694' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi057-77785-7"}]: dispatch 2024-06-11T13:31:12.513 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:12.514 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[29805]: osdmap e83: 8 total, 8 up, 8 in 2024-06-11T13:31:12.514 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[29805]: from='client.25279 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi057-77506-10"}]: dispatch 2024-06-11T13:31:12.514 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[29805]: from='client.35297 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi057-77494-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:12.514 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[29805]: from='client.35306 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi057-77396-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:12.514 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[29805]: from='client.35312 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi057-77374-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:12.514 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[29805]: from='client.25285 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:12.514 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[29805]: from='client.35114 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi057-77362-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:12.514 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[29805]: from='client.35318 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi057-78504-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:12.515 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[40588]: from='client.25108 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:12.515 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2870457208' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:12.515 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:12.515 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2607108900' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi057-77831-7"}]: dispatch 2024-06-11T13:31:12.515 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3748941041' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi057-77494-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:12.516 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1129714156' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi057-77396-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:12.516 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1420997228' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi057-77374-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:12.516 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3479760694' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi057-77785-7"}]: dispatch 2024-06-11T13:31:12.516 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:12.516 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[40588]: osdmap e83: 8 total, 8 up, 8 in 2024-06-11T13:31:12.516 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[40588]: from='client.25279 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi057-77506-10"}]: dispatch 2024-06-11T13:31:12.516 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[40588]: from='client.35297 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi057-77494-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:12.516 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[40588]: from='client.35306 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi057-77396-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:12.516 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[40588]: from='client.35312 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi057-77374-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:12.516 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[40588]: from='client.25285 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:12.517 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[40588]: from='client.35114 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi057-77362-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:12.517 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[40588]: from='client.35318 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi057-78504-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:12.517 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2571911560' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi057-78504-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi057-78504-2"}]: dispatch 2024-06-11T13:31:12.517 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/410451071' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:12.517 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2305832357' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi057-78571-1"}]: dispatch 2024-06-11T13:31:12.517 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/71948999' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi057-77362-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-06-11T13:31:12.517 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/289483390' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi057-77882-1"}]: dispatch 2024-06-11T13:31:12.517 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2747983990' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-06-11T13:31:12.518 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/232412548' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi057-77506-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:12.518 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:12.518 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[29805]: osdmap e84: 8 total, 8 up, 8 in 2024-06-11T13:31:12.518 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[29805]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi057-78504-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi057-78504-2"}]: dispatch 2024-06-11T13:31:12.518 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[29805]: from='client.35114 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi057-77362-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-06-11T13:31:12.518 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[29805]: from='client.34727 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi057-77882-1"}]: dispatch 2024-06-11T13:31:12.518 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2571911560' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi057-78504-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi057-78504-2"}]: dispatch 2024-06-11T13:31:12.518 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/410451071' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:12.518 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2305832357' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi057-78571-1"}]: dispatch 2024-06-11T13:31:12.518 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/71948999' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi057-77362-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-06-11T13:31:12.519 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/289483390' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi057-77882-1"}]: dispatch 2024-06-11T13:31:12.519 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2747983990' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-06-11T13:31:12.519 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/232412548' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi057-77506-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:12.519 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:12.519 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[40588]: osdmap e84: 8 total, 8 up, 8 in 2024-06-11T13:31:12.519 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[40588]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi057-78504-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi057-78504-2"}]: dispatch 2024-06-11T13:31:12.519 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[40588]: from='client.35114 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi057-77362-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-06-11T13:31:12.519 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:12 smithi057 ceph-mon[40588]: from='client.34727 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi057-77882-1"}]: dispatch 2024-06-11T13:31:13.452 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:11.587136+0000 mon.a [INF] from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi057-78504-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi057-78504-2"}]: dispatch 2024-06-11T13:31:13.452 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:11.587663+0000 mon.a [INF] from='client.35114 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi057-77362-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-06-11T13:31:13.452 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:11.587972+0000 mon.a [INF] from='client.34727 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi057-77882-1"}]: dispatch 2024-06-11T13:31:13.452 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:10.734314+0000 client.admin [INF] onexx 2024-06-11T13:31:13.452 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:12.037367+0000 mon.a [WRN] Health check update: 18 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:31:13.452 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:12.040768+0000 mon.a [INF] from='client.35348 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-06-11T13:31:13.452 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:12.046008+0000 mon.a [INF] from='client.25297 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi057-77831-7"}]: dispatch 2024-06-11T13:31:13.452 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:12.046248+0000 mon.a [INF] from='client.35297 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi057-77494-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:13.453 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:12.046339+0000 mon.a [INF] from='client.35306 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi057-77396-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:13.453 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:12.046413+0000 mon.a [INF] from='client.35312 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi057-77374-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:13.453 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:12.046502+0000 mon.a [INF] from='client.25285 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:13.453 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:12.046574+0000 mon.a [INF] from='client.35114 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi057-77362-3", "field": "max_bytes", "val": "4096"}]': finished 2024-06-11T13:31:13.453 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:12.046641+0000 mon.a [INF] from='client.34727 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi057-77882-1"}]': finished 2024-06-11T13:31:13.453 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:12.054976+0000 mon.c [INF] from='client.? v1:172.21.15.57:0/1129714156' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi057-77396-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi057-77396-22"}]: dispatch 2024-06-11T13:31:13.453 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:12.055145+0000 mon.c [INF] from='client.? v1:172.21.15.57:0/3748941041' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi057-77494-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi057-77494-10"}]: dispatch 2024-06-11T13:31:13.453 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:12.058022+0000 mon.b [INF] from='client.? v1:172.21.15.57:0/2245769429' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:13.488 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:12.058235+0000 mo watch_notify: Running main() from gmock_main.cc 2024-06-11T13:31:13.488 INFO:tasks.workunit.client.0.smithi057.stdout: watch_notify: [==========] Running 2 tests from 1 test suite. 2024-06-11T13:31:13.489 INFO:tasks.workunit.client.0.smithi057.stdout: watch_notify: [----------] Global test environment set-up. 2024-06-11T13:31:13.489 INFO:tasks.workunit.client.0.smithi057.stdout: watch_notify: [----------] 2 tests from NeoRadosWatchNotify 2024-06-11T13:31:13.489 INFO:tasks.workunit.client.0.smithi057.stdout: watch_notify: [ RUN ] NeoRadosWatchNotify.WatchNotify 2024-06-11T13:31:13.489 INFO:tasks.workunit.client.0.smithi057.stdout: watch_notify: handle_notify cookie 93857258112736 notify_id 326417514496 notifier_gid 34898 2024-06-11T13:31:13.489 INFO:tasks.workunit.client.0.smithi057.stdout: watch_notify: [ OK ] NeoRadosWatchNotify.WatchNotify (17603 ms) 2024-06-11T13:31:13.489 INFO:tasks.workunit.client.0.smithi057.stdout: watch_notify: [ RUN ] NeoRadosWatchNotify.WatchNotifyTimeout 2024-06-11T13:31:13.489 INFO:tasks.workunit.client.0.smithi057.stdout: watch_notify: Trying... 2024-06-11T13:31:13.489 INFO:tasks.workunit.client.0.smithi057.stdout: watch_notify: handle_notify cookie 93857258390160 notify_id 347892350977 notifier_gid 35099 2024-06-11T13:31:13.489 INFO:tasks.workunit.client.0.smithi057.stdout: watch_notify: Waiting for 3.000000000s 2024-06-11T13:31:13.489 INFO:tasks.workunit.client.0.smithi057.stdout: watch_notify: Timed out. 2024-06-11T13:31:13.489 INFO:tasks.workunit.client.0.smithi057.stdout: watch_notify: Flushing... 2024-06-11T13:31:13.489 INFO:tasks.workunit.client.0.smithi057.stdout: watch_notify: Flushed... 2024-06-11T13:31:13.490 INFO:tasks.workunit.client.0.smithi057.stdout: watch_notify: [ OK ] NeoRadosWatchNotify.WatchNotifyTimeout (11775 ms) 2024-06-11T13:31:13.490 INFO:tasks.workunit.client.0.smithi057.stdout: watch_notify: [----------] 2 tests from NeoRadosWatchNotify (29378 ms total) 2024-06-11T13:31:13.490 INFO:tasks.workunit.client.0.smithi057.stdout: watch_notify: 2024-06-11T13:31:13.490 INFO:tasks.workunit.client.0.smithi057.stdout: watch_notify: [----------] Global test environment tear-down 2024-06-11T13:31:13.490 INFO:tasks.workunit.client.0.smithi057.stdout: watch_notify: [==========] 2 tests from 1 test suite ran. (29378 ms total) 2024-06-11T13:31:13.490 INFO:tasks.workunit.client.0.smithi057.stdout: watch_notify: [ PASSED ] 2 tests. 2024-06-11T13:31:13.733 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[29805]: pgmap v86: 1012 pgs: 288 unknown, 188 creating+peering, 536 active+clean; 145 MiB data, 717 MiB used, 715 GiB / 715 GiB avail; 9.7 MiB/s rd, 30 MiB/s wr, 154 op/s 2024-06-11T13:31:13.733 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[29805]: onexx 2024-06-11T13:31:13.733 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:31:13.733 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:31:13.733 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[29805]: Health check update: 18 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:31:13.733 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[29805]: from='client.35348 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-06-11T13:31:13.733 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[29805]: from='client.25297 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi057-77831-7"}]: dispatch 2024-06-11T13:31:13.733 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[29805]: from='client.35297 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi057-77494-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:13.733 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[29805]: from='client.35306 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi057-77396-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:13.733 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[29805]: from='client.35312 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi057-77374-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:13.734 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[29805]: from='client.25285 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:13.734 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[29805]: from='client.35114 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi057-77362-3", "field": "max_bytes", "val": "4096"}]': finished 2024-06-11T13:31:13.734 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[29805]: from='client.34727 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi057-77882-1"}]': finished 2024-06-11T13:31:13.734 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1129714156' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi057-77396-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi057-77396-22"}]: dispatch 2024-06-11T13:31:13.734 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3748941041' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi057-77494-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi057-77494-10"}]: dispatch 2024-06-11T13:31:13.734 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2245769429' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:13.734 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/289483390' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi057-77882-1"}]: dispatch 2024-06-11T13:31:13.734 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1420997228' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi057-77374-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi057-77374-16"}]: dispatch 2024-06-11T13:31:13.734 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[40588]: pgmap v86: 1012 pgs: 288 unknown, 188 creating+peering, 536 active+clean; 145 MiB data, 717 MiB used, 715 GiB / 715 GiB avail; 9.7 MiB/s rd, 30 MiB/s wr, 154 op/s 2024-06-11T13:31:13.734 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[40588]: onexx 2024-06-11T13:31:13.734 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:31:13.735 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:31:13.735 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[40588]: Health check update: 18 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:31:13.735 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[40588]: from='client.35348 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-06-11T13:31:13.735 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[40588]: from='client.25297 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi057-77831-7"}]: dispatch 2024-06-11T13:31:13.735 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[40588]: from='client.35297 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi057-77494-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:13.735 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[40588]: from='client.35306 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi057-77396-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:13.735 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[40588]: from='client.35312 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi057-77374-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:13.735 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[40588]: from='client.25285 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:13.735 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[40588]: from='client.35114 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi057-77362-3", "field": "max_bytes", "val": "4096"}]': finished 2024-06-11T13:31:13.735 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[40588]: from='client.34727 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi057-77882-1"}]': finished 2024-06-11T13:31:13.735 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1129714156' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi057-77396-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi057-77396-22"}]: dispatch 2024-06-11T13:31:13.735 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3748941041' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi057-77494-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi057-77494-10"}]: dispatch 2024-06-11T13:31:13.735 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2245769429' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:13.736 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/289483390' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi057-77882-1"}]: dispatch 2024-06-11T13:31:13.736 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1420997228' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi057-77374-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi057-77374-16"}]: dispatch 2024-06-11T13:31:13.736 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/4284754880' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi057-77351-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:13.736 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:13.736 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2607108900' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi057-77831-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:13.736 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[29805]: osdmap e85: 8 total, 8 up, 8 in 2024-06-11T13:31:13.736 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[29805]: from='client.25303 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi057-77785-7"}]: dispatch 2024-06-11T13:31:13.737 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[29805]: from='client.25288 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:13.737 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3479760694' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi057-77785-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:13.737 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[29805]: from='client.24901 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi057-78571-1"}]: dispatch 2024-06-11T13:31:13.737 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[29805]: from='client.35306 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi057-77396-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi057-77396-22"}]: dispatch 2024-06-11T13:31:13.737 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[29805]: from='client.35297 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi057-77494-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi057-77494-10"}]: dispatch 2024-06-11T13:31:13.737 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[29805]: from='client.34727 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi057-77882-1"}]: dispatch 2024-06-11T13:31:13.737 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/4284754880' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi057-77351-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:13.737 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:13.737 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2607108900' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi057-77831-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:13.737 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[40588]: osdmap e85: 8 total, 8 up, 8 in 2024-06-11T13:31:13.738 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[40588]: from='client.25303 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi057-77785-7"}]: dispatch 2024-06-11T13:31:13.738 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[40588]: from='client.25288 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:13.738 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3479760694' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi057-77785-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:13.738 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[40588]: from='client.24901 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi057-78571-1"}]: dispatch 2024-06-11T13:31:13.738 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[40588]: from='client.35306 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi057-77396-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi057-77396-22"}]: dispatch 2024-06-11T13:31:13.738 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[40588]: from='client.35297 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi057-77494-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi057-77494-10"}]: dispatch 2024-06-11T13:31:13.738 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-mon[40588]: from='client.34727 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi057-77882-1"}]: dispatch 2024-06-11T13:31:13.738 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:13 smithi107 ceph-mon[33178]: pgmap v86: 1012 pgs: 288 unknown, 188 creating+peering, 536 active+clean; 145 MiB data, 717 MiB used, 715 GiB / 715 GiB avail; 9.7 MiB/s rd, 30 MiB/s wr, 154 op/s 2024-06-11T13:31:13.739 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:13 smithi107 ceph-mon[33178]: onexx 2024-06-11T13:31:13.739 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:13 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:31:13.739 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:13 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:31:13.739 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:13 smithi107 ceph-mon[33178]: Health check update: 18 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:31:13.739 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:13 smithi107 ceph-mon[33178]: from='client.35348 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-06-11T13:31:13.739 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:13 smithi107 ceph-mon[33178]: from='client.25297 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi057-77831-7"}]: dispatch 2024-06-11T13:31:13.739 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:13 smithi107 ceph-mon[33178]: from='client.35297 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi057-77494-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:13.739 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:13 smithi107 ceph-mon[33178]: from='client.35306 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi057-77396-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:13.739 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:13 smithi107 ceph-mon[33178]: from='client.35312 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi057-77374-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:13.739 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:13 smithi107 ceph-mon[33178]: from='client.25285 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:13.740 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:13 smithi107 ceph-mon[33178]: from='client.35114 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi057-77362-3", "field": "max_bytes", "val": "4096"}]': finished 2024-06-11T13:31:13.740 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:13 smithi107 ceph-mon[33178]: from='client.34727 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi057-77882-1"}]': finished 2024-06-11T13:31:13.740 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:13 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1129714156' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi057-77396-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi057-77396-22"}]: dispatch 2024-06-11T13:31:13.740 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:13 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3748941041' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi057-77494-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi057-77494-10"}]: dispatch 2024-06-11T13:31:13.740 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:13 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2245769429' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:13.740 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:13 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/289483390' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi057-77882-1"}]: dispatch 2024-06-11T13:31:13.740 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:13 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1420997228' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi057-77374-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi057-77374-16"}]: dispatch 2024-06-11T13:31:13.740 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:13 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/4284754880' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi057-77351-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:13.740 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:13 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:13.740 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:13 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2607108900' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi057-77831-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:13.740 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:13 smithi107 ceph-mon[33178]: osdmap e85: 8 total, 8 up, 8 in 2024-06-11T13:31:13.740 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:13 smithi107 ceph-mon[33178]: from='client.25303 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi057-77785-7"}]: dispatch 2024-06-11T13:31:13.740 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:13 smithi107 ceph-mon[33178]: from='client.25288 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:13.740 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:13 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3479760694' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi057-77785-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:13.741 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:13 smithi107 ceph-mon[33178]: from='client.24901 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi057-78571-1"}]: dispatch 2024-06-11T13:31:13.741 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:13 smithi107 ceph-mon[33178]: from='client.35306 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi057-77396-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi057-77396-22"}]: dispatch 2024-06-11T13:31:13.741 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:13 smithi107 ceph-mon[33178]: from='client.35297 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi057-77494-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi057-77494-10"}]: dispatch 2024-06-11T13:31:13.741 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:13 smithi107 ceph-mon[33178]: from='client.34727 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi057-77882-1"}]: dispatch 2024-06-11T13:31:14.032 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: [==========] Running 4 tests from 1 test suite. 2024-06-11T13:31:14.032 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: [----------] Global test environment set-up. 2024-06-11T13:31:14.032 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP 2024-06-11T13:31:14.032 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterEarly 2024-06-11T13:31:14.032 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterEarly (5048 ms) 2024-06-11T13:31:14.032 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterLate 2024-06-11T13:31:14.033 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterLate (47 ms) 2024-06-11T13:31:14.033 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Status 2024-06-11T13:31:14.033 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: [ OK ] LibRadosServicePP.Status (20031 ms) 2024-06-11T13:31:14.033 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Close 2024-06-11T13:31:14.033 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: attempt 0 of 20 2024-06-11T13:31:14.033 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: failed to deregister: 2024-06-11T13:31:14.033 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: { 2024-06-11T13:31:14.033 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "epoch": 18, 2024-06-11T13:31:14.033 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "modified": "2024-06-11T13:31:09.997169+0000", 2024-06-11T13:31:14.033 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "services": { 2024-06-11T13:31:14.033 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "laundry": { 2024-06-11T13:31:14.033 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "daemons": { 2024-06-11T13:31:14.034 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "summary": "", 2024-06-11T13:31:14.034 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "close-test-pid78049": { 2024-06-11T13:31:14.034 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "start_epoch": 18, 2024-06-11T13:31:14.034 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "start_stamp": "2024-06-11T13:31:09.018862+0000", 2024-06-11T13:31:14.034 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "gid": 35324, 2024-06-11T13:31:14.034 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "addr": "172.21.15.57:0/242207835", 2024-06-11T13:31:14.034 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "metadata": { 2024-06-11T13:31:14.034 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "arch": "x86_64", 2024-06-11T13:31:14.034 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "ceph_release": "squid", 2024-06-11T13:31:14.034 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "ceph_version": "ceph version 19.0.0-2976-g3f51ea39 (3f51ea3944d39ca13aa40153c9a4ee298306679d) squid (rc)", 2024-06-11T13:31:14.034 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "ceph_version_short": "19.0.0-2976-g3f51ea39", 2024-06-11T13:31:14.034 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2024-06-11T13:31:14.035 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "distro": "centos", 2024-06-11T13:31:14.035 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2024-06-11T13:31:14.035 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "distro_version": "9", 2024-06-11T13:31:14.035 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "foo": "bar", 2024-06-11T13:31:14.035 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "hostname": "smithi057", 2024-06-11T13:31:14.035 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024", 2024-06-11T13:31:14.035 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "kernel_version": "5.14.0-452.el9.x86_64", 2024-06-11T13:31:14.035 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "mem_swap_kb": "0", 2024-06-11T13:31:14.035 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "mem_total_kb": "32494356", 2024-06-11T13:31:14.035 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "os": "Linux", 2024-06-11T13:31:14.035 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "this": "that" 2024-06-11T13:31:14.035 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: }, 2024-06-11T13:31:14.035 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "task_status": {} 2024-06-11T13:31:14.036 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: }, 2024-06-11T13:31:14.036 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "pid77983": { 2024-06-11T13:31:14.036 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "start_epoch": 17, 2024-06-11T13:31:14.036 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "start_stamp": "2024-06-11T13:31:01.334683+0000", 2024-06-11T13:31:14.036 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "gid": 25120, 2024-06-11T13:31:14.036 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "addr": "172.21.15.57:0/4124237174", 2024-06-11T13:31:14.036 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "metadata": { 2024-06-11T13:31:14.036 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "arch": "x86_64", 2024-06-11T13:31:14.036 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "ceph_release": "squid", 2024-06-11T13:31:14.036 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "ceph_version": "ceph version 19.0.0-2976-g3f51ea39 (3f51ea3944d39ca13aa40153c9a4ee298306679d) squid (rc)", 2024-06-11T13:31:14.036 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "ceph_version_short": "19.0.0-2976-g3f51ea39", 2024-06-11T13:31:14.036 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2024-06-11T13:31:14.036 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "distro": "centos", 2024-06-11T13:31:14.037 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2024-06-11T13:31:14.037 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "distro_version": "9", 2024-06-11T13:31:14.037 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "foo": "bar", 2024-06-11T13:31:14.037 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "hostname": "smithi057", 2024-06-11T13:31:14.037 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024", 2024-06-11T13:31:14.037 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "kernel_version": "5.14.0-452.el9.x86_64", 2024-06-11T13:31:14.037 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "mem_swap_kb": "0", 2024-06-11T13:31:14.037 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "mem_total_kb": "32494356", 2024-06-11T13:31:14.037 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "os": "Linux", 2024-06-11T13:31:14.037 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "this": "that" 2024-06-11T13:31:14.037 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: }, 2024-06-11T13:31:14.037 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "task_status": {} 2024-06-11T13:31:14.037 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: } 2024-06-11T13:31:14.038 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: } 2024-06-11T13:31:14.038 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: }, 2024-06-11T13:31:14.038 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "rgw": { 2024-06-11T13:31:14.038 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "daemons": { 2024-06-11T13:31:14.038 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "summary": "", 2024-06-11T13:31:14.038 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "24379": { 2024-06-11T13:31:14.038 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "start_epoch": 12, 2024-06-11T13:31:14.038 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "start_stamp": "2024-06-11T13:28:20.560327+0000", 2024-06-11T13:31:14.038 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "gid": 24379, 2024-06-11T13:31:14.038 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "addr": "172.21.15.57:0/2628526915", 2024-06-11T13:31:14.038 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "metadata": { 2024-06-11T13:31:14.038 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "arch": "x86_64", 2024-06-11T13:31:14.038 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "ceph_release": "squid", 2024-06-11T13:31:14.039 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "ceph_version": "ceph version 19.0.0-2976-g3f51ea39 (3f51ea3944d39ca13aa40153c9a4ee298306679d) squid (rc)", 2024-06-11T13:31:14.039 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "ceph_version_short": "19.0.0-2976-g3f51ea39", 2024-06-11T13:31:14.039 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "container_hostname": "smithi057", 2024-06-11T13:31:14.039 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "container_image": "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310", 2024-06-11T13:31:14.039 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2024-06-11T13:31:14.039 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "distro": "centos", 2024-06-11T13:31:14.039 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2024-06-11T13:31:14.039 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "distro_version": "9", 2024-06-11T13:31:14.039 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "frontend_config#0": "beast port=80", 2024-06-11T13:31:14.039 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "frontend_type#0": "beast", 2024-06-11T13:31:14.039 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "hostname": "smithi057", 2024-06-11T13:31:14.039 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "id": "foo.a", 2024-06-11T13:31:14.040 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024", 2024-06-11T13:31:14.040 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "kernel_version": "5.14.0-452.el9.x86_64", 2024-06-11T13:31:14.040 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "mem_swap_kb": "0", 2024-06-11T13:31:14.040 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "mem_total_kb": "32494356", 2024-06-11T13:31:14.040 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "num_handles": "1", 2024-06-11T13:31:14.040 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "os": "Linux", 2024-06-11T13:31:14.040 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "pid": "2", 2024-06-11T13:31:14.040 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "realm_id": "", 2024-06-11T13:31:14.040 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "realm_name": "", 2024-06-11T13:31:14.040 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "zone_id": "f480d397-ad70-4c16-844a-5ad58a2b8f31", 2024-06-11T13:31:14.040 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "zone_name": "default", 2024-06-11T13:31:14.040 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "zonegroup_id": "f9ffc93c-24d2-4773-8bd8-bf937361fcfc", 2024-06-11T13:31:14.040 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "zonegroup_name": "default" 2024-06-11T13:31:14.041 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:31:13 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:31:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:31:15.164 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:14 smithi107 ceph-mon[33178]: pgmap v88: 900 pgs: 157 creating+peering, 35 unknown, 708 active+clean; 145 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 925 B/s rd, 10 KiB/s wr, 11 op/s 2024-06-11T13:31:15.164 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:14 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:15.164 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:14 smithi107 ceph-mon[33178]: from='client.35312 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi057-77374-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi057-77374-16"}]: dispatch 2024-06-11T13:31:15.164 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:14 smithi107 ceph-mon[33178]: from='client.35348 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-06-11T13:31:15.164 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:14 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2747983990' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-06-11T13:31:15.164 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:14 smithi107 ceph-mon[33178]: twoxx 2024-06-11T13:31:15.164 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:14 smithi107 ceph-mon[33178]: from='client.35330 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi057-77351-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:15.164 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:14 smithi107 ceph-mon[33178]: from='client.35318 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi057-78504-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi057-78504-2"}]': finished 2024-06-11T13:31:15.164 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:14 smithi107 ceph-mon[33178]: from='client.25288 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:15.165 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:14 smithi107 ceph-mon[33178]: from='client.24901 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi057-78571-1"}]': finished 2024-06-11T13:31:15.165 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:14 smithi107 ceph-mon[33178]: from='client.34727 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi057-77882-1"}]': finished 2024-06-11T13:31:15.165 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:14 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2305832357' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi057-78571-1"}]: dispatch 2024-06-11T13:31:15.165 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:14 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/77244975' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-06-11T13:31:15.165 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:14 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:15.165 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:14 smithi107 ceph-mon[33178]: osdmap e86: 8 total, 8 up, 8 in 2024-06-11T13:31:15.165 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:14 smithi107 ceph-mon[33178]: from='client.35348 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-06-11T13:31:15.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:14 smithi057 ceph-mon[40588]: pgmap v88: 900 pgs: 157 creating+peering, 35 unknown, 708 active+clean; 145 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 925 B/s rd, 10 KiB/s wr, 11 op/s 2024-06-11T13:31:15.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:14 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:15.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:14 smithi057 ceph-mon[40588]: from='client.35312 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi057-77374-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi057-77374-16"}]: dispatch 2024-06-11T13:31:15.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:14 smithi057 ceph-mon[40588]: from='client.35348 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-06-11T13:31:15.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:14 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2747983990' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-06-11T13:31:15.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:14 smithi057 ceph-mon[40588]: twoxx 2024-06-11T13:31:15.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:14 smithi057 ceph-mon[40588]: from='client.35330 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi057-77351-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:15.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:14 smithi057 ceph-mon[40588]: from='client.35318 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi057-78504-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi057-78504-2"}]': finished 2024-06-11T13:31:15.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:14 smithi057 ceph-mon[40588]: from='client.25288 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:15.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:14 smithi057 ceph-mon[40588]: from='client.24901 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi057-78571-1"}]': finished 2024-06-11T13:31:15.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:14 smithi057 ceph-mon[40588]: from='client.34727 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi057-77882-1"}]': finished 2024-06-11T13:31:15.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:14 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2305832357' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi057-78571-1"}]: dispatch 2024-06-11T13:31:15.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:14 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/77244975' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-06-11T13:31:15.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:14 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:15.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:14 smithi057 ceph-mon[40588]: osdmap e86: 8 total, 8 up, 8 in 2024-06-11T13:31:15.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:14 smithi057 ceph-mon[40588]: from='client.35348 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-06-11T13:31:15.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:14 smithi057 ceph-mon[29805]: pgmap v88: 900 pgs: 157 creating+peering, 35 unknown, 708 active+clean; 145 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 925 B/s rd, 10 KiB/s wr, 11 op/s 2024-06-11T13:31:15.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:14 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:15.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:14 smithi057 ceph-mon[29805]: from='client.35312 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi057-77374-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi057-77374-16"}]: dispatch 2024-06-11T13:31:15.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:14 smithi057 ceph-mon[29805]: from='client.35348 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-06-11T13:31:15.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:14 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2747983990' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-06-11T13:31:15.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:14 smithi057 ceph-mon[29805]: twoxx 2024-06-11T13:31:15.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:14 smithi057 ceph-mon[29805]: from='client.35330 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi057-77351-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:15.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:14 smithi057 ceph-mon[29805]: from='client.35318 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi057-78504-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi057-78504-2"}]': finished 2024-06-11T13:31:15.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:14 smithi057 ceph-mon[29805]: from='client.25288 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:15.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:14 smithi057 ceph-mon[29805]: from='client.24901 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi057-78571-1"}]': finished 2024-06-11T13:31:15.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:14 smithi057 ceph-mon[29805]: from='client.34727 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi057-77882-1"}]': finished 2024-06-11T13:31:15.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:14 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2305832357' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi057-78571-1"}]: dispatch 2024-06-11T13:31:15.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:14 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/77244975' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-06-11T13:31:15.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:14 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:15.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:14 smithi057 ceph-mon[29805]: osdmap e86: 8 total, 8 up, 8 in 2024-06-11T13:31:15.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:14 smithi057 ceph-mon[29805]: from='client.35348 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-06-11T13:31:15.325 INFO:tasks.workunit.client.0.smithi057.stdout: open_pools_parallel: process_1_[78226]: starting. 2024-06-11T13:31:15.325 INFO:tasks.workunit.client.0.smithi057.stdout: open_pools_parallel: process_1_[78226]: creating pool ceph_test_rados_open_pools_parallel.smithi057-78151 2024-06-11T13:31:15.325 INFO:tasks.workunit.client.0.smithi057.stdout: open_pools_parallel: process_1_[78226]: created object 0... 2024-06-11T13:31:15.325 INFO:tasks.workunit.client.0.smithi057.stdout: open_pools_parallel: process_1_[78226]: created object 25... 2024-06-11T13:31:15.325 INFO:tasks.workunit.client.0.smithi057.stdout: open_pools_parallel: process_1_[78226]: created object 49... 2024-06-11T13:31:15.326 INFO:tasks.workunit.client.0.smithi057.stdout: open_pools_parallel: process_1_[78226]: finishing. 2024-06-11T13:31:15.326 INFO:tasks.workunit.client.0.smithi057.stdout: open_pools_parallel: process_1_[78226]: shutting down. 2024-06-11T13:31:15.326 INFO:tasks.workunit.client.0.smithi057.stdout: open_pools_parallel: process_2_[78228]: starting. 2024-06-11T13:31:15.326 INFO:tasks.workunit.client.0.smithi057.stdout: open_pools_parallel: process_2_[78228]: rados_pool_create. 2024-06-11T13:31:15.326 INFO:tasks.workunit.client.0.smithi057.stdout: open_pools_parallel: process_2_[78228]: rados_ioctx_create. 2024-06-11T13:31:15.326 INFO:tasks.workunit.client.0.smithi057.stdout: open_pools_parallel: process_2_[78228]: shutting down. 2024-06-11T13:31:15.326 INFO:tasks.workunit.client.0.smithi057.stdout: open_pools_parallel: ******************************* 2024-06-11T13:31:15.326 INFO:tasks.workunit.client.0.smithi057.stdout: open_pools_parallel: process_3_[79861]: starting. 2024-06-11T13:31:15.326 INFO:tasks.workunit.client.0.smithi057.stdout: open_pools_parallel: process_3_[79861]: creating pool ceph_test_rados_open_pools_parallel.smithi057-78151 2024-06-11T13:31:15.326 INFO:tasks.workunit.client.0.smithi057.stdout: open_pools_parallel: process_3_[79861]: created object 0... 2024-06-11T13:31:15.326 INFO:tasks.workunit.client.0.smithi057.stdout: open_pools_parallel: process_3_[79861]: created object 25... 2024-06-11T13:31:15.326 INFO:tasks.workunit.client.0.smithi057.stdout: open_pools_parallel: process_3_[79861]: created object 49... 2024-06-11T13:31:15.327 INFO:tasks.workunit.client.0.smithi057.stdout: open_pools_parallel: process_3_[79861]: finishing. 2024-06-11T13:31:15.327 INFO:tasks.workunit.client.0.smithi057.stdout: open_pools_parallel: process_3_[79861]: shutting down. 2024-06-11T13:31:15.327 INFO:tasks.workunit.client.0.smithi057.stdout: open_pools_parallel: ******************************* 2024-06-11T13:31:15.327 INFO:tasks.workunit.client.0.smithi057.stdout: open_pools_parallel: process_4_[79862]: starting. 2024-06-11T13:31:15.327 INFO:tasks.workunit.client.0.smithi057.stdout: open_pools_parallel: process_4_[79862]: rados_pool_create. 2024-06-11T13:31:15.327 INFO:tasks.workunit.client.0.smithi057.stdout: open_pools_parallel: process_4_[79862]: rados_ioctx_create. 2024-06-11T13:31:15.327 INFO:tasks.workunit.client.0.smithi057.stdout: open_pools_parallel: process_4_[79862]: shutting down. 2024-06-11T13:31:15.327 INFO:tasks.workunit.client.0.smithi057.stdout: open_pools_parallel: ******************************* 2024-06-11T13:31:15.327 INFO:tasks.workunit.client.0.smithi057.stdout: open_pools_parallel: ******************************* 2024-06-11T13:31:15.327 INFO:tasks.workunit.client.0.smithi057.stdout: open_pools_parallel: ******* SUCCESS ********** 2024-06-11T13:31:15.795 INFO:tasks.workunit.client.0.smithi057.stdout: delete_pools_parallel: process_1_[78399]: starting. 2024-06-11T13:31:15.795 INFO:tasks.workunit.client.0.smithi057.stdout: delete_pools_parallel: process_1_[78399]: creating pool ceph_test_rados_delete_pools_parallel.smithi057-78314 2024-06-11T13:31:15.795 INFO:tasks.workunit.client.0.smithi057.stdout: delete_pools_parallel: process_1_[78399]: created object 0... 2024-06-11T13:31:15.795 INFO:tasks.workunit.client.0.smithi057.stdout: delete_pools_parallel: process_1_[78399]: created object 25... 2024-06-11T13:31:15.795 INFO:tasks.workunit.client.0.smithi057.stdout: delete_pools_parallel: process_1_[78399]: created object 49... 2024-06-11T13:31:15.795 INFO:tasks.workunit.client.0.smithi057.stdout: delete_pools_parallel: process_1_[78399]: finishing. 2024-06-11T13:31:15.795 INFO:tasks.workunit.client.0.smithi057.stdout: delete_pools_parallel: process_1_[78399]: shutting down. 2024-06-11T13:31:15.795 INFO:tasks.workunit.client.0.smithi057.stdout: delete_pools_parallel: process_2_[78402]: starting. 2024-06-11T13:31:15.795 INFO:tasks.workunit.client.0.smithi057.stdout: delete_pools_parallel: process_2_[78402]: deleting pool ceph_test_rados_delete_pools_parallel.smithi057-78314 2024-06-11T13:31:15.796 INFO:tasks.workunit.client.0.smithi057.stdout: delete_pools_parallel: process_2_[78402]: shutting down. 2024-06-11T13:31:15.796 INFO:tasks.workunit.client.0.smithi057.stdout: delete_pools_parallel: ******************************* 2024-06-11T13:31:15.796 INFO:tasks.workunit.client.0.smithi057.stdout: delete_pools_parallel: process_3_[79767]: starting. 2024-06-11T13:31:15.796 INFO:tasks.workunit.client.0.smithi057.stdout: delete_pools_parallel: process_3_[79767]: creating pool ceph_test_rados_delete_pools_parallel.smithi057-78314 2024-06-11T13:31:15.796 INFO:tasks.workunit.client.0.smithi057.stdout: delete_pools_parallel: process_3_[79767]: created object 0... 2024-06-11T13:31:15.796 INFO:tasks.workunit.client.0.smithi057.stdout: delete_pools_parallel: process_3_[79767]: created object 25... 2024-06-11T13:31:15.796 INFO:tasks.workunit.client.0.smithi057.stdout: delete_pools_parallel: process_3_[79767]: created object 49... 2024-06-11T13:31:15.796 INFO:tasks.workunit.client.0.smithi057.stdout: delete_pools_parallel: process_3_[79767]: finishing. 2024-06-11T13:31:15.796 INFO:tasks.workunit.client.0.smithi057.stdout: delete_pools_parallel: process_3_[79767]: shutting down. 2024-06-11T13:31:15.796 INFO:tasks.workunit.client.0.smithi057.stdout: delete_pools_parallel: ******************************* 2024-06-11T13:31:15.796 INFO:tasks.workunit.client.0.smithi057.stdout: delete_pools_parallel: process_5_[79769]: starting. 2024-06-11T13:31:15.796 INFO:tasks.workunit.client.0.smithi057.stdout: delete_pools_parallel: process_5_[79769]: listing objects. 2024-06-11T13:31:15.796 INFO:tasks.workunit.client.0.smithi057.stdout: delete_pools_parallel: process_5_[79769]: listed object 0... 2024-06-11T13:31:15.796 INFO:tasks.workunit.client.0.smithi057.stdout: delete_pools_parallel: process_5_[79769]: listed object 25... 2024-06-11T13:31:15.797 INFO:tasks.workunit.client.0.smithi057.stdout: delete_pools_parallel: process_5_[79769]: saw 50 objects 2024-06-11T13:31:15.797 INFO:tasks.workunit.client.0.smithi057.stdout: delete_pools_parallel: process_5_[79769]: shutting down. 2024-06-11T13:31:15.797 INFO:tasks.workunit.client.0.smithi057.stdout: delete_pools_parallel: ******************************* 2024-06-11T13:31:15.797 INFO:tasks.workunit.client.0.smithi057.stdout: delete_pools_parallel: process_4_[79768]: starting. 2024-06-11T13:31:15.797 INFO:tasks.workunit.client.0.smithi057.stdout: delete_pools_parallel: process_4_[79768]: deleting pool ceph_test_rados_delete_pools_parallel.smithi057-78314 2024-06-11T13:31:15.797 INFO:tasks.workunit.client.0.smithi057.stdout: delete_pools_parallel: process_4_[79768]: shutting down. 2024-06-11T13:31:15.797 INFO:tasks.workunit.client.0.smithi057.stdout: delete_pools_parallel: ******************************* 2024-06-11T13:31:15.797 INFO:tasks.workunit.client.0.smithi057.stdout: delete_pools_parallel: ******************************* 2024-06-11T13:31:15.797 INFO:tasks.workunit.client.0.smithi057.stdout: delete_pools_parallel: ******* SUCCESS ********** 2024-06-11T13:31:16.847 INFO:tasks.workunit.client.0.smithi057.stdout:n.c [INF] from='client.? v1:172.21.15.57:0/289483390' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi057-77882-1"}]: dispatch 2024-06-11T13:31:16.847 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:12.062770+0000 mon.c [INF] from='client.? v1:172.21.15.57:0/1420997228' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi057-77374-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi057-77374-16"}]: dispatch 2024-06-11T13:31:16.847 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:12.063924+0000 mon.c [INF] from='client.? v1:172.21.15.57:0/4284754880' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi057-77351-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:16.847 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:12.530036+0000 mon.b [INF] from='client.? v1:172.21.15.57:0/2607108900' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi057-77831-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:16.847 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:12.976294+0000 mon.a [INF] from='client.25303 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi057-77785-7"}]: dispatch 2024-06-11T13:31:16.847 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:12.976790+0000 mon.a [INF] from='client.25288 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:16.847 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:12.976993+0000 mon.b [INF] from='client.? v1:172.21.15.57:0/3479760694' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi057-77785-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:16.847 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:12.977242+0000 mon.a [INF] from='client.24901 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi057-78571-1"}]: dispatch 2024-06-11T13:31:16.847 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:12.981482+0000 mon.a [INF] from='client.35306 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi057-77396-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi057-77396-22"}]: dispatch 2024-06-11T13:31:16.848 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:12.982123+0000 mon.a [INF] from='client.35297 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi057-77494-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi057-77494-10"}]: dispatch 2024-06-11T13:31:16.848 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:12.982746+0000 mon.a [INF] from='client.34727 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi057-77882-1"}]: dispatch 2024-06-11T13:31:16.848 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:13.452972+0000 mon.a [INF] from='client.35312 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi057-77374-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi057-77374-16"}]: dispatch 2024-06-11T13:31:16.848 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:13.453344+0000 mon.a [INF] from='client.35348 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-06-11T13:31:16.848 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:13.459331+0000 mon.c [INF] from='client.? v1:172.21.15.57:0/2747983990' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-06-11T13:31:16.848 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:13.459966+0000 client.admin [INF] twoxx 2024-06-11T13:31:16.848 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:13.461120+0000 mon.a [INF] from='client.35330 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi057-77351-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:17.153 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:13.463088+0000 mon.a [INF] from='client.35318 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi057-78504-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi057-78504-2"}]': finished 2024-06-11T13:31:17.153 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:13.463158+0000 mon.a [INF] from='client.25288 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:17.153 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:13.463218+0000 mon.a [INF] from='client.24901 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi057-78571-1"}]': finished 2024-06-11T13:31:17.154 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:13.463251+0000 mon.a [INF] from='client.34727 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi057-77882-1"}]': finished 2024-06-11T13:31:17.154 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:13.467661+0000 mon.b [INF] from='client.? v1:172.21.15.57:0/2305832357' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi057-78571-1"}]: dispatch 2024-06-11T13:31:17.154 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:14.384896+0000 mon.a [INF] from='client.35348 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-06-11T13:31:17.154 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:14.839931+0000 mon.a [INF] from='client.25279 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi057-77506-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:17.154 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:14.840315+0000 mon.a [INF] from='client.25108 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:17.154 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:14.842794+0000 mon.a [INF] from='client.35348 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-06-11T13:31:17.154 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:14.854241+0000 mon.a [INF] from='client.35306 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi057-77396-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi057-77396-22"}]': finished 2024-06-11T13:31:17.154 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:14.854327+0000 mon.a [INF] from='client.35297 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi057-77494-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi057-77494-10"}]': finished 2024-06-11T13:31:17.154 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:14.854404+0000 mon.a [INF] from='client.35330 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi057-77351-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:17.154 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:15.324556+0000 mon.c [INF] from='client.? v1:172.21.15.57:0/197523128' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:17.154 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:15.776891+0000 mon.a [INF] from='client.35363 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:17.155 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:15.777707+0000 mon.a [INF] from='client.25297 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi057-77831-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:17.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:17 smithi057 ceph-mon[29805]: pgmap v91: 1004 pgs: 157 creating+peering, 203 unknown, 644 active+clean; 145 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-06-11T13:31:17.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:17 smithi057 ceph-mon[29805]: from='mon.? v1:172.21.15.57:0/45435016' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-06-11T13:31:17.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:17 smithi057 ceph-mon[29805]: from='client.25279 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi057-77506-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:17.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:17 smithi057 ceph-mon[29805]: from='client.25108 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:17.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:17 smithi057 ceph-mon[29805]: from='client.35348 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-06-11T13:31:17.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:17 smithi057 ceph-mon[29805]: from='client.35306 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi057-77396-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi057-77396-22"}]': finished 2024-06-11T13:31:17.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:17 smithi057 ceph-mon[29805]: from='client.35297 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi057-77494-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi057-77494-10"}]': finished 2024-06-11T13:31:17.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:17 smithi057 ceph-mon[29805]: from='client.35330 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi057-77351-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:17.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:17 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:17.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:17 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/197523128' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:17.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:17 smithi057 ceph-mon[29805]: osdmap e87: 8 total, 8 up, 8 in 2024-06-11T13:31:17.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:17 smithi057 ceph-mon[29805]: from='client.35363 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:17.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:17 smithi057 ceph-mon[29805]: from='client.25297 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi057-77831-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:17.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:17 smithi057 ceph-mon[29805]: from='client.25303 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi057-77785-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:17.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:17 smithi057 ceph-mon[29805]: from='client.24901 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi057-78571-1"}]: dispatch 2024-06-11T13:31:17.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:17 smithi057 ceph-mon[40588]: pgmap v91: 1004 pgs: 157 creating+peering, 203 unknown, 644 active+clean; 145 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-06-11T13:31:17.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:17 smithi057 ceph-mon[40588]: from='mon.? v1:172.21.15.57:0/45435016' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-06-11T13:31:17.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:17 smithi057 ceph-mon[40588]: from='client.25279 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi057-77506-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:17.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:17 smithi057 ceph-mon[40588]: from='client.25108 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:17.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:17 smithi057 ceph-mon[40588]: from='client.35348 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-06-11T13:31:17.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:17 smithi057 ceph-mon[40588]: from='client.35306 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi057-77396-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi057-77396-22"}]': finished 2024-06-11T13:31:17.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:17 smithi057 ceph-mon[40588]: from='client.35297 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi057-77494-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi057-77494-10"}]': finished 2024-06-11T13:31:17.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:17 smithi057 ceph-mon[40588]: from='client.35330 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi057-77351-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:17.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:17 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:17.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:17 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/197523128' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:17.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:17 smithi057 ceph-mon[40588]: osdmap e87: 8 total, 8 up, 8 in 2024-06-11T13:31:17.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:17 smithi057 ceph-mon[40588]: from='client.35363 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:17.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:17 smithi057 ceph-mon[40588]: from='client.25297 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi057-77831-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:17.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:17 smithi057 ceph-mon[40588]: from='client.25303 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi057-77785-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:17.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:17 smithi057 ceph-mon[40588]: from='client.24901 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi057-78571-1"}]: dispatch 2024-06-11T13:31:17.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:17 smithi107 ceph-mon[33178]: pgmap v91: 1004 pgs: 157 creating+peering, 203 unknown, 644 active+clean; 145 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-06-11T13:31:17.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:17 smithi107 ceph-mon[33178]: from='mon.? v1:172.21.15.57:0/45435016' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-06-11T13:31:17.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:17 smithi107 ceph-mon[33178]: from='client.25279 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi057-77506-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:17.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:17 smithi107 ceph-mon[33178]: from='client.25108 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:17.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:17 smithi107 ceph-mon[33178]: from='client.35348 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-06-11T13:31:17.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:17 smithi107 ceph-mon[33178]: from='client.35306 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi057-77396-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi057-77396-22"}]': finished 2024-06-11T13:31:17.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:17 smithi107 ceph-mon[33178]: from='client.35297 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi057-77494-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi057-77494-10"}]': finished 2024-06-11T13:31:17.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:17 smithi107 ceph-mon[33178]: from='client.35330 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi057-77351-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:17.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:17 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:17.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:17 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/197523128' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:17.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:17 smithi107 ceph-mon[33178]: osdmap e87: 8 total, 8 up, 8 in 2024-06-11T13:31:17.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:17 smithi107 ceph-mon[33178]: from='client.35363 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:17.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:17 smithi107 ceph-mon[33178]: from='client.25297 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi057-77831-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:17.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:17 smithi107 ceph-mon[33178]: from='client.25303 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi057-77785-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:17.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:17 smithi107 ceph-mon[33178]: from='client.24901 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi057-78571-1"}]: dispatch 2024-06-11T13:31:18.547 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:15.777837+0000 mon.a [INF] from='client.25303 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi057-77785-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:18.547 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:15.777977+0000 mon.a [INF] from='client.24901 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi057-78571-1"}]: dispatch 2024-06-11T13:31:18.547 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:15.779478+0000 mon.a [INF] from='client.35312 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi057-77374-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi057-77374-16"}]': finished 2024-06-11T13:31:18.547 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:15.779548+0000 mon.a [INF] from='client.25279 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi057-77506-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:18.547 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:15.779603+0000 mon.a [INF] from='client.25108 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:18.547 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:15.783661+0000 mon.b [INF] from='client.? v1:172.21.15.57:0/232412548' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi057-77506-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi057-77506-10"}]: dispatch 2024-06-11T13:31:18.548 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:15.796489+0000 mon.b [INF] from='client.? v1:172.21.15.57:0/2245769429' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-4", "tierpool": "test-rados-api-smithi057-77599-4-cache"}]: dispatch 2024-06-11T13:31:18.548 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:15.873049+0000 mon.c [INF] from='client.? v1:172.21.15.57:0/3799830007' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi057-77874-12"}]: dispatch 2024-06-11T13:31:18.548 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:16.851308+0000 mon.c [INF] from='client.? v1:172.21.15.57:0/2747983990' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-06-11T13:31:18.548 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:17.140351+0000 mon.a [INF] from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:31:18.548 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:17.140808+0000 client.admin [INF] threexx 2024-06-11T13:31:18.548 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:17.141469+0000 mon.a [INF] from='client.35402 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi057-77874-12"}]: dispatch 2024-06-11T13:31:18.548 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:17.144180+0000 mon.a [INF] from='client.35348 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-06-11T13:31:18.548 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:17.145173+0000 mon.a [INF] from='client.25279 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi057-77506-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi057-77506-10"}]: dispatch 2024-06-11T13:31:18.548 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:17.145356+0000 mon.a [INF] from='client.25108 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-4", "tierpool": "test-rados-api-smithi057-77599-4-cache"}]: dispatch 2024-06-11T13:31:18.548 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:17.146449+0000 mon.a [INF] from='client.35363 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:18.565 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:17.146530+0000 mon.a [INF] from='client.25297 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStat cls: Running main() from gmock_main.cc 2024-06-11T13:31:18.565 INFO:tasks.workunit.client.0.smithi057.stdout: cls: [==========] Running 2 tests from 1 test suite. 2024-06-11T13:31:18.565 INFO:tasks.workunit.client.0.smithi057.stdout: cls: [----------] Global test environment set-up. 2024-06-11T13:31:18.566 INFO:tasks.workunit.client.0.smithi057.stdout: cls: [----------] 2 tests from NeoRadosCls 2024-06-11T13:31:18.566 INFO:tasks.workunit.client.0.smithi057.stdout: cls: [ RUN ] NeoRadosCls.DNE 2024-06-11T13:31:18.566 INFO:tasks.workunit.client.0.smithi057.stdout: cls: [ OK ] NeoRadosCls.DNE (22504 ms) 2024-06-11T13:31:18.566 INFO:tasks.workunit.client.0.smithi057.stdout: cls: [ RUN ] NeoRadosCls.RemoteReads 2024-06-11T13:31:18.566 INFO:tasks.workunit.client.0.smithi057.stdout: cls: [ OK ] NeoRadosCls.RemoteReads (12075 ms) 2024-06-11T13:31:18.566 INFO:tasks.workunit.client.0.smithi057.stdout: cls: [----------] 2 tests from NeoRadosCls (34579 ms total) 2024-06-11T13:31:18.566 INFO:tasks.workunit.client.0.smithi057.stdout: cls: 2024-06-11T13:31:18.566 INFO:tasks.workunit.client.0.smithi057.stdout: cls: [----------] Global test environment tear-down 2024-06-11T13:31:18.566 INFO:tasks.workunit.client.0.smithi057.stdout: cls: [==========] 2 tests from 1 test suite ran. (34581 ms total) 2024-06-11T13:31:18.566 INFO:tasks.workunit.client.0.smithi057.stdout: cls: [ PASSED ] 2 tests. 2024-06-11T13:31:18.682 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: Running main() from gmock_main.cc 2024-06-11T13:31:18.682 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [==========] Running 37 tests from 2 test suites. 2024-06-11T13:31:18.682 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [----------] Global test environment set-up. 2024-06-11T13:31:18.682 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP 2024-06-11T13:31:18.682 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: seed 77396 2024-06-11T13:31:18.682 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TooBigPP 2024-06-11T13:31:18.682 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ OK ] LibRadosIoPP.TooBigPP (0 ms) 2024-06-11T13:31:18.682 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SimpleWritePP 2024-06-11T13:31:18.682 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ OK ] LibRadosIoPP.SimpleWritePP (179 ms) 2024-06-11T13:31:18.682 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadOpPP 2024-06-11T13:31:18.682 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadOpPP (10 ms) 2024-06-11T13:31:18.683 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SparseReadOpPP 2024-06-11T13:31:18.683 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ OK ] LibRadosIoPP.SparseReadOpPP (7 ms) 2024-06-11T13:31:18.683 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP 2024-06-11T13:31:18.683 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP (8 ms) 2024-06-11T13:31:18.683 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP2 2024-06-11T13:31:18.683 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP2 (3 ms) 2024-06-11T13:31:18.683 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ RUN ] LibRadosIoPP.Checksum 2024-06-11T13:31:18.683 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ OK ] LibRadosIoPP.Checksum (6 ms) 2024-06-11T13:31:18.683 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadIntoBufferlist 2024-06-11T13:31:18.683 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadIntoBufferlist (3 ms) 2024-06-11T13:31:18.683 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ RUN ] LibRadosIoPP.OverlappingWriteRoundTripPP 2024-06-11T13:31:18.683 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ OK ] LibRadosIoPP.OverlappingWriteRoundTripPP (7 ms) 2024-06-11T13:31:18.683 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP 2024-06-11T13:31:18.683 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP (3 ms) 2024-06-11T13:31:18.683 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP2 2024-06-11T13:31:18.684 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP2 (2 ms) 2024-06-11T13:31:18.684 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ RUN ] LibRadosIoPP.AppendRoundTripPP 2024-06-11T13:31:18.684 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ OK ] LibRadosIoPP.AppendRoundTripPP (7 ms) 2024-06-11T13:31:18.684 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TruncTestPP 2024-06-11T13:31:18.684 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ OK ] LibRadosIoPP.TruncTestPP (4 ms) 2024-06-11T13:31:18.684 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RemoveTestPP 2024-06-11T13:31:18.684 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ OK ] LibRadosIoPP.RemoveTestPP (2 ms) 2024-06-11T13:31:18.684 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrsRoundTripPP 2024-06-11T13:31:18.684 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrsRoundTripPP (3 ms) 2024-06-11T13:31:18.684 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RmXattrPP 2024-06-11T13:31:18.684 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ OK ] LibRadosIoPP.RmXattrPP (10 ms) 2024-06-11T13:31:18.684 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-06-11T13:31:18.684 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrListPP (4 ms) 2024-06-11T13:31:18.684 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtPP 2024-06-11T13:31:18.685 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtPP (6 ms) 2024-06-11T13:31:18.685 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtDNEPP 2024-06-11T13:31:18.685 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtDNEPP (4 ms) 2024-06-11T13:31:18.685 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtMismatchPP 2024-06-11T13:31:18.685 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtMismatchPP (6 ms) 2024-06-11T13:31:18.685 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP (274 ms total) 2024-06-11T13:31:18.685 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: 2024-06-11T13:31:18.685 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP 2024-06-11T13:31:18.685 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SimpleWritePP 2024-06-11T13:31:18.685 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SimpleWritePP (3784 ms) 2024-06-11T13:31:18.685 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.ReadOpPP 2024-06-11T13:31:18.685 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ OK ] LibRadosIoECPP.ReadOpPP (24 ms) 2024-06-11T13:31:18.685 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SparseReadOpPP 2024-06-11T13:31:18.685 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SparseReadOpPP (5 ms) 2024-06-11T13:31:18.686 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP 2024-06-11T13:31:18.686 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP (3 ms) 2024-06-11T13:31:18.686 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP2 2024-06-11T13:31:18.686 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP2 (7 ms) 2024-06-11T13:31:18.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:18 smithi107 ceph-mon[33178]: from='client.35312 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi057-77374-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi057-77374-16"}]': finished 2024-06-11T13:31:18.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:18 smithi107 ceph-mon[33178]: from='client.25279 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi057-77506-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:18.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:18 smithi107 ceph-mon[33178]: from='client.25108 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:18.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:18 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/232412548' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi057-77506-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi057-77506-10"}]: dispatch 2024-06-11T13:31:18.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:18 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2245769429' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-4", "tierpool": "test-rados-api-smithi057-77599-4-cache"}]: dispatch 2024-06-11T13:31:18.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:18 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3799830007' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi057-77874-12"}]: dispatch 2024-06-11T13:31:18.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:18 smithi107 ceph-mon[33178]: pgmap v94: 804 pgs: 86 creating+peering, 20 unknown, 698 active+clean; 145 MiB data, 768 MiB used, 714 GiB / 715 GiB avail; 8.7 KiB/s rd, 3 op/s 2024-06-11T13:31:18.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:18 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:18.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:18 smithi107 ceph-mon[33178]: osdmap e88: 8 total, 8 up, 8 in 2024-06-11T13:31:18.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:18 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2747983990' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-06-11T13:31:18.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:18 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:31:18.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:18 smithi107 ceph-mon[33178]: threexx 2024-06-11T13:31:18.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:18 smithi107 ceph-mon[33178]: from='client.35402 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi057-77874-12"}]: dispatch 2024-06-11T13:31:18.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:18 smithi107 ceph-mon[33178]: from='client.35348 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-06-11T13:31:18.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:18 smithi107 ceph-mon[33178]: from='client.25279 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi057-77506-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi057-77506-10"}]: dispatch 2024-06-11T13:31:18.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:18 smithi107 ceph-mon[33178]: from='client.25108 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-4", "tierpool": "test-rados-api-smithi057-77599-4-cache"}]: dispatch 2024-06-11T13:31:18.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:18 smithi107 ceph-mon[33178]: from='client.35363 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:18.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:18 smithi107 ceph-mon[33178]: from='client.25297 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi057-77831-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:18.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:18 smithi107 ceph-mon[33178]: from='client.25303 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi057-77785-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:18.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:18 smithi107 ceph-mon[33178]: from='client.24901 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi057-78571-1"}]': finished 2024-06-11T13:31:18.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:18 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2607108900' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi057-77831-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi057-77831-7"}]: dispatch 2024-06-11T13:31:18.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:18 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3479760694' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi057-77785-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi057-77785-7"}]: dispatch 2024-06-11T13:31:18.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:18 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3799830007' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi057-77874-12"}]: dispatch 2024-06-11T13:31:18.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:18 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/995766982' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:18.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:18 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2501793636' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi057-78571-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:18.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:18 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/197523128' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi057-77882-6", "pg_num": 4}]: dispatch 2024-06-11T13:31:18.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:18 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:18.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:18 smithi107 ceph-mon[33178]: osdmap e89: 8 total, 8 up, 8 in 2024-06-11T13:31:18.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:18 smithi107 ceph-mon[33178]: from='client.35402 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi057-77874-12"}]: dispatch 2024-06-11T13:31:18.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:18 smithi107 ceph-mon[33178]: from='client.35399 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:18.800 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:18 smithi107 ceph-mon[33178]: from='client.35414 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi057-78571-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:18.800 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:18 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3799830007' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi057-77874-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:18.800 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:18 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:31:19.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[29805]: from='client.35312 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi057-77374-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi057-77374-16"}]': finished 2024-06-11T13:31:19.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[29805]: from='client.25279 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi057-77506-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:19.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[29805]: from='client.25108 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:19.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/232412548' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi057-77506-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi057-77506-10"}]: dispatch 2024-06-11T13:31:19.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2245769429' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-4", "tierpool": "test-rados-api-smithi057-77599-4-cache"}]: dispatch 2024-06-11T13:31:19.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3799830007' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi057-77874-12"}]: dispatch 2024-06-11T13:31:19.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[29805]: pgmap v94: 804 pgs: 86 creating+peering, 20 unknown, 698 active+clean; 145 MiB data, 768 MiB used, 714 GiB / 715 GiB avail; 8.7 KiB/s rd, 3 op/s 2024-06-11T13:31:19.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:19.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[29805]: osdmap e88: 8 total, 8 up, 8 in 2024-06-11T13:31:19.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2747983990' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-06-11T13:31:19.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:31:19.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[29805]: threexx 2024-06-11T13:31:19.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[29805]: from='client.35402 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi057-77874-12"}]: dispatch 2024-06-11T13:31:19.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[29805]: from='client.35348 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-06-11T13:31:19.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[29805]: from='client.25279 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi057-77506-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi057-77506-10"}]: dispatch 2024-06-11T13:31:19.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[29805]: from='client.25108 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-4", "tierpool": "test-rados-api-smithi057-77599-4-cache"}]: dispatch 2024-06-11T13:31:19.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[29805]: from='client.35363 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:19.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[29805]: from='client.25297 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi057-77831-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:19.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[29805]: from='client.25303 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi057-77785-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:19.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[29805]: from='client.24901 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi057-78571-1"}]': finished 2024-06-11T13:31:19.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2607108900' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi057-77831-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi057-77831-7"}]: dispatch 2024-06-11T13:31:19.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3479760694' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi057-77785-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi057-77785-7"}]: dispatch 2024-06-11T13:31:19.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3799830007' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi057-77874-12"}]: dispatch 2024-06-11T13:31:19.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/995766982' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:19.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2501793636' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi057-78571-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:19.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/197523128' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi057-77882-6", "pg_num": 4}]: dispatch 2024-06-11T13:31:19.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:19.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[29805]: osdmap e89: 8 total, 8 up, 8 in 2024-06-11T13:31:19.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[29805]: from='client.35402 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi057-77874-12"}]: dispatch 2024-06-11T13:31:19.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[29805]: from='client.35399 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:19.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[29805]: from='client.35414 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi057-78571-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:19.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3799830007' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi057-77874-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:19.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:31:19.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[40588]: from='client.35312 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi057-77374-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi057-77374-16"}]': finished 2024-06-11T13:31:19.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[40588]: from='client.25279 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi057-77506-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:19.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[40588]: from='client.25108 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:19.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/232412548' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi057-77506-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi057-77506-10"}]: dispatch 2024-06-11T13:31:19.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2245769429' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-4", "tierpool": "test-rados-api-smithi057-77599-4-cache"}]: dispatch 2024-06-11T13:31:19.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3799830007' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi057-77874-12"}]: dispatch 2024-06-11T13:31:19.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[40588]: pgmap v94: 804 pgs: 86 creating+peering, 20 unknown, 698 active+clean; 145 MiB data, 768 MiB used, 714 GiB / 715 GiB avail; 8.7 KiB/s rd, 3 op/s 2024-06-11T13:31:19.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:19.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[40588]: osdmap e88: 8 total, 8 up, 8 in 2024-06-11T13:31:19.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2747983990' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-06-11T13:31:19.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:31:19.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[40588]: threexx 2024-06-11T13:31:19.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[40588]: from='client.35402 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi057-77874-12"}]: dispatch 2024-06-11T13:31:19.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[40588]: from='client.35348 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-06-11T13:31:19.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[40588]: from='client.25279 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi057-77506-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi057-77506-10"}]: dispatch 2024-06-11T13:31:19.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[40588]: from='client.25108 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-4", "tierpool": "test-rados-api-smithi057-77599-4-cache"}]: dispatch 2024-06-11T13:31:19.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[40588]: from='client.35363 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:19.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[40588]: from='client.25297 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi057-77831-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:19.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[40588]: from='client.25303 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi057-77785-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:19.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[40588]: from='client.24901 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi057-78571-1"}]': finished 2024-06-11T13:31:19.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2607108900' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi057-77831-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi057-77831-7"}]: dispatch 2024-06-11T13:31:19.039 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3479760694' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi057-77785-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi057-77785-7"}]: dispatch 2024-06-11T13:31:19.039 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3799830007' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi057-77874-12"}]: dispatch 2024-06-11T13:31:19.039 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/995766982' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:19.039 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2501793636' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi057-78571-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:19.039 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/197523128' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi057-77882-6", "pg_num": 4}]: dispatch 2024-06-11T13:31:19.039 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:19.039 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[40588]: osdmap e89: 8 total, 8 up, 8 in 2024-06-11T13:31:19.039 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[40588]: from='client.35402 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi057-77874-12"}]: dispatch 2024-06-11T13:31:19.039 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[40588]: from='client.35399 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:19.039 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[40588]: from='client.35414 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi057-78571-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:19.039 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3799830007' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi057-77874-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:19.039 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:18 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:31:19.080 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ RUN ] api_service_pp: }, 2024-06-11T13:31:19.080 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "task_status": {} 2024-06-11T13:31:19.080 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: } 2024-06-11T13:31:19.081 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: } 2024-06-11T13:31:19.081 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: } 2024-06-11T13:31:19.081 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: } 2024-06-11T13:31:19.081 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: } 2024-06-11T13:31:19.081 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: 2024-06-11T13:31:19.081 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: attempt 1 of 20 2024-06-11T13:31:19.081 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: failed to deregister: 2024-06-11T13:31:19.081 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: { 2024-06-11T13:31:19.081 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "epoch": 20, 2024-06-11T13:31:19.081 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "modified": "2024-06-11T13:31:16.002115+0000", 2024-06-11T13:31:19.081 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "services": { 2024-06-11T13:31:19.081 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "laundry": { 2024-06-11T13:31:19.081 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "daemons": { 2024-06-11T13:31:19.082 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "summary": "", 2024-06-11T13:31:19.082 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "close-test-pid78049": { 2024-06-11T13:31:19.082 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "start_epoch": 20, 2024-06-11T13:31:19.082 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "start_stamp": "2024-06-11T13:31:14.052157+0000", 2024-06-11T13:31:19.082 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "gid": 35375, 2024-06-11T13:31:19.082 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "addr": "172.21.15.57:0/4087307784", 2024-06-11T13:31:19.082 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "metadata": { 2024-06-11T13:31:19.082 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "arch": "x86_64", 2024-06-11T13:31:19.082 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "ceph_release": "squid", 2024-06-11T13:31:19.082 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "ceph_version": "ceph version 19.0.0-2976-g3f51ea39 (3f51ea3944d39ca13aa40153c9a4ee298306679d) squid (rc)", 2024-06-11T13:31:19.082 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "ceph_version_short": "19.0.0-2976-g3f51ea39", 2024-06-11T13:31:19.082 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2024-06-11T13:31:19.083 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "distro": "centos", 2024-06-11T13:31:19.083 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2024-06-11T13:31:19.083 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "distro_version": "9", 2024-06-11T13:31:19.083 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "foo": "bar", 2024-06-11T13:31:19.083 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "hostname": "smithi057", 2024-06-11T13:31:19.083 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024", 2024-06-11T13:31:19.083 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "kernel_version": "5.14.0-452.el9.x86_64", 2024-06-11T13:31:19.083 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "mem_swap_kb": "0", 2024-06-11T13:31:19.083 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "mem_total_kb": "32494356", 2024-06-11T13:31:19.083 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "os": "Linux", 2024-06-11T13:31:19.083 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "this": "that" 2024-06-11T13:31:19.083 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: }, 2024-06-11T13:31:19.083 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "task_status": {} 2024-06-11T13:31:19.083 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: }, 2024-06-11T13:31:19.084 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "pid77983": { 2024-06-11T13:31:19.084 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "start_epoch": 17, 2024-06-11T13:31:19.084 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "start_stamp": "2024-06-11T13:31:01.334683+0000", 2024-06-11T13:31:19.084 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "gid": 25120, 2024-06-11T13:31:19.084 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "addr": "172.21.15.57:0/4124237174", 2024-06-11T13:31:19.084 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "metadata": { 2024-06-11T13:31:19.084 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "arch": "x86_64", 2024-06-11T13:31:19.084 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "ceph_release": "squid", 2024-06-11T13:31:19.084 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "ceph_version": "ceph version 19.0.0-2976-g3f51ea39 (3f51ea3944d39ca13aa40153c9a4ee298306679d) squid (rc)", 2024-06-11T13:31:19.084 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "ceph_version_short": "19.0.0-2976-g3f51ea39", 2024-06-11T13:31:19.084 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2024-06-11T13:31:19.084 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "distro": "centos", 2024-06-11T13:31:19.084 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2024-06-11T13:31:19.084 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "distro_version": "9", 2024-06-11T13:31:19.085 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "foo": "bar", 2024-06-11T13:31:19.085 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "hostname": "smithi057", 2024-06-11T13:31:21.339 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: ce: id: 98a6635e-27f5-11ef-bc9b-c7b262605968 2024-06-11T13:31:21.339 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: health: HEALTH_WARN 2024-06-11T13:31:21.339 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 26 pool(s) do not have an application enabled 2024-06-11T13:31:21.340 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 2024-06-11T13:31:21.340 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: services: 2024-06-11T13:31:21.340 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-06-11T13:31:21.340 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: mgr: y(active, since 2m), standbys: x 2024-06-11T13:31:21.340 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: osd: 8 osds: 8 up (since 3m), 8 in (since 3m) 2024-06-11T13:31:21.340 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: foo: 16 portals active (1 hosts, 3 zones) 2024-06-11T13:31:21.340 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: laundry: 1 daemon active (1 hosts) 2024-06-11T13:31:21.340 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-06-11T13:31:21.340 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 2024-06-11T13:31:21.340 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: data: 2024-06-11T13:31:21.340 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: pools: 40 pools, 1220 pgs 2024-06-11T13:31:21.340 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: objects: 198 objects, 583 KiB 2024-06-11T13:31:21.340 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: usage: 247 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:31:21.340 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: pgs: 39.590% pgs unknown 2024-06-11T13:31:21.341 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 49.590% pgs not active 2024-06-11T13:31:21.341 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 605 creating+peering 2024-06-11T13:31:21.341 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 483 unknown 2024-06-11T13:31:21.341 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 132 active+clean 2024-06-11T13:31:21.341 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 2024-06-11T13:31:21.341 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: io: 2024-06-11T13:31:21.341 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: client: 639 B/s rd, 0 op/s rd, 0 op/s wr 2024-06-11T13:31:21.341 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 2024-06-11T13:31:21.341 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 2024-06-11T13:31:21.341 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: [ OK ] LibRadosService.StatusFormat (12384 ms) 2024-06-11T13:31:21.341 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: [ RUN ] LibRadosService.Status 2024-06-11T13:31:21.341 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: [ OK ] LibRadosService.Status (20020 ms) 2024-06-11T13:31:21.341 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: [----------] 4 tests from LibRadosService (37476 ms total) 2024-06-11T13:31:21.342 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: 2024-06-11T13:31:21.342 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: [----------] Global test environment tear-down 2024-06-11T13:31:21.342 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: [==========] 4 tests from 1 test suite ran. (37477 ms total) 2024-06-11T13:31:21.342 INFO:tasks.workunit.client.0.smithi057.stdout: api_service: [ PASSED ] 4 tests. 2024-06-11T13:31:21.547 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:31:21 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:31:22.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[29805]: pgmap v96: 868 pgs: 86 creating+peering, 116 unknown, 666 active+clean; 145 MiB data, 768 MiB used, 714 GiB / 715 GiB avail; 7.7 KiB/s rd, 3 op/s 2024-06-11T13:31:22.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:22.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[29805]: Health check update: 13 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:31:22.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:31:22.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[29805]: from='client.35363 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi057-77882-6", "pg_num": 4}]: dispatch 2024-06-11T13:31:22.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[29805]: from='client.35402 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi057-77874-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:22.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[29805]: from='client.25297 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi057-77831-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi057-77831-7"}]: dispatch 2024-06-11T13:31:22.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[29805]: from='client.25303 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi057-77785-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi057-77785-7"}]: dispatch 2024-06-11T13:31:22.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[29805]: from='client.25108 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-4", "tierpool": "test-rados-api-smithi057-77599-4-cache"}]': finished 2024-06-11T13:31:22.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[29805]: from='client.35399 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:22.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[29805]: from='client.35414 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi057-78571-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:22.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[29805]: from='client.35348 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-06-11T13:31:22.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2245769429' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-4", "overlaypool": "test-rados-api-smithi057-77599-4-cache"}]: dispatch 2024-06-11T13:31:22.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2501793636' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi057-78571-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi057-78571-2"}]: dispatch 2024-06-11T13:31:22.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2747983990' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-06-11T13:31:22.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[29805]: fourxx 2024-06-11T13:31:22.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2571911560' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi057-78504-2"}]: dispatch 2024-06-11T13:31:22.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2147982944' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-06-11T13:31:22.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[29805]: from='mon.? v1:172.21.15.57:0/45435016' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-06-11T13:31:22.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:22.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[29805]: osdmap e90: 8 total, 8 up, 8 in 2024-06-11T13:31:22.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[40588]: pgmap v96: 868 pgs: 86 creating+peering, 116 unknown, 666 active+clean; 145 MiB data, 768 MiB used, 714 GiB / 715 GiB avail; 7.7 KiB/s rd, 3 op/s 2024-06-11T13:31:22.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:22.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[40588]: Health check update: 13 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:31:22.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:31:22.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[40588]: from='client.35363 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi057-77882-6", "pg_num": 4}]: dispatch 2024-06-11T13:31:22.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[40588]: from='client.35402 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi057-77874-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:22.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[40588]: from='client.25297 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi057-77831-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi057-77831-7"}]: dispatch 2024-06-11T13:31:22.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[40588]: from='client.25303 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi057-77785-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi057-77785-7"}]: dispatch 2024-06-11T13:31:22.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[40588]: from='client.25108 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-4", "tierpool": "test-rados-api-smithi057-77599-4-cache"}]': finished 2024-06-11T13:31:22.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[40588]: from='client.35399 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:22.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[40588]: from='client.35414 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi057-78571-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:22.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[40588]: from='client.35348 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-06-11T13:31:22.538 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2245769429' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-4", "overlaypool": "test-rados-api-smithi057-77599-4-cache"}]: dispatch 2024-06-11T13:31:22.538 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2501793636' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi057-78571-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi057-78571-2"}]: dispatch 2024-06-11T13:31:22.538 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2747983990' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-06-11T13:31:22.538 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[40588]: fourxx 2024-06-11T13:31:22.538 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2571911560' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi057-78504-2"}]: dispatch 2024-06-11T13:31:22.538 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2147982944' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-06-11T13:31:22.538 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[40588]: from='mon.? v1:172.21.15.57:0/45435016' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-06-11T13:31:22.538 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:22.539 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[40588]: osdmap e90: 8 total, 8 up, 8 in 2024-06-11T13:31:22.539 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[40588]: from='client.35414 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi057-78571-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi057-78571-2"}]: dispatch 2024-06-11T13:31:22.539 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[40588]: from='client.35348 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-06-11T13:31:22.539 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[40588]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi057-78504-2"}]: dispatch 2024-06-11T13:31:22.539 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[40588]: from='client.25108 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-4", "overlaypool": "test-rados-api-smithi057-77599-4-cache"}]: dispatch 2024-06-11T13:31:22.539 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:31:22.539 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[40588]: from='client.25279 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi057-77506-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi057-77506-10"}]': finished 2024-06-11T13:31:22.539 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[29805]: from='client.35414 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi057-78571-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi057-78571-2"}]: dispatch 2024-06-11T13:31:22.539 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[29805]: from='client.35348 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-06-11T13:31:22.539 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[29805]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi057-78504-2"}]: dispatch 2024-06-11T13:31:22.540 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[29805]: from='client.25108 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-4", "overlaypool": "test-rados-api-smithi057-77599-4-cache"}]: dispatch 2024-06-11T13:31:22.540 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:31:22.540 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:22 smithi057 ceph-mon[29805]: from='client.25279 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi057-77506-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi057-77506-10"}]': finished 2024-06-11T13:31:22.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:22 smithi107 ceph-mon[33178]: pgmap v96: 868 pgs: 86 creating+peering, 116 unknown, 666 active+clean; 145 MiB data, 768 MiB used, 714 GiB / 715 GiB avail; 7.7 KiB/s rd, 3 op/s 2024-06-11T13:31:22.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:22 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:22.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:22 smithi107 ceph-mon[33178]: Health check update: 13 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:31:22.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:22 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:31:22.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:22 smithi107 ceph-mon[33178]: from='client.35363 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi057-77882-6", "pg_num": 4}]: dispatch 2024-06-11T13:31:22.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:22 smithi107 ceph-mon[33178]: from='client.35402 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi057-77874-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:22.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:22 smithi107 ceph-mon[33178]: from='client.25297 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi057-77831-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi057-77831-7"}]: dispatch 2024-06-11T13:31:22.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:22 smithi107 ceph-mon[33178]: from='client.25303 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi057-77785-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi057-77785-7"}]: dispatch 2024-06-11T13:31:22.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:22 smithi107 ceph-mon[33178]: from='client.25108 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-4", "tierpool": "test-rados-api-smithi057-77599-4-cache"}]': finished 2024-06-11T13:31:22.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:22 smithi107 ceph-mon[33178]: from='client.35399 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:22.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:22 smithi107 ceph-mon[33178]: from='client.35414 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi057-78571-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:22.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:22 smithi107 ceph-mon[33178]: from='client.35348 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-06-11T13:31:22.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:22 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2245769429' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-4", "overlaypool": "test-rados-api-smithi057-77599-4-cache"}]: dispatch 2024-06-11T13:31:22.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:22 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2501793636' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi057-78571-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi057-78571-2"}]: dispatch 2024-06-11T13:31:22.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:22 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2747983990' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-06-11T13:31:22.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:22 smithi107 ceph-mon[33178]: fourxx 2024-06-11T13:31:22.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:22 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2571911560' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi057-78504-2"}]: dispatch 2024-06-11T13:31:22.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:22 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2147982944' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-06-11T13:31:22.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:22 smithi107 ceph-mon[33178]: from='mon.? v1:172.21.15.57:0/45435016' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-06-11T13:31:22.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:22 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:22.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:22 smithi107 ceph-mon[33178]: osdmap e90: 8 total, 8 up, 8 in 2024-06-11T13:31:22.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:22 smithi107 ceph-mon[33178]: from='client.35414 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi057-78571-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi057-78571-2"}]: dispatch 2024-06-11T13:31:22.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:22 smithi107 ceph-mon[33178]: from='client.35348 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-06-11T13:31:22.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:22 smithi107 ceph-mon[33178]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi057-78504-2"}]: dispatch 2024-06-11T13:31:22.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:22 smithi107 ceph-mon[33178]: from='client.25108 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-4", "overlaypool": "test-rados-api-smithi057-77599-4-cache"}]: dispatch 2024-06-11T13:31:22.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:22 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:31:22.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:22 smithi107 ceph-mon[33178]: from='client.25279 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi057-77506-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi057-77506-10"}]': finished 2024-06-11T13:31:22.689 INFO:tasks.workunit.client.0.smithi057.stdout: list: Running main() from gmock_main.cc 2024-06-11T13:31:22.689 INFO:tasks.workunit.client.0.smithi057.stdout: list: [==========] Running 3 tests from 1 test suite. 2024-06-11T13:31:22.689 INFO:tasks.workunit.client.0.smithi057.stdout: list: [----------] Global test environment set-up. 2024-06-11T13:31:22.689 INFO:tasks.workunit.client.0.smithi057.stdout: list: [----------] 3 tests from NeoradosList 2024-06-11T13:31:22.689 INFO:tasks.workunit.client.0.smithi057.stdout: list: [ RUN ] NeoradosList.ListObjects 2024-06-11T13:31:22.689 INFO:tasks.workunit.client.0.smithi057.stdout: list: [ OK ] NeoradosList.ListObjects (17670 ms) 2024-06-11T13:31:22.689 INFO:tasks.workunit.client.0.smithi057.stdout: list: [ RUN ] NeoradosList.ListObjectsNS 2024-06-11T13:31:22.690 INFO:tasks.workunit.client.0.smithi057.stdout: list: [ OK ] NeoradosList.ListObjectsNS (8956 ms) 2024-06-11T13:31:22.690 INFO:tasks.workunit.client.0.smithi057.stdout: list: [ RUN ] NeoradosList.ListObjectsMany 2024-06-11T13:31:22.690 INFO:tasks.workunit.client.0.smithi057.stdout: list: [ OK ] NeoradosList.ListObjectsMany (12023 ms) 2024-06-11T13:31:22.690 INFO:tasks.workunit.client.0.smithi057.stdout: list: [----------] 3 tests from NeoradosList (38649 ms total) 2024-06-11T13:31:22.690 INFO:tasks.workunit.client.0.smithi057.stdout: list: 2024-06-11T13:31:22.690 INFO:tasks.workunit.client.0.smithi057.stdout: list: [----------] Global test environment tear-down 2024-06-11T13:31:22.690 INFO:tasks.workunit.client.0.smithi057.stdout: list: [==========] 3 tests from 1 test suite ran. (38649 ms total) 2024-06-11T13:31:22.690 INFO:tasks.workunit.client.0.smithi057.stdout: list: [ PASSED ] 3 tests. 2024-06-11T13:31:23.247 INFO:tasks.workunit.client.0.smithi057.stdout: ec_io: Running main() from gmock_main.cc 2024-06-11T13:31:23.247 INFO:tasks.workunit.client.0.smithi057.stdout: ec_io: [==========] Running 2 tests from 1 test suite. 2024-06-11T13:31:23.247 INFO:tasks.workunit.client.0.smithi057.stdout: ec_io: [----------] Global test environment set-up. 2024-06-11T13:31:23.247 INFO:tasks.workunit.client.0.smithi057.stdout: ec_io: [----------] 2 tests from NeoRadosECIo 2024-06-11T13:31:23.247 INFO:tasks.workunit.client.0.smithi057.stdout: ec_io: [ RUN ] NeoRadosECIo.SimpleWrite 2024-06-11T13:31:23.248 INFO:tasks.workunit.client.0.smithi057.stdout: ec_io: [ OK ] NeoRadosECIo.SimpleWrite (23877 ms) 2024-06-11T13:31:23.248 INFO:tasks.workunit.client.0.smithi057.stdout: ec_io: [ RUN ] NeoRadosECIo.ReadOp 2024-06-11T13:31:23.248 INFO:tasks.workunit.client.0.smithi057.stdout: ec_io: [ OK ] NeoRadosECIo.ReadOp (15334 ms) 2024-06-11T13:31:23.248 INFO:tasks.workunit.client.0.smithi057.stdout: ec_io: [----------] 2 tests from NeoRadosECIo (39211 ms total) 2024-06-11T13:31:23.248 INFO:tasks.workunit.client.0.smithi057.stdout: ec_io: 2024-06-11T13:31:23.248 INFO:tasks.workunit.client.0.smithi057.stdout: ec_io: [----------] Global test environment tear-down 2024-06-11T13:31:23.248 INFO:tasks.workunit.client.0.smithi057.stdout: ec_io: [==========] 2 tests from 1 test suite ran. (39211 ms total) 2024-06-11T13:31:23.248 INFO:tasks.workunit.client.0.smithi057.stdout: ec_io: [ PASSED ] 2 tests. 2024-06-11T13:31:23.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[29805]: from='client.35363 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi057-77882-6", "pg_num": 4}]': finished 2024-06-11T13:31:23.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[29805]: from='client.35402 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi057-77874-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:23.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3799830007' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi057-77874-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi057-77874-12"}]: dispatch 2024-06-11T13:31:23.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/197523128' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4", "tierpool": "test-rados-api-smithi057-77882-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:31:23.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[29805]: pgmap v99: 872 pgs: 85 creating+peering, 224 unknown, 563 active+clean; 145 MiB data, 768 MiB used, 714 GiB / 715 GiB avail; 970 B/s rd, 3.8 KiB/s wr, 1 op/s 2024-06-11T13:31:23.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:23.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:31:23.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:31:23.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[29805]: pgmap v100: 872 pgs: 85 creating+peering, 224 unknown, 563 active+clean; 145 MiB data, 768 MiB used, 714 GiB / 715 GiB avail 2024-06-11T13:31:23.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:31:23.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:31:23.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:23.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[29805]: osdmap e91: 8 total, 8 up, 8 in 2024-06-11T13:31:23.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[29805]: pool 'PoolQuotaPP_smithi057-77362-3' is full (reached quota's max_bytes: 4 KiB) 2024-06-11T13:31:23.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[29805]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-06-11T13:31:23.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[29805]: from='client.35402 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi057-77874-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi057-77874-12"}]: dispatch 2024-06-11T13:31:23.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[29805]: from='client.35363 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4", "tierpool": "test-rados-api-smithi057-77882-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:31:23.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[29805]: from='client.35348 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-06-11T13:31:23.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[29805]: from='client.25297 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi057-77831-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi057-77831-7"}]': finished 2024-06-11T13:31:23.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[40588]: from='client.35363 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi057-77882-6", "pg_num": 4}]': finished 2024-06-11T13:31:23.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[40588]: from='client.35402 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi057-77874-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:23.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3799830007' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi057-77874-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi057-77874-12"}]: dispatch 2024-06-11T13:31:23.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/197523128' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4", "tierpool": "test-rados-api-smithi057-77882-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:31:23.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[40588]: pgmap v99: 872 pgs: 85 creating+peering, 224 unknown, 563 active+clean; 145 MiB data, 768 MiB used, 714 GiB / 715 GiB avail; 970 B/s rd, 3.8 KiB/s wr, 1 op/s 2024-06-11T13:31:23.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:23.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:31:23.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:31:23.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[40588]: pgmap v100: 872 pgs: 85 creating+peering, 224 unknown, 563 active+clean; 145 MiB data, 768 MiB used, 714 GiB / 715 GiB avail 2024-06-11T13:31:23.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:31:23.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:31:23.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:23.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[40588]: osdmap e91: 8 total, 8 up, 8 in 2024-06-11T13:31:23.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[40588]: pool 'PoolQuotaPP_smithi057-77362-3' is full (reached quota's max_bytes: 4 KiB) 2024-06-11T13:31:23.288 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[40588]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-06-11T13:31:23.288 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[40588]: from='client.35402 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi057-77874-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi057-77874-12"}]: dispatch 2024-06-11T13:31:23.288 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[40588]: from='client.35363 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4", "tierpool": "test-rados-api-smithi057-77882-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:31:23.288 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[40588]: from='client.35348 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-06-11T13:31:23.288 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[40588]: from='client.25297 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi057-77831-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi057-77831-7"}]': finished 2024-06-11T13:31:23.288 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[29805]: from='client.25303 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi057-77785-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi057-77785-7"}]': finished 2024-06-11T13:31:23.288 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[29805]: from='client.35318 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi057-78504-2"}]': finished 2024-06-11T13:31:23.288 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[29805]: from='client.25108 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-4", "overlaypool": "test-rados-api-smithi057-77599-4-cache"}]': finished 2024-06-11T13:31:23.288 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2245769429' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-4-cache", "mode": "writeback"}]: dispatch 2024-06-11T13:31:23.288 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2571911560' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi057-78504-2"}]: dispatch 2024-06-11T13:31:23.289 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:23.289 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[29805]: pgmap v102: 904 pgs: 123 creating+peering, 129 unknown, 652 active+clean; 145 MiB data, 777 MiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 6 op/s 2024-06-11T13:31:23.289 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[29805]: osdmap e92: 8 total, 8 up, 8 in 2024-06-11T13:31:23.289 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2823879459' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi057-77351-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:23.289 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2132302726' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:23.289 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1420997228' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi057-77374-16"}]: dispatch 2024-06-11T13:31:23.289 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1129714156' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi057-77396-22"}]: dispatch 2024-06-11T13:31:23.289 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:31:23.289 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[29805]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi057-78504-2"}]: dispatch 2024-06-11T13:31:23.289 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[29805]: from='client.35423 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:23.290 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[29805]: from='client.35312 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi057-77374-16"}]: dispatch 2024-06-11T13:31:23.290 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[29805]: from='client.35306 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi057-77396-22"}]: dispatch 2024-06-11T13:31:23.290 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[29805]: from='client.25108 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-4-cache", "mode": "writeback"}]: dispatch 2024-06-11T13:31:23.290 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[29805]: from='client.25438 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi057-77351-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:23.290 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[40588]: from='client.25303 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi057-77785-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi057-77785-7"}]': finished 2024-06-11T13:31:23.290 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[40588]: from='client.35318 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi057-78504-2"}]': finished 2024-06-11T13:31:23.290 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[40588]: from='client.25108 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-4", "overlaypool": "test-rados-api-smithi057-77599-4-cache"}]': finished 2024-06-11T13:31:23.290 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2245769429' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-4-cache", "mode": "writeback"}]: dispatch 2024-06-11T13:31:23.290 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2571911560' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi057-78504-2"}]: dispatch 2024-06-11T13:31:23.291 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:23.291 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[40588]: pgmap v102: 904 pgs: 123 creating+peering, 129 unknown, 652 active+clean; 145 MiB data, 777 MiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 6 op/s 2024-06-11T13:31:23.291 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[40588]: osdmap e92: 8 total, 8 up, 8 in 2024-06-11T13:31:23.291 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2823879459' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi057-77351-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:23.291 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2132302726' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:23.291 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1420997228' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi057-77374-16"}]: dispatch 2024-06-11T13:31:23.291 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1129714156' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi057-77396-22"}]: dispatch 2024-06-11T13:31:23.291 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:31:23.291 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[40588]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi057-78504-2"}]: dispatch 2024-06-11T13:31:23.291 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[40588]: from='client.35423 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:23.291 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[40588]: from='client.35312 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi057-77374-16"}]: dispatch 2024-06-11T13:31:23.291 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[40588]: from='client.35306 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi057-77396-22"}]: dispatch 2024-06-11T13:31:23.291 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[40588]: from='client.25108 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-4-cache", "mode": "writeback"}]: dispatch 2024-06-11T13:31:23.291 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-mon[40588]: from='client.25438 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi057-77351-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:23.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:23 smithi107 ceph-mon[33178]: from='client.35363 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi057-77882-6", "pg_num": 4}]': finished 2024-06-11T13:31:23.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:23 smithi107 ceph-mon[33178]: from='client.35402 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi057-77874-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:23.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:23 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3799830007' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi057-77874-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi057-77874-12"}]: dispatch 2024-06-11T13:31:23.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:23 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/197523128' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4", "tierpool": "test-rados-api-smithi057-77882-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:31:23.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:23 smithi107 ceph-mon[33178]: pgmap v99: 872 pgs: 85 creating+peering, 224 unknown, 563 active+clean; 145 MiB data, 768 MiB used, 714 GiB / 715 GiB avail; 970 B/s rd, 3.8 KiB/s wr, 1 op/s 2024-06-11T13:31:23.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:23 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:23.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:23 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:31:23.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:23 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:31:23.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:23 smithi107 ceph-mon[33178]: pgmap v100: 872 pgs: 85 creating+peering, 224 unknown, 563 active+clean; 145 MiB data, 768 MiB used, 714 GiB / 715 GiB avail 2024-06-11T13:31:23.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:23 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:31:23.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:23 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:31:23.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:23 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:23.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:23 smithi107 ceph-mon[33178]: osdmap e91: 8 total, 8 up, 8 in 2024-06-11T13:31:23.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:23 smithi107 ceph-mon[33178]: pool 'PoolQuotaPP_smithi057-77362-3' is full (reached quota's max_bytes: 4 KiB) 2024-06-11T13:31:23.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:23 smithi107 ceph-mon[33178]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-06-11T13:31:23.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:23 smithi107 ceph-mon[33178]: from='client.35402 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi057-77874-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi057-77874-12"}]: dispatch 2024-06-11T13:31:23.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:23 smithi107 ceph-mon[33178]: from='client.35363 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4", "tierpool": "test-rados-api-smithi057-77882-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:31:23.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:23 smithi107 ceph-mon[33178]: from='client.35348 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-06-11T13:31:23.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:23 smithi107 ceph-mon[33178]: from='client.25297 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi057-77831-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi057-77831-7"}]': finished 2024-06-11T13:31:23.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:23 smithi107 ceph-mon[33178]: from='client.25303 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi057-77785-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi057-77785-7"}]': finished 2024-06-11T13:31:23.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:23 smithi107 ceph-mon[33178]: from='client.35318 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi057-78504-2"}]': finished 2024-06-11T13:31:23.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:23 smithi107 ceph-mon[33178]: from='client.25108 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-4", "overlaypool": "test-rados-api-smithi057-77599-4-cache"}]': finished 2024-06-11T13:31:23.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:23 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2245769429' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-4-cache", "mode": "writeback"}]: dispatch 2024-06-11T13:31:23.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:23 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2571911560' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi057-78504-2"}]: dispatch 2024-06-11T13:31:23.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:23 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:23.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:23 smithi107 ceph-mon[33178]: pgmap v102: 904 pgs: 123 creating+peering, 129 unknown, 652 active+clean; 145 MiB data, 777 MiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 6 op/s 2024-06-11T13:31:23.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:23 smithi107 ceph-mon[33178]: osdmap e92: 8 total, 8 up, 8 in 2024-06-11T13:31:23.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:23 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2823879459' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi057-77351-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:23.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:23 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2132302726' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:23.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:23 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1420997228' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi057-77374-16"}]: dispatch 2024-06-11T13:31:23.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:23 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1129714156' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi057-77396-22"}]: dispatch 2024-06-11T13:31:23.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:23 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:31:23.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:23 smithi107 ceph-mon[33178]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi057-78504-2"}]: dispatch 2024-06-11T13:31:23.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:23 smithi107 ceph-mon[33178]: from='client.35423 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:23.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:23 smithi107 ceph-mon[33178]: from='client.35312 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi057-77374-16"}]: dispatch 2024-06-11T13:31:23.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:23 smithi107 ceph-mon[33178]: from='client.35306 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi057-77396-22"}]: dispatch 2024-06-11T13:31:23.551 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:23 smithi107 ceph-mon[33178]: from='client.25108 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-4-cache", "mode": "writeback"}]: dispatch 2024-06-11T13:31:23.551 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:23 smithi107 ceph-mon[33178]: from='client.25438 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi057-77351-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:23.731 INFO:tasks.workunit.client.0.smithi057.stdout: cmd: Running main() from gmock_main.cc 2024-06-11T13:31:23.731 INFO:tasks.workunit.client.0.smithi057.stdout: cmd: [==========] Running 3 tests from 1 test suite. 2024-06-11T13:31:23.731 INFO:tasks.workunit.client.0.smithi057.stdout: cmd: [----------] Global test environment set-up. 2024-06-11T13:31:23.731 INFO:tasks.workunit.client.0.smithi057.stdout: cmd: [----------] 3 tests from NeoRadosCmd 2024-06-11T13:31:23.731 INFO:tasks.workunit.client.0.smithi057.stdout: cmd: [ RUN ] NeoRadosCmd.MonDescribe 2024-06-11T13:31:23.732 INFO:tasks.workunit.client.0.smithi057.stdout: cmd: [ OK ] NeoRadosCmd.MonDescribe (22498 ms) 2024-06-11T13:31:23.732 INFO:tasks.workunit.client.0.smithi057.stdout: cmd: [ RUN ] NeoRadosCmd.OSDCmd 2024-06-11T13:31:23.732 INFO:tasks.workunit.client.0.smithi057.stdout: cmd: [ OK ] NeoRadosCmd.OSDCmd (8821 ms) 2024-06-11T13:31:23.732 INFO:tasks.workunit.client.0.smithi057.stdout: cmd: [ RUN ] NeoRadosCmd.PGCmd 2024-06-11T13:31:23.732 INFO:tasks.workunit.client.0.smithi057.stdout: cmd: [ OK ] NeoRadosCmd.PGCmd (8408 ms) 2024-06-11T13:31:23.732 INFO:tasks.workunit.client.0.smithi057.stdout: cmd: [----------] 3 tests from NeoRadosCmd (39727 ms total) 2024-06-11T13:31:23.732 INFO:tasks.workunit.client.0.smithi057.stdout: cmd: 2024-06-11T13:31:23.732 INFO:tasks.workunit.client.0.smithi057.stdout: cmd: [----------] Global test environment tear-down 2024-06-11T13:31:23.733 INFO:tasks.workunit.client.0.smithi057.stdout: cmd: [==========] 3 tests from 1 test suite ran. (39727 ms total) 2024-06-11T13:31:23.733 INFO:tasks.workunit.client.0.smithi057.stdout: cmd: [ PASSED ] 3 tests. 2024-06-11T13:31:24.033 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:31:23 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:31:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:31:24.131 INFO:tasks.workunit.client.0.smithi057.stdout: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024", 2024-06-11T13:31:24.135 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "kernel_version": "5.14.0-452.el9.x86_64", 2024-06-11T13:31:24.135 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "mem_swap_kb": "0", 2024-06-11T13:31:24.135 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "mem_total_kb": "32494356", 2024-06-11T13:31:24.136 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "os": "Linux", 2024-06-11T13:31:24.136 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "this": "that" 2024-06-11T13:31:24.137 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: }, 2024-06-11T13:31:24.137 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "task_status": {} 2024-06-11T13:31:24.137 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: } 2024-06-11T13:31:24.138 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: } 2024-06-11T13:31:24.138 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: }, 2024-06-11T13:31:24.138 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "rgw": { 2024-06-11T13:31:24.138 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "daemons": { 2024-06-11T13:31:24.138 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "summary": "", 2024-06-11T13:31:24.138 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "24379": { 2024-06-11T13:31:24.139 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "start_epoch": 12, 2024-06-11T13:31:24.139 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "start_stamp": "2024-06-11T13:28:20.560327+0000", 2024-06-11T13:31:24.139 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "gid": 24379, 2024-06-11T13:31:24.139 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "addr": "172.21.15.57:0/2628526915", 2024-06-11T13:31:24.139 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "metadata": { 2024-06-11T13:31:24.139 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "arch": "x86_64", 2024-06-11T13:31:24.140 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "ceph_release": "squid", 2024-06-11T13:31:24.140 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "ceph_version": "ceph version 19.0.0-2976-g3f51ea39 (3f51ea3944d39ca13aa40153c9a4ee298306679d) squid (rc)", 2024-06-11T13:31:24.140 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "ceph_version_short": "19.0.0-2976-g3f51ea39", 2024-06-11T13:31:24.140 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "container_hostname": "smithi057", 2024-06-11T13:31:24.140 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "container_image": "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310", 2024-06-11T13:31:24.140 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2024-06-11T13:31:24.141 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "distro": "centos", 2024-06-11T13:31:24.141 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2024-06-11T13:31:24.141 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "distro_version": "9", 2024-06-11T13:31:24.141 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "frontend_config#0": "beast port=80", 2024-06-11T13:31:24.141 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "frontend_type#0": "beast", 2024-06-11T13:31:24.141 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "hostname": "smithi057", 2024-06-11T13:31:24.142 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "id": "foo.a", 2024-06-11T13:31:24.142 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Sat May 18 20:39:48 UTC 2024", 2024-06-11T13:31:24.142 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "kernel_version": "5.14.0-452.el9.x86_64", 2024-06-11T13:31:24.142 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "mem_swap_kb": "0", 2024-06-11T13:31:24.142 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "mem_total_kb": "32494356", 2024-06-11T13:31:24.142 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "num_handles": "1", 2024-06-11T13:31:24.143 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "os": "Linux", 2024-06-11T13:31:24.143 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "pid": "2", 2024-06-11T13:31:24.143 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "realm_id": "", 2024-06-11T13:31:24.143 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "realm_name": "", 2024-06-11T13:31:24.143 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "zone_id": "f480d397-ad70-4c16-844a-5ad58a2b8f31", 2024-06-11T13:31:24.143 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "zone_name": "default", 2024-06-11T13:31:24.144 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "zonegroup_id": "f9ffc93c-24d2-4773-8bd8-bf937361fcfc", 2024-06-11T13:31:24.144 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "zonegroup_name": "default" 2024-06-11T13:31:24.144 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: }, 2024-06-11T13:31:24.144 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: "task_status": {} 2024-06-11T13:31:24.144 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: } 2024-06-11T13:31:24.144 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: } 2024-06-11T13:31:24.144 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: } 2024-06-11T13:31:24.145 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: } 2024-06-11T13:31:24.145 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: } 2024-06-11T13:31:24.145 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: 2024-06-11T13:31:24.145 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: attempt 2 of 20 2024-06-11T13:31:24.146 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: [ OK ] LibRadosServicePP.Close (15132 ms) 2024-06-11T13:31:24.146 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP (40258 ms total) 2024-06-11T13:31:24.146 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: 2024-06-11T13:31:24.147 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: [----------] Global test environment tear-down 2024-06-11T13:31:24.147 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: [==========] 4 tests from 1 test suite ran. (40258 ms total) 2024-06-11T13:31:24.147 INFO:tasks.workunit.client.0.smithi057.stdout: api_service_pp: [ PASSED ] 4 tests. 2024-06-11T13:31:24.219 INFO:tasks.workunit.client.0.smithi057.stdout:ECPP_smithi057-77831-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:24.219 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:17.146593+0000 mon.a [INF] from='client.25303 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi057-77785-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:24.219 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:17.146639+0000 mon.a [INF] from='client.24901 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi057-78571-1"}]': finished 2024-06-11T13:31:24.219 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:17.152641+0000 mon.b [INF] from='client.? v1:172.21.15.57:0/2607108900' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi057-77831-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi057-77831-7"}]: dispatch 2024-06-11T13:31:24.220 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:17.152819+0000 mon.b [INF] from='client.? v1:172.21.15.57:0/3479760694' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi057-77785-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi057-77785-7"}]: dispatch 2024-06-11T13:31:24.220 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:17.164220+0000 mon.c [INF] from='client.? v1:172.21.15.57:0/3799830007' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi057-77874-12"}]: dispatch 2024-06-11T13:31:24.220 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:17.164786+0000 mon.c [INF] from='client.? v1:172.21.15.57:0/995766982' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:24.220 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:17.201447+0000 mon.c [INF] from='client.? v1:172.21.15.57:0/2501793636' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi057-78571-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:24.220 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:17.227780+0000 mon.c [INF] from='client.? v1:172.21.15.57:0/197523128' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi057-77882-6", "pg_num": 4}]: dispatch 2024-06-11T13:31:24.220 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:17.646891+0000 mon.a [INF] from='client.35402 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi057-77874-12"}]: dispatch 2024-06-11T13:31:24.220 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:17.647317+0000 mon.a [INF] from='client.35399 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:24.220 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:17.647945+0000 mon.a [INF] from='client.35414 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi057-78571-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:24.220 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:17.652542+0000 mon.c [INF] from='client.? v1:172.21.15.57:0/3799830007' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi057-77874-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:24.220 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: got: 2024-06-11T13:31:18.093907+0000 mon.a [INF] from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:31:24.220 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: [ OK ] LibRadosCmd.WatchLog (13549 ms) 2024-06-11T13:31:24.220 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: [----------] 4 tests from LibRadosCmd (40414 ms total) 2024-06-11T13:31:24.221 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: 2024-06-11T13:31:24.221 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: [----------] Global test environment tear-down 2024-06-11T13:31:24.221 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: [==========] 4 tests from 1 test suite ran. (40414 ms total) 2024-06-11T13:31:24.221 INFO:tasks.workunit.client.0.smithi057.stdout: api_cmd: [ PASSED ] 4 tests. 2024-06-11T13:31:24.665 INFO:tasks.workunit.client.0.smithi057.stdout: LibRadosIoECPP.OverlappingWriteRoundTripPP 2024-06-11T13:31:24.665 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ OK ] LibRadosIoECPP.OverlappingWriteRoundTripPP (6 ms) 2024-06-11T13:31:24.665 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP 2024-06-11T13:31:24.665 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP (6 ms) 2024-06-11T13:31:24.665 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP2 2024-06-11T13:31:24.665 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP2 (6 ms) 2024-06-11T13:31:24.665 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.AppendRoundTripPP 2024-06-11T13:31:24.665 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ OK ] LibRadosIoECPP.AppendRoundTripPP (8 ms) 2024-06-11T13:31:24.665 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.TruncTestPP 2024-06-11T13:31:24.665 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ OK ] LibRadosIoECPP.TruncTestPP (8 ms) 2024-06-11T13:31:24.666 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RemoveTestPP 2024-06-11T13:31:24.666 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RemoveTestPP (4 ms) 2024-06-11T13:31:24.666 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrsRoundTripPP 2024-06-11T13:31:24.666 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrsRoundTripPP (4 ms) 2024-06-11T13:31:24.666 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RmXattrPP 2024-06-11T13:31:24.666 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RmXattrPP (8 ms) 2024-06-11T13:31:24.666 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrListPP 2024-06-11T13:31:24.666 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrListPP (5 ms) 2024-06-11T13:31:24.666 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtPP 2024-06-11T13:31:24.666 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtPP (5 ms) 2024-06-11T13:31:24.667 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtDNEPP 2024-06-11T13:31:24.667 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtDNEPP (4 ms) 2024-06-11T13:31:24.667 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtMismatchPP 2024-06-11T13:31:24.667 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtMismatchPP (4 ms) 2024-06-11T13:31:24.667 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP (3891 ms total) 2024-06-11T13:31:24.667 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: 2024-06-11T13:31:24.667 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [----------] Global test environment tear-down 2024-06-11T13:31:24.667 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [==========] 37 tests from 2 test suites ran. (41091 ms total) 2024-06-11T13:31:24.667 INFO:tasks.workunit.client.0.smithi057.stdout: api_io_pp: [ PASSED ] 37 tests. 2024-06-11T13:31:24.667 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: Running main() from gmock_main.cc 2024-06-11T13:31:24.668 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [==========] Running 24 tests from 2 test suites. 2024-06-11T13:31:24.668 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [----------] Global test environment set-up. 2024-06-11T13:31:24.668 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [----------] 14 tests from LibRadosIo 2024-06-11T13:31:24.668 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ RUN ] LibRadosIo.SimpleWrite 2024-06-11T13:31:24.668 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ OK ] LibRadosIo.SimpleWrite (174 ms) 2024-06-11T13:31:24.668 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ RUN ] LibRadosIo.TooBig 2024-06-11T13:31:24.668 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ OK ] LibRadosIo.TooBig (0 ms) 2024-06-11T13:31:24.668 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ RUN ] LibRadosIo.ReadTimeout 2024-06-11T13:31:24.668 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: no timeout :/ 2024-06-11T13:31:24.668 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: no timeout :/ 2024-06-11T13:31:24.668 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: no timeout :/ 2024-06-11T13:31:24.668 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: no timeout :/ 2024-06-11T13:31:24.668 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: no timeout :/ 2024-06-11T13:31:24.669 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ OK ] LibRadosIo.ReadTimeout (30 ms) 2024-06-11T13:31:24.669 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ RUN ] LibRadosIo.RoundTrip 2024-06-11T13:31:24.669 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ OK ] LibRadosIo.RoundTrip (10 ms) 2024-06-11T13:31:24.669 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ RUN ] LibRadosIo.Checksum 2024-06-11T13:31:24.669 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ OK ] LibRadosIo.Checksum (5 ms) 2024-06-11T13:31:24.669 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ RUN ] LibRadosIo.OverlappingWriteRoundTrip 2024-06-11T13:31:24.669 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ OK ] LibRadosIo.OverlappingWriteRoundTrip (7 ms) 2024-06-11T13:31:24.669 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ RUN ] LibRadosIo.WriteFullRoundTrip 2024-06-11T13:31:24.669 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ OK ] LibRadosIo.WriteFullRoundTrip (4 ms) 2024-06-11T13:31:24.669 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ RUN ] LibRadosIo.AppendRoundTrip 2024-06-11T13:31:24.669 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ OK ] LibRadosIo.AppendRoundTrip (6 ms) 2024-06-11T13:31:24.669 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ RUN ] LibRadosIo.ZeroLenZero 2024-06-11T13:31:24.669 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ OK ] LibRadosIo.ZeroLenZero (1 ms) 2024-06-11T13:31:24.669 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ RUN ] LibRadosIo.TruncTest 2024-06-11T13:31:24.670 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ OK ] LibRadosIo.TruncTest (9 ms) 2024-06-11T13:31:24.670 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ RUN ] LibRadosIo.RemoveTest 2024-06-11T13:31:24.670 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ OK ] LibRadosIo.RemoveTest (4 ms) 2024-06-11T13:31:24.670 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ RUN ] LibRadosIo.XattrsRoundTrip 2024-06-11T13:31:24.670 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ OK ] LibRadosIo.XattrsRoundTrip (3 ms) 2024-06-11T13:31:24.670 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ RUN ] LibRadosIo.RmXattr 2024-06-11T13:31:24.670 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ OK ] LibRadosIo.RmXattr (9 ms) 2024-06-11T13:31:24.670 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ RUN ] LibRadosIo.XattrIter 2024-06-11T13:31:24.670 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ OK ] LibRadosIo.XattrIter (4 ms) 2024-06-11T13:31:24.670 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [----------] 14 tests from LibRadosIo (266 ms total) 2024-06-11T13:31:24.670 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: 2024-06-11T13:31:24.670 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [----------] 10 tests from LibRadosIoEC 2024-06-11T13:31:24.670 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ RUN ] LibRadosIoEC.SimpleWrite 2024-06-11T13:31:24.671 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ OK ] LibRadosIoEC.SimpleWrite (2868 ms) 2024-06-11T13:31:24.671 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ RUN ] LibRadosIoEC.RoundTrip 2024-06-11T13:31:24.671 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ OK ] LibRadosIoEC.RoundTrip (17 ms) 2024-06-11T13:31:24.671 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ RUN ] LibRadosIoEC.OverlappingWriteRoundTrip 2024-06-11T13:31:24.671 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ OK ] LibRadosIoEC.OverlappingWriteRoundTrip (7 ms) 2024-06-11T13:31:24.671 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ RUN ] LibRadosIoEC.WriteFullRoundTrip 2024-06-11T13:31:24.671 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ OK ] LibRadosIoEC.WriteFullRoundTrip (7 ms) 2024-06-11T13:31:24.671 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ RUN ] LibRadosIoEC.AppendRoundTrip 2024-06-11T13:31:24.671 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ OK ] LibRadosIoEC.AppendRoundTrip (11 ms) 2024-06-11T13:31:24.671 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ RUN ] LibRadosIoEC.TruncTest 2024-06-11T13:31:24.671 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ OK ] LibRadosIoEC.TruncTest (7 ms) 2024-06-11T13:31:24.671 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ RUN ] LibRadosIoEC.RemoveTest 2024-06-11T13:31:24.671 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ OK ] LibRadosIoEC.RemoveTest (3 ms) 2024-06-11T13:31:24.672 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ RUN ] LibRadosIoEC.XattrsRoundTrip 2024-06-11T13:31:24.672 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ OK ] LibRadosIoEC.XattrsRoundTrip (3 ms) 2024-06-11T13:31:24.672 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ RUN ] LibRadosIoEC.RmXattr 2024-06-11T13:31:24.672 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ OK ] LibRadosIoEC.RmXattr (9 ms) 2024-06-11T13:31:24.672 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ RUN ] LibRadosIoEC.XattrIter 2024-06-11T13:31:24.672 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ OK ] LibRadosIoEC.XattrIter (5 ms) 2024-06-11T13:31:24.672 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [----------] 10 tests from LibRadosIoEC (2937 ms total) 2024-06-11T13:31:24.672 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: 2024-06-11T13:31:24.672 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [----------] Global test environment tear-down 2024-06-11T13:31:24.672 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [==========] 24 tests from 2 test suites ran. (41106 ms total) 2024-06-11T13:31:24.673 INFO:tasks.workunit.client.0.smithi057.stdout: api_io: [ PASSED ] 24 tests. 2024-06-11T13:31:25.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[29805]: Health check failed: 2 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-06-11T13:31:25.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[29805]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:31:25.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[29805]: from='client.35414 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi057-78571-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi057-78571-2"}]': finished 2024-06-11T13:31:25.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[29805]: from='client.35363 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4", "tierpool": "test-rados-api-smithi057-77882-6", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:31:25.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[29805]: from='client.35318 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi057-78504-2"}]': finished 2024-06-11T13:31:25.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[29805]: from='client.35423 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:25.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[29805]: from='client.35312 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi057-77374-16"}]': finished 2024-06-11T13:31:25.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[29805]: from='client.35306 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi057-77396-22"}]': finished 2024-06-11T13:31:25.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[29805]: from='client.25108 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-4-cache", "mode": "writeback"}]': finished 2024-06-11T13:31:25.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[29805]: from='client.25438 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi057-77351-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:25.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/197523128' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4", "overlaypool": "test-rados-api-smithi057-77882-6"}]: dispatch 2024-06-11T13:31:25.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1129714156' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi057-77396-22"}]: dispatch 2024-06-11T13:31:25.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1420997228' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi057-77374-16"}]: dispatch 2024-06-11T13:31:25.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:25.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2245769429' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-4"}]: dispatch 2024-06-11T13:31:25.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[29805]: osdmap e93: 8 total, 8 up, 8 in 2024-06-11T13:31:25.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/45811190' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:25.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3748941041' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi057-77494-10"}]: dispatch 2024-06-11T13:31:25.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[29805]: from='client.35363 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4", "overlaypool": "test-rados-api-smithi057-77882-6"}]: dispatch 2024-06-11T13:31:25.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[40588]: Health check failed: 2 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-06-11T13:31:25.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[40588]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:31:25.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[40588]: from='client.35414 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi057-78571-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi057-78571-2"}]': finished 2024-06-11T13:31:25.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[40588]: from='client.35363 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4", "tierpool": "test-rados-api-smithi057-77882-6", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:31:25.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[40588]: from='client.35318 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi057-78504-2"}]': finished 2024-06-11T13:31:25.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[40588]: from='client.35423 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:25.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[40588]: from='client.35312 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi057-77374-16"}]': finished 2024-06-11T13:31:25.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[40588]: from='client.35306 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi057-77396-22"}]': finished 2024-06-11T13:31:25.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[40588]: from='client.25108 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-4-cache", "mode": "writeback"}]': finished 2024-06-11T13:31:25.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[40588]: from='client.25438 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi057-77351-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:25.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/197523128' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4", "overlaypool": "test-rados-api-smithi057-77882-6"}]: dispatch 2024-06-11T13:31:25.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1129714156' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi057-77396-22"}]: dispatch 2024-06-11T13:31:25.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1420997228' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi057-77374-16"}]: dispatch 2024-06-11T13:31:25.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:25.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2245769429' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-4"}]: dispatch 2024-06-11T13:31:25.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[40588]: osdmap e93: 8 total, 8 up, 8 in 2024-06-11T13:31:25.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/45811190' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:25.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3748941041' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi057-77494-10"}]: dispatch 2024-06-11T13:31:25.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[40588]: from='client.35363 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4", "overlaypool": "test-rados-api-smithi057-77882-6"}]: dispatch 2024-06-11T13:31:25.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[29805]: from='client.35306 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi057-77396-22"}]: dispatch 2024-06-11T13:31:25.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[29805]: from='client.35312 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi057-77374-16"}]: dispatch 2024-06-11T13:31:25.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[29805]: from='client.35297 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi057-77494-10"}]: dispatch 2024-06-11T13:31:25.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[29805]: from='client.25108 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-4"}]: dispatch 2024-06-11T13:31:25.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[29805]: from='client.25468 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:25.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1093209324' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-06-11T13:31:25.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[40588]: from='client.35306 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi057-77396-22"}]: dispatch 2024-06-11T13:31:25.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[40588]: from='client.35312 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi057-77374-16"}]: dispatch 2024-06-11T13:31:25.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[40588]: from='client.35297 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi057-77494-10"}]: dispatch 2024-06-11T13:31:25.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[40588]: from='client.25108 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-4"}]: dispatch 2024-06-11T13:31:25.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[40588]: from='client.25468 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:25.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:24 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1093209324' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-06-11T13:31:25.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:24 smithi107 ceph-mon[33178]: Health check failed: 2 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-06-11T13:31:25.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:24 smithi107 ceph-mon[33178]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:31:25.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:24 smithi107 ceph-mon[33178]: from='client.35414 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi057-78571-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi057-78571-2"}]': finished 2024-06-11T13:31:25.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:24 smithi107 ceph-mon[33178]: from='client.35363 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4", "tierpool": "test-rados-api-smithi057-77882-6", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:31:25.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:24 smithi107 ceph-mon[33178]: from='client.35318 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi057-78504-2"}]': finished 2024-06-11T13:31:25.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:24 smithi107 ceph-mon[33178]: from='client.35423 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:25.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:24 smithi107 ceph-mon[33178]: from='client.35312 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi057-77374-16"}]': finished 2024-06-11T13:31:25.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:24 smithi107 ceph-mon[33178]: from='client.35306 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi057-77396-22"}]': finished 2024-06-11T13:31:25.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:24 smithi107 ceph-mon[33178]: from='client.25108 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-4-cache", "mode": "writeback"}]': finished 2024-06-11T13:31:25.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:24 smithi107 ceph-mon[33178]: from='client.25438 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi057-77351-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:25.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:24 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/197523128' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4", "overlaypool": "test-rados-api-smithi057-77882-6"}]: dispatch 2024-06-11T13:31:25.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:24 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1129714156' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi057-77396-22"}]: dispatch 2024-06-11T13:31:25.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:24 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1420997228' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi057-77374-16"}]: dispatch 2024-06-11T13:31:25.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:24 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:25.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:24 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2245769429' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-4"}]: dispatch 2024-06-11T13:31:25.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:24 smithi107 ceph-mon[33178]: osdmap e93: 8 total, 8 up, 8 in 2024-06-11T13:31:25.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:24 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/45811190' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:25.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:24 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3748941041' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi057-77494-10"}]: dispatch 2024-06-11T13:31:25.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:24 smithi107 ceph-mon[33178]: from='client.35363 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4", "overlaypool": "test-rados-api-smithi057-77882-6"}]: dispatch 2024-06-11T13:31:25.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:24 smithi107 ceph-mon[33178]: from='client.35306 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi057-77396-22"}]: dispatch 2024-06-11T13:31:25.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:24 smithi107 ceph-mon[33178]: from='client.35312 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi057-77374-16"}]: dispatch 2024-06-11T13:31:25.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:24 smithi107 ceph-mon[33178]: from='client.35297 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi057-77494-10"}]: dispatch 2024-06-11T13:31:25.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:24 smithi107 ceph-mon[33178]: from='client.25108 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-4"}]: dispatch 2024-06-11T13:31:25.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:24 smithi107 ceph-mon[33178]: from='client.25468 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:25.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:24 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1093209324' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-06-11T13:31:26.047 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock: Running main() from gmock_main.cc 2024-06-11T13:31:26.047 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock: [==========] Running 16 tests from 2 test suites. 2024-06-11T13:31:26.047 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock: [----------] Global test environment set-up. 2024-06-11T13:31:26.047 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock: [----------] 8 tests from LibRadosLock 2024-06-11T13:31:26.047 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusive 2024-06-11T13:31:26.047 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock: [ OK ] LibRadosLock.LockExclusive (151 ms) 2024-06-11T13:31:26.047 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock: [ RUN ] LibRadosLock.LockShared 2024-06-11T13:31:26.047 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock: [ OK ] LibRadosLock.LockShared (21 ms) 2024-06-11T13:31:26.047 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusiveDur 2024-06-11T13:31:26.048 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock: [ OK ] LibRadosLock.LockExclusiveDur (1116 ms) 2024-06-11T13:31:26.048 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock: [ RUN ] LibRadosLock.LockSharedDur 2024-06-11T13:31:26.048 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock: [ OK ] LibRadosLock.LockSharedDur (1025 ms) 2024-06-11T13:31:26.048 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock: [ RUN ] LibRadosLock.LockMayRenew 2024-06-11T13:31:26.048 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock: [ OK ] LibRadosLock.LockMayRenew (3 ms) 2024-06-11T13:31:26.048 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock: [ RUN ] LibRadosLock.Unlock 2024-06-11T13:31:26.048 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock: [ OK ] LibRadosLock.Unlock (9 ms) 2024-06-11T13:31:26.048 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock: [ RUN ] LibRadosLock.ListLockers 2024-06-11T13:31:26.048 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock: [ OK ] LibRadosLock.ListLockers (6 ms) 2024-06-11T13:31:26.048 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock: [ RUN ] LibRadosLock.BreakLock 2024-06-11T13:31:26.048 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock: [ OK ] LibRadosLock.BreakLock (3 ms) 2024-06-11T13:31:26.048 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock: [----------] 8 tests from LibRadosLock (2334 ms total) 2024-06-11T13:31:26.048 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock: 2024-06-11T13:31:26.048 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock: [----------] 8 tests from LibRadosLockEC 2024-06-11T13:31:26.049 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusive 2024-06-11T13:31:26.049 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusive (3767 ms) 2024-06-11T13:31:26.049 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock: [ RUN ] LibRadosLockEC.LockShared 2024-06-11T13:31:26.049 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock: [ OK ] LibRadosLockEC.LockShared (44 ms) 2024-06-11T13:31:26.049 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusiveDur 2024-06-11T13:31:26.049 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusiveDur (1006 ms) 2024-06-11T13:31:26.049 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock: [ RUN ] LibRadosLockEC.LockSharedDur 2024-06-11T13:31:26.049 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock: [ OK ] LibRadosLockEC.LockSharedDur (1004 ms) 2024-06-11T13:31:26.049 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock: [ RUN ] LibRadosLockEC.LockMayRenew 2024-06-11T13:31:26.049 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock: [ OK ] LibRadosLockEC.LockMayRenew (4 ms) 2024-06-11T13:31:26.049 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock: [ RUN ] LibRadosLockEC.Unlock 2024-06-11T13:31:26.049 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock: [ OK ] LibRadosLockEC.Unlock (4 ms) 2024-06-11T13:31:26.049 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock: [ RUN ] LibRadosLockEC.ListLockers 2024-06-11T13:31:26.049 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock: [ OK ] LibRadosLockEC.ListLockers (5 ms) 2024-06-11T13:31:26.050 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock: [ RUN ] LibRadosLockEC.BreakLock 2024-06-11T13:31:26.050 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock: [ OK ] LibRadosLockEC.BreakLock (2 ms) 2024-06-11T13:31:26.050 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock: [----------] 8 tests from LibRadosLockEC (5836 ms total) 2024-06-11T13:31:26.050 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock: 2024-06-11T13:31:26.050 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock: [----------] Global test environment tear-down 2024-06-11T13:31:26.050 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock: [==========] 16 tests from 2 test suites ran. (42427 ms total) 2024-06-11T13:31:26.050 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock: [ PASSED ] 16 tests. 2024-06-11T13:31:26.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:26 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:26.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:26 smithi057 ceph-mon[40588]: from='client.35402 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi057-77874-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi057-77874-12"}]': finished 2024-06-11T13:31:26.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:26 smithi057 ceph-mon[40588]: from='client.35363 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4", "overlaypool": "test-rados-api-smithi057-77882-6"}]': finished 2024-06-11T13:31:26.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:26 smithi057 ceph-mon[40588]: from='client.35306 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi057-77396-22"}]': finished 2024-06-11T13:31:26.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:26 smithi057 ceph-mon[40588]: from='client.35312 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi057-77374-16"}]': finished 2024-06-11T13:31:26.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:26 smithi057 ceph-mon[40588]: from='client.35297 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi057-77494-10"}]': finished 2024-06-11T13:31:26.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:26 smithi057 ceph-mon[40588]: from='client.25108 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-4"}]': finished 2024-06-11T13:31:26.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:26 smithi057 ceph-mon[40588]: from='client.25468 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:26.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:26 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/197523128' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77882-6", "mode": "writeback"}]: dispatch 2024-06-11T13:31:26.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:26 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3748941041' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi057-77494-10"}]: dispatch 2024-06-11T13:31:26.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:26 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2245769429' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-4", "tierpool": "test-rados-api-smithi057-77599-4-cache"}]: dispatch 2024-06-11T13:31:26.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:26 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3479760694' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi057-77785-7"}]: dispatch 2024-06-11T13:31:26.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:26 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2607108900' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi057-77831-7"}]: dispatch 2024-06-11T13:31:26.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:26 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:26.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:26 smithi057 ceph-mon[40588]: osdmap e94: 8 total, 8 up, 8 in 2024-06-11T13:31:26.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:26 smithi057 ceph-mon[40588]: Health check update: 11 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:31:26.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:26 smithi057 ceph-mon[40588]: from='client.35363 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77882-6", "mode": "writeback"}]: dispatch 2024-06-11T13:31:26.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:26 smithi057 ceph-mon[40588]: from='client.25108 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-4", "tierpool": "test-rados-api-smithi057-77599-4-cache"}]: dispatch 2024-06-11T13:31:26.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:26 smithi057 ceph-mon[40588]: from='client.35297 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi057-77494-10"}]: dispatch 2024-06-11T13:31:26.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:26 smithi057 ceph-mon[40588]: from='client.25303 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi057-77785-7"}]: dispatch 2024-06-11T13:31:26.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:26 smithi057 ceph-mon[40588]: from='client.25297 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi057-77831-7"}]: dispatch 2024-06-11T13:31:26.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:26 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:26.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:26 smithi057 ceph-mon[29805]: from='client.35402 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi057-77874-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi057-77874-12"}]': finished 2024-06-11T13:31:26.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:26 smithi057 ceph-mon[29805]: from='client.35363 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4", "overlaypool": "test-rados-api-smithi057-77882-6"}]': finished 2024-06-11T13:31:26.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:26 smithi057 ceph-mon[29805]: from='client.35306 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi057-77396-22"}]': finished 2024-06-11T13:31:26.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:26 smithi057 ceph-mon[29805]: from='client.35312 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi057-77374-16"}]': finished 2024-06-11T13:31:26.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:26 smithi057 ceph-mon[29805]: from='client.35297 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi057-77494-10"}]': finished 2024-06-11T13:31:26.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:26 smithi057 ceph-mon[29805]: from='client.25108 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-4"}]': finished 2024-06-11T13:31:26.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:26 smithi057 ceph-mon[29805]: from='client.25468 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:26.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:26 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/197523128' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77882-6", "mode": "writeback"}]: dispatch 2024-06-11T13:31:26.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:26 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3748941041' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi057-77494-10"}]: dispatch 2024-06-11T13:31:26.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:26 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2245769429' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-4", "tierpool": "test-rados-api-smithi057-77599-4-cache"}]: dispatch 2024-06-11T13:31:26.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:26 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3479760694' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi057-77785-7"}]: dispatch 2024-06-11T13:31:26.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:26 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2607108900' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi057-77831-7"}]: dispatch 2024-06-11T13:31:26.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:26 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:26.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:26 smithi057 ceph-mon[29805]: osdmap e94: 8 total, 8 up, 8 in 2024-06-11T13:31:26.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:26 smithi057 ceph-mon[29805]: Health check update: 11 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:31:26.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:26 smithi057 ceph-mon[29805]: from='client.35363 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77882-6", "mode": "writeback"}]: dispatch 2024-06-11T13:31:26.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:26 smithi057 ceph-mon[29805]: from='client.25108 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-4", "tierpool": "test-rados-api-smithi057-77599-4-cache"}]: dispatch 2024-06-11T13:31:26.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:26 smithi057 ceph-mon[29805]: from='client.35297 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi057-77494-10"}]: dispatch 2024-06-11T13:31:26.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:26 smithi057 ceph-mon[29805]: from='client.25303 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi057-77785-7"}]: dispatch 2024-06-11T13:31:26.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:26 smithi057 ceph-mon[29805]: from='client.25297 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi057-77831-7"}]: dispatch 2024-06-11T13:31:26.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:26 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:26.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:26 smithi107 ceph-mon[33178]: from='client.35402 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi057-77874-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi057-77874-12"}]': finished 2024-06-11T13:31:26.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:26 smithi107 ceph-mon[33178]: from='client.35363 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4", "overlaypool": "test-rados-api-smithi057-77882-6"}]': finished 2024-06-11T13:31:26.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:26 smithi107 ceph-mon[33178]: from='client.35306 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi057-77396-22"}]': finished 2024-06-11T13:31:26.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:26 smithi107 ceph-mon[33178]: from='client.35312 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi057-77374-16"}]': finished 2024-06-11T13:31:26.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:26 smithi107 ceph-mon[33178]: from='client.35297 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi057-77494-10"}]': finished 2024-06-11T13:31:26.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:26 smithi107 ceph-mon[33178]: from='client.25108 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-4"}]': finished 2024-06-11T13:31:26.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:26 smithi107 ceph-mon[33178]: from='client.25468 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:26.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:26 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/197523128' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77882-6", "mode": "writeback"}]: dispatch 2024-06-11T13:31:26.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:26 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3748941041' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi057-77494-10"}]: dispatch 2024-06-11T13:31:26.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:26 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2245769429' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-4", "tierpool": "test-rados-api-smithi057-77599-4-cache"}]: dispatch 2024-06-11T13:31:26.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:26 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3479760694' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi057-77785-7"}]: dispatch 2024-06-11T13:31:26.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:26 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2607108900' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi057-77831-7"}]: dispatch 2024-06-11T13:31:26.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:26 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:26.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:26 smithi107 ceph-mon[33178]: osdmap e94: 8 total, 8 up, 8 in 2024-06-11T13:31:26.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:26 smithi107 ceph-mon[33178]: Health check update: 11 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:31:26.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:26 smithi107 ceph-mon[33178]: from='client.35363 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77882-6", "mode": "writeback"}]: dispatch 2024-06-11T13:31:26.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:26 smithi107 ceph-mon[33178]: from='client.25108 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-4", "tierpool": "test-rados-api-smithi057-77599-4-cache"}]: dispatch 2024-06-11T13:31:26.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:26 smithi107 ceph-mon[33178]: from='client.35297 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi057-77494-10"}]: dispatch 2024-06-11T13:31:26.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:26 smithi107 ceph-mon[33178]: from='client.25303 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi057-77785-7"}]: dispatch 2024-06-11T13:31:26.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:26 smithi107 ceph-mon[33178]: from='client.25297 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi057-77831-7"}]: dispatch 2024-06-11T13:31:27.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:27 smithi057 ceph-mon[29805]: from='mon.? v1:172.21.15.57:0/45435016' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-06-11T13:31:27.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:27 smithi057 ceph-mon[29805]: pgmap v104: 776 pgs: 27 creating+peering, 137 unknown, 612 active+clean; 145 MiB data, 777 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 5.2 KiB/s wr, 3 op/s 2024-06-11T13:31:27.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:27 smithi057 ceph-mon[29805]: from='client.35363 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77882-6", "mode": "writeback"}]': finished 2024-06-11T13:31:27.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:27 smithi057 ceph-mon[40588]: from='mon.? v1:172.21.15.57:0/45435016' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-06-11T13:31:27.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:27 smithi057 ceph-mon[40588]: pgmap v104: 776 pgs: 27 creating+peering, 137 unknown, 612 active+clean; 145 MiB data, 777 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 5.2 KiB/s wr, 3 op/s 2024-06-11T13:31:27.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:27 smithi057 ceph-mon[40588]: from='client.35363 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77882-6", "mode": "writeback"}]': finished 2024-06-11T13:31:27.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:27 smithi107 ceph-mon[33178]: from='mon.? v1:172.21.15.57:0/45435016' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-06-11T13:31:27.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:27 smithi107 ceph-mon[33178]: pgmap v104: 776 pgs: 27 creating+peering, 137 unknown, 612 active+clean; 145 MiB data, 777 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 5.2 KiB/s wr, 3 op/s 2024-06-11T13:31:27.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:27 smithi107 ceph-mon[33178]: from='client.35363 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77882-6", "mode": "writeback"}]': finished 2024-06-11T13:31:28.405 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat_pp: Running main() from gmock_main.cc 2024-06-11T13:31:28.405 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat_pp: [==========] Running 9 tests from 2 test suites. 2024-06-11T13:31:28.405 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat_pp: [----------] Global test environment set-up. 2024-06-11T13:31:28.405 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP 2024-06-11T13:31:28.406 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat_pp: seed 77831 2024-06-11T13:31:28.406 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPP 2024-06-11T13:31:28.406 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPP (243 ms) 2024-06-11T13:31:28.406 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.Stat2Mtime2PP 2024-06-11T13:31:28.406 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat_pp: [ OK ] LibRadosStatPP.Stat2Mtime2PP (3 ms) 2024-06-11T13:31:28.406 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.ClusterStatPP 2024-06-11T13:31:28.406 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat_pp: [ OK ] LibRadosStatPP.ClusterStatPP (1 ms) 2024-06-11T13:31:28.406 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.PoolStatPP 2024-06-11T13:31:28.406 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat_pp: [ OK ] LibRadosStatPP.PoolStatPP (3 ms) 2024-06-11T13:31:28.406 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPPNS 2024-06-11T13:31:28.406 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPPNS (9 ms) 2024-06-11T13:31:28.406 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP (259 ms total) 2024-06-11T13:31:28.407 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat_pp: 2024-06-11T13:31:28.407 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP 2024-06-11T13:31:28.407 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPP 2024-06-11T13:31:28.407 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPP (1141 ms) 2024-06-11T13:31:28.407 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.ClusterStatPP 2024-06-11T13:31:28.407 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.ClusterStatPP (0 ms) 2024-06-11T13:31:28.407 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.PoolStatPP 2024-06-11T13:31:28.407 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.PoolStatPP (8 ms) 2024-06-11T13:31:28.407 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPPNS 2024-06-11T13:31:28.407 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPPNS (13 ms) 2024-06-11T13:31:28.407 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP (1162 ms total) 2024-06-11T13:31:28.408 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat_pp: 2024-06-11T13:31:28.408 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat_pp: [----------] Global test environment tear-down 2024-06-11T13:31:28.408 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat_pp: [==========] 9 tests from 2 test suites ran. (44634 ms total) 2024-06-11T13:31:28.408 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat_pp: [ PASSED ] 9 tests. 2024-06-11T13:31:28.408 INFO:tasks.workunit.client.0.smithi057.stdout: pool: Running main() from gmock_main.cc 2024-06-11T13:31:28.408 INFO:tasks.workunit.client.0.smithi057.stdout: pool: [==========] Running 6 tests from 1 test suite. 2024-06-11T13:31:28.408 INFO:tasks.workunit.client.0.smithi057.stdout: pool: [----------] Global test environment set-up. 2024-06-11T13:31:28.408 INFO:tasks.workunit.client.0.smithi057.stdout: pool: [----------] 6 tests from NeoRadosPools 2024-06-11T13:31:28.408 INFO:tasks.workunit.client.0.smithi057.stdout: pool: [ RUN ] NeoRadosPools.PoolList 2024-06-11T13:31:28.408 INFO:tasks.workunit.client.0.smithi057.stdout: pool: [ OK ] NeoRadosPools.PoolList (16121 ms) 2024-06-11T13:31:28.408 INFO:tasks.workunit.client.0.smithi057.stdout: pool: [ RUN ] NeoRadosPools.PoolLookup 2024-06-11T13:31:28.408 INFO:tasks.workunit.client.0.smithi057.stdout: pool: [ OK ] NeoRadosPools.PoolLookup (4834 ms) 2024-06-11T13:31:28.409 INFO:tasks.workunit.client.0.smithi057.stdout: pool: [ RUN ] NeoRadosPools.PoolLookupOtherInstance 2024-06-11T13:31:28.409 INFO:tasks.workunit.client.0.smithi057.stdout: pool: [ OK ] NeoRadosPools.PoolLookupOtherInstance (5618 ms) 2024-06-11T13:31:28.409 INFO:tasks.workunit.client.0.smithi057.stdout: pool: [ RUN ] NeoRadosPools.PoolDelete 2024-06-11T13:31:28.409 INFO:tasks.workunit.client.0.smithi057.stdout: pool: [ OK ] NeoRadosPools.PoolDelete (8378 ms) 2024-06-11T13:31:28.409 INFO:tasks.workunit.client.0.smithi057.stdout: pool: [ RUN ] NeoRadosPools.PoolCreateDelete 2024-06-11T13:31:28.409 INFO:tasks.workunit.client.0.smithi057.stdout: pool: [ OK ] NeoRadosPools.PoolCreateDelete (4686 ms) 2024-06-11T13:31:28.409 INFO:tasks.workunit.client.0.smithi057.stdout: pool: [ RUN ] NeoRadosPools.PoolCreateWithCrushRule 2024-06-11T13:31:28.409 INFO:tasks.workunit.client.0.smithi057.stdout: pool: [ OK ] NeoRadosPools.PoolCreateWithCrushRule (4673 ms) 2024-06-11T13:31:28.409 INFO:tasks.workunit.client.0.smithi057.stdout: pool: [----------] 6 tests from NeoRadosPools (44310 ms total) 2024-06-11T13:31:28.409 INFO:tasks.workunit.client.0.smithi057.stdout: pool: 2024-06-11T13:31:28.409 INFO:tasks.workunit.client.0.smithi057.stdout: pool: [----------] Global test environment tear-down 2024-06-11T13:31:28.409 INFO:tasks.workunit.client.0.smithi057.stdout: pool: [==========] 6 tests from 1 test suite ran. (44310 ms total) 2024-06-11T13:31:28.410 INFO:tasks.workunit.client.0.smithi057.stdout: pool: [ PASSED ] 6 tests. 2024-06-11T13:31:28.410 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat: Running main() from gmock_main.cc 2024-06-11T13:31:28.410 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat: [==========] Running 9 tests from 2 test suites. 2024-06-11T13:31:28.410 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat: [----------] Global test environment set-up. 2024-06-11T13:31:28.410 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat: [----------] 5 tests from LibRadosStat 2024-06-11T13:31:28.410 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat: [ RUN ] LibRadosStat.Stat 2024-06-11T13:31:28.410 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat: [ OK ] LibRadosStat.Stat (206 ms) 2024-06-11T13:31:28.410 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat: [ RUN ] LibRadosStat.Stat2 2024-06-11T13:31:28.411 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat: [ OK ] LibRadosStat.Stat2 (33 ms) 2024-06-11T13:31:28.411 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat: [ RUN ] LibRadosStat.StatNS 2024-06-11T13:31:28.411 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat: [ OK ] LibRadosStat.StatNS (9 ms) 2024-06-11T13:31:28.411 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat: [ RUN ] LibRadosStat.ClusterStat 2024-06-11T13:31:28.411 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat: [ OK ] LibRadosStat.ClusterStat (1 ms) 2024-06-11T13:31:28.411 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat: [ RUN ] LibRadosStat.PoolStat 2024-06-11T13:31:28.411 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat: [ OK ] LibRadosStat.PoolStat (3 ms) 2024-06-11T13:31:28.411 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat: [----------] 5 tests from LibRadosStat (252 ms total) 2024-06-11T13:31:28.411 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat: 2024-06-11T13:31:28.411 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat: [----------] 4 tests from LibRadosStatEC 2024-06-11T13:31:28.411 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat: [ RUN ] LibRadosStatEC.Stat 2024-06-11T13:31:28.411 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat: [ OK ] LibRadosStatEC.Stat (1105 ms) 2024-06-11T13:31:28.412 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat: [ RUN ] LibRadosStatEC.StatNS 2024-06-11T13:31:28.412 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat: [ OK ] LibRadosStatEC.StatNS (61 ms) 2024-06-11T13:31:28.412 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat: [ RUN ] LibRadosStatEC.ClusterStat 2024-06-11T13:31:28.412 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat: [ OK ] LibRadosStatEC.ClusterStat (0 ms) 2024-06-11T13:31:28.412 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat: [ RUN ] LibRadosStatEC.PoolStat 2024-06-11T13:31:28.412 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat: [ OK ] LibRadosStatEC.PoolStat (1 ms) 2024-06-11T13:31:28.412 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat: [----------] 4 tests from LibRadosStatEC (1167 ms total) 2024-06-11T13:31:28.412 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat: 2024-06-11T13:31:28.412 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat: [----------] Global test environment tear-down 2024-06-11T13:31:28.412 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat: [==========] 9 tests from 2 test suites ran. (44663 ms total) 2024-06-11T13:31:28.412 INFO:tasks.workunit.client.0.smithi057.stdout: api_stat: [ PASSED ] 9 tests. 2024-06-11T13:31:28.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:28 smithi057 ceph-mon[40588]: from='client.25108 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-4", "tierpool": "test-rados-api-smithi057-77599-4-cache"}]': finished 2024-06-11T13:31:28.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:28 smithi057 ceph-mon[40588]: from='client.35297 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi057-77494-10"}]': finished 2024-06-11T13:31:28.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:28 smithi057 ceph-mon[40588]: from='client.25303 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi057-77785-7"}]': finished 2024-06-11T13:31:28.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:28 smithi057 ceph-mon[40588]: from='client.25297 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi057-77831-7"}]': finished 2024-06-11T13:31:28.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:28 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2607108900' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi057-77831-7"}]: dispatch 2024-06-11T13:31:28.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:28 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3479760694' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi057-77785-7"}]: dispatch 2024-06-11T13:31:28.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:28 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:28.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:28 smithi057 ceph-mon[40588]: pgmap v107: 800 pgs: 96 unknown, 72 creating+peering, 632 active+clean; 145 MiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 4.1 KiB/s wr, 4 op/s 2024-06-11T13:31:28.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:28 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:28.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:28 smithi057 ceph-mon[40588]: osdmap e95: 8 total, 8 up, 8 in 2024-06-11T13:31:28.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:28 smithi057 ceph-mon[40588]: from='client.25297 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi057-77831-7"}]: dispatch 2024-06-11T13:31:28.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:28 smithi057 ceph-mon[40588]: from='client.25303 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi057-77785-7"}]: dispatch 2024-06-11T13:31:28.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:28 smithi057 ceph-mon[40588]: osdmap e96: 8 total, 8 up, 8 in 2024-06-11T13:31:28.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:28 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/4071677954' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:28.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:28 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/232412548' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi057-77506-10"}]: dispatch 2024-06-11T13:31:28.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:28 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2501793636' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi057-78571-2"}]: dispatch 2024-06-11T13:31:28.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:28 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2090315408' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi057-77351-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:28.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:28 smithi057 ceph-mon[40588]: from='client.25516 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:28.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:28 smithi057 ceph-mon[40588]: from='client.35414 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi057-78571-2"}]: dispatch 2024-06-11T13:31:28.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:28 smithi057 ceph-mon[40588]: from='client.35504 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi057-77351-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:28.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:28 smithi057 ceph-mon[40588]: from='client.25279 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi057-77506-10"}]: dispatch 2024-06-11T13:31:28.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:28 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:28.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:28 smithi057 ceph-mon[29805]: from='client.25108 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-4", "tierpool": "test-rados-api-smithi057-77599-4-cache"}]': finished 2024-06-11T13:31:28.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:28 smithi057 ceph-mon[29805]: from='client.35297 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi057-77494-10"}]': finished 2024-06-11T13:31:28.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:28 smithi057 ceph-mon[29805]: from='client.25303 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi057-77785-7"}]': finished 2024-06-11T13:31:28.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:28 smithi057 ceph-mon[29805]: from='client.25297 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi057-77831-7"}]': finished 2024-06-11T13:31:28.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:28 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2607108900' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi057-77831-7"}]: dispatch 2024-06-11T13:31:28.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:28 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3479760694' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi057-77785-7"}]: dispatch 2024-06-11T13:31:28.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:28 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:28.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:28 smithi057 ceph-mon[29805]: pgmap v107: 800 pgs: 96 unknown, 72 creating+peering, 632 active+clean; 145 MiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 4.1 KiB/s wr, 4 op/s 2024-06-11T13:31:28.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:28 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:28.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:28 smithi057 ceph-mon[29805]: osdmap e95: 8 total, 8 up, 8 in 2024-06-11T13:31:28.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:28 smithi057 ceph-mon[29805]: from='client.25297 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi057-77831-7"}]: dispatch 2024-06-11T13:31:28.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:28 smithi057 ceph-mon[29805]: from='client.25303 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi057-77785-7"}]: dispatch 2024-06-11T13:31:28.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:28 smithi057 ceph-mon[29805]: osdmap e96: 8 total, 8 up, 8 in 2024-06-11T13:31:28.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:28 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/4071677954' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:28.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:28 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/232412548' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi057-77506-10"}]: dispatch 2024-06-11T13:31:28.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:28 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2501793636' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi057-78571-2"}]: dispatch 2024-06-11T13:31:28.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:28 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2090315408' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi057-77351-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:28.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:28 smithi057 ceph-mon[29805]: from='client.25516 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:28.788 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:28 smithi057 ceph-mon[29805]: from='client.35414 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi057-78571-2"}]: dispatch 2024-06-11T13:31:28.788 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:28 smithi057 ceph-mon[29805]: from='client.35504 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi057-77351-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:28.788 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:28 smithi057 ceph-mon[29805]: from='client.25279 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi057-77506-10"}]: dispatch 2024-06-11T13:31:28.788 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:28 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:28.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:28 smithi107 ceph-mon[33178]: from='client.25108 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-4", "tierpool": "test-rados-api-smithi057-77599-4-cache"}]': finished 2024-06-11T13:31:28.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:28 smithi107 ceph-mon[33178]: from='client.35297 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi057-77494-10"}]': finished 2024-06-11T13:31:28.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:28 smithi107 ceph-mon[33178]: from='client.25303 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi057-77785-7"}]': finished 2024-06-11T13:31:28.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:28 smithi107 ceph-mon[33178]: from='client.25297 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi057-77831-7"}]': finished 2024-06-11T13:31:28.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:28 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2607108900' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi057-77831-7"}]: dispatch 2024-06-11T13:31:28.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:28 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3479760694' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi057-77785-7"}]: dispatch 2024-06-11T13:31:28.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:28 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:28.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:28 smithi107 ceph-mon[33178]: pgmap v107: 800 pgs: 96 unknown, 72 creating+peering, 632 active+clean; 145 MiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 4.1 KiB/s wr, 4 op/s 2024-06-11T13:31:28.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:28 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:28.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:28 smithi107 ceph-mon[33178]: osdmap e95: 8 total, 8 up, 8 in 2024-06-11T13:31:28.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:28 smithi107 ceph-mon[33178]: from='client.25297 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi057-77831-7"}]: dispatch 2024-06-11T13:31:28.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:28 smithi107 ceph-mon[33178]: from='client.25303 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi057-77785-7"}]: dispatch 2024-06-11T13:31:28.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:28 smithi107 ceph-mon[33178]: osdmap e96: 8 total, 8 up, 8 in 2024-06-11T13:31:28.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:28 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/4071677954' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:28.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:28 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/232412548' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi057-77506-10"}]: dispatch 2024-06-11T13:31:28.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:28 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2501793636' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi057-78571-2"}]: dispatch 2024-06-11T13:31:28.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:28 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2090315408' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi057-77351-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:28.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:28 smithi107 ceph-mon[33178]: from='client.25516 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:28.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:28 smithi107 ceph-mon[33178]: from='client.35414 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi057-78571-2"}]: dispatch 2024-06-11T13:31:28.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:28 smithi107 ceph-mon[33178]: from='client.35504 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi057-77351-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:28.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:28 smithi107 ceph-mon[33178]: from='client.25279 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi057-77506-10"}]: dispatch 2024-06-11T13:31:28.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:28 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:29.111 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: Running main() from gmock_main.cc 2024-06-11T13:31:29.111 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: [==========] Running 16 tests from 2 test suites. 2024-06-11T13:31:29.111 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: [----------] Global test environment set-up. 2024-06-11T13:31:29.111 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP 2024-06-11T13:31:29.111 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotify 2024-06-11T13:31:29.111 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: notify 2024-06-11T13:31:29.111 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotify (2468 ms) 2024-06-11T13:31:29.111 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotifyTimeout 2024-06-11T13:31:29.111 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotifyTimeout (1563 ms) 2024-06-11T13:31:29.111 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP (4031 ms total) 2024-06-11T13:31:29.112 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: 2024-06-11T13:31:29.112 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP 2024-06-11T13:31:29.112 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 2024-06-11T13:31:29.112 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: notify 2024-06-11T13:31:29.112 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 (64 ms) 2024-06-11T13:31:29.112 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 2024-06-11T13:31:29.112 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: notify 2024-06-11T13:31:29.112 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 (8846 ms) 2024-06-11T13:31:29.112 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 2024-06-11T13:31:29.112 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 (4 ms) 2024-06-11T13:31:29.112 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 2024-06-11T13:31:29.112 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 (5 ms) 2024-06-11T13:31:29.112 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 2024-06-11T13:31:29.112 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: handle_notify cookie 94514181174464 notify_id 408021893122 notifier_gid 35363 2024-06-11T13:31:29.113 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 (5 ms) 2024-06-11T13:31:29.113 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 2024-06-11T13:31:29.113 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: handle_notify cookie 94514181174464 notify_id 408021893123 notifier_gid 35363 2024-06-11T13:31:29.113 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 (5 ms) 2024-06-11T13:31:29.113 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 2024-06-11T13:31:29.113 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: handle_notify cookie 94514181176640 notify_id 408021893121 notifier_gid 35363 2024-06-11T13:31:29.113 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 (5 ms) 2024-06-11T13:31:29.113 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 2024-06-11T13:31:29.113 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: handle_notify cookie 94514182242688 notify_id 408021893126 notifier_gid 35363 2024-06-11T13:31:29.113 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 (5 ms) 2024-06-11T13:31:29.113 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 2024-06-11T13:31:29.113 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: handle_notify cookie 94514182242688 notify_id 408021893124 notifier_gid 35363 2024-06-11T13:31:29.113 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 (5 ms) 2024-06-11T13:31:29.113 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 2024-06-11T13:31:29.114 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: handle_notify cookie 94514182242688 notify_id 408021893125 notifier_gid 35363 2024-06-11T13:31:29.114 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 (4 ms) 2024-06-11T13:31:29.114 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 2024-06-11T13:31:29.114 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: trying... 2024-06-11T13:31:29.114 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: handle_notify cookie 94514182264688 notify_id 408021893127 notifier_gid 35363 2024-06-11T13:31:29.114 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: timed out 2024-06-11T13:31:29.114 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: flushing 2024-06-11T13:31:29.825 INFO:tasks.workunit.client.0.smithi057.stdout: api_wa api_lock_pp: Running main() from gmock_main.cc 2024-06-11T13:31:29.825 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock_pp: [==========] Running 16 tests from 2 test suites. 2024-06-11T13:31:29.825 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock_pp: [----------] Global test environment set-up. 2024-06-11T13:31:29.825 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP 2024-06-11T13:31:29.825 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock_pp: seed 77506 2024-06-11T13:31:29.825 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusivePP 2024-06-11T13:31:29.825 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusivePP (150 ms) 2024-06-11T13:31:29.825 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedPP 2024-06-11T13:31:29.825 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedPP (29 ms) 2024-06-11T13:31:29.825 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusiveDurPP 2024-06-11T13:31:29.825 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusiveDurPP (1136 ms) 2024-06-11T13:31:29.826 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedDurPP 2024-06-11T13:31:29.826 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedDurPP (1004 ms) 2024-06-11T13:31:29.826 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockMayRenewPP 2024-06-11T13:31:29.826 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockMayRenewPP (4 ms) 2024-06-11T13:31:29.826 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.UnlockPP 2024-06-11T13:31:29.826 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock_pp: [ OK ] LibRadosLockPP.UnlockPP (5 ms) 2024-06-11T13:31:29.826 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.ListLockersPP 2024-06-11T13:31:29.826 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock_pp: [ OK ] LibRadosLockPP.ListLockersPP (7 ms) 2024-06-11T13:31:29.826 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.BreakLockPP 2024-06-11T13:31:29.826 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock_pp: [ OK ] LibRadosLockPP.BreakLockPP (3 ms) 2024-06-11T13:31:29.826 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP (2339 ms total) 2024-06-11T13:31:29.826 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock_pp: 2024-06-11T13:31:29.826 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP 2024-06-11T13:31:29.827 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusivePP 2024-06-11T13:31:29.827 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusivePP (3353 ms) 2024-06-11T13:31:29.827 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedPP 2024-06-11T13:31:29.827 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedPP (64 ms) 2024-06-11T13:31:29.827 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusiveDurPP 2024-06-11T13:31:29.827 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusiveDurPP (1005 ms) 2024-06-11T13:31:29.827 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedDurPP 2024-06-11T13:31:29.827 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedDurPP (1004 ms) 2024-06-11T13:31:29.827 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockMayRenewPP 2024-06-11T13:31:29.827 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockMayRenewPP (4 ms) 2024-06-11T13:31:29.827 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.UnlockPP 2024-06-11T13:31:29.827 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.UnlockPP (3 ms) 2024-06-11T13:31:29.827 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.ListLockersPP 2024-06-11T13:31:29.827 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.ListLockersPP (4 ms) 2024-06-11T13:31:29.829 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.BreakLockPP 2024-06-11T13:31:29.829 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.BreakLockPP (3 ms) 2024-06-11T13:31:29.829 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP (5440 ms total) 2024-06-11T13:31:29.829 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock_pp: 2024-06-11T13:31:29.830 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock_pp: [----------] Global test environment tear-down 2024-06-11T13:31:29.830 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock_pp: [==========] 16 tests from 2 test suites ran. (46206 ms total) 2024-06-11T13:31:29.830 INFO:tasks.workunit.client.0.smithi057.stdout: api_lock_pp: [ PASSED ] 16 tests. 2024-06-11T13:31:30.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:29 smithi057 ceph-mon[29805]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-06-11T13:31:30.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:29 smithi057 ceph-mon[29805]: from='client.25297 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi057-77831-7"}]': finished 2024-06-11T13:31:30.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:29 smithi057 ceph-mon[29805]: from='client.25303 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi057-77785-7"}]': finished 2024-06-11T13:31:30.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:29 smithi057 ceph-mon[29805]: from='client.25516 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:30.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:29 smithi057 ceph-mon[29805]: from='client.35414 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi057-78571-2"}]': finished 2024-06-11T13:31:30.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:29 smithi057 ceph-mon[29805]: from='client.35504 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi057-77351-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:30.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:29 smithi057 ceph-mon[29805]: from='client.25279 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi057-77506-10"}]': finished 2024-06-11T13:31:30.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:29 smithi057 ceph-mon[29805]: osdmap e97: 8 total, 8 up, 8 in 2024-06-11T13:31:30.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:29 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2501793636' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi057-78571-2"}]: dispatch 2024-06-11T13:31:30.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:29 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3799830007' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi057-77874-12"}]: dispatch 2024-06-11T13:31:30.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:29 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/232412548' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi057-77506-10"}]: dispatch 2024-06-11T13:31:30.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:29 smithi057 ceph-mon[29805]: from='client.35414 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi057-78571-2"}]: dispatch 2024-06-11T13:31:30.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:29 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2194111342' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:30.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:29 smithi057 ceph-mon[29805]: from='client.25279 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi057-77506-10"}]: dispatch 2024-06-11T13:31:30.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:29 smithi057 ceph-mon[29805]: from='client.35402 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi057-77874-12"}]: dispatch 2024-06-11T13:31:30.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:29 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:30.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:29 smithi057 ceph-mon[40588]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-06-11T13:31:30.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:29 smithi057 ceph-mon[40588]: from='client.25297 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi057-77831-7"}]': finished 2024-06-11T13:31:30.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:29 smithi057 ceph-mon[40588]: from='client.25303 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi057-77785-7"}]': finished 2024-06-11T13:31:30.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:29 smithi057 ceph-mon[40588]: from='client.25516 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:30.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:29 smithi057 ceph-mon[40588]: from='client.35414 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi057-78571-2"}]': finished 2024-06-11T13:31:30.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:29 smithi057 ceph-mon[40588]: from='client.35504 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi057-77351-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:30.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:29 smithi057 ceph-mon[40588]: from='client.25279 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi057-77506-10"}]': finished 2024-06-11T13:31:30.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:29 smithi057 ceph-mon[40588]: osdmap e97: 8 total, 8 up, 8 in 2024-06-11T13:31:30.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:29 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2501793636' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi057-78571-2"}]: dispatch 2024-06-11T13:31:30.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:29 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3799830007' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi057-77874-12"}]: dispatch 2024-06-11T13:31:30.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:29 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/232412548' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi057-77506-10"}]: dispatch 2024-06-11T13:31:30.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:29 smithi057 ceph-mon[40588]: from='client.35414 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi057-78571-2"}]: dispatch 2024-06-11T13:31:30.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:29 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2194111342' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:30.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:29 smithi057 ceph-mon[40588]: from='client.25279 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi057-77506-10"}]: dispatch 2024-06-11T13:31:30.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:29 smithi057 ceph-mon[40588]: from='client.35402 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi057-77874-12"}]: dispatch 2024-06-11T13:31:30.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:29 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:30.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:29 smithi107 ceph-mon[33178]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-06-11T13:31:30.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:29 smithi107 ceph-mon[33178]: from='client.25297 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi057-77831-7"}]': finished 2024-06-11T13:31:30.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:29 smithi107 ceph-mon[33178]: from='client.25303 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi057-77785-7"}]': finished 2024-06-11T13:31:30.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:29 smithi107 ceph-mon[33178]: from='client.25516 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:30.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:29 smithi107 ceph-mon[33178]: from='client.35414 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi057-78571-2"}]': finished 2024-06-11T13:31:30.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:29 smithi107 ceph-mon[33178]: from='client.35504 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi057-77351-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:30.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:29 smithi107 ceph-mon[33178]: from='client.25279 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi057-77506-10"}]': finished 2024-06-11T13:31:30.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:29 smithi107 ceph-mon[33178]: osdmap e97: 8 total, 8 up, 8 in 2024-06-11T13:31:30.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:29 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2501793636' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi057-78571-2"}]: dispatch 2024-06-11T13:31:30.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:29 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3799830007' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi057-77874-12"}]: dispatch 2024-06-11T13:31:30.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:29 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/232412548' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi057-77506-10"}]: dispatch 2024-06-11T13:31:30.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:29 smithi107 ceph-mon[33178]: from='client.35414 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi057-78571-2"}]: dispatch 2024-06-11T13:31:30.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:29 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2194111342' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:30.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:29 smithi107 ceph-mon[33178]: from='client.25279 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi057-77506-10"}]: dispatch 2024-06-11T13:31:30.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:29 smithi107 ceph-mon[33178]: from='client.35402 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi057-77874-12"}]: dispatch 2024-06-11T13:31:30.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:29 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:30.845 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: Running main() from gmock_main.cc 2024-06-11T13:31:30.846 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: [==========] Running 11 tests from 2 test suites. 2024-06-11T13:31:30.846 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: [----------] Global test environment set-up. 2024-06-11T13:31:30.846 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify 2024-06-11T13:31:30.846 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify 2024-06-11T13:31:30.846 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: watch_notify_test_cb 2024-06-11T13:31:30.846 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify (230 ms) 2024-06-11T13:31:30.846 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch2Delete 2024-06-11T13:31:30.846 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94562487564912 err -107 2024-06-11T13:31:30.846 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-06-11T13:31:30.846 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch2Delete (22 ms) 2024-06-11T13:31:30.846 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete 2024-06-11T13:31:30.846 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-06-11T13:31:30.847 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94562487589024 err -107 2024-06-11T13:31:30.847 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete (1010 ms) 2024-06-11T13:31:30.847 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2 2024-06-11T13:31:30.847 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: watch_notify2_test_cb from 24763 notify_id 330712481792 cookie 94562487588960 2024-06-11T13:31:30.847 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2 (8 ms) 2024-06-11T13:31:30.847 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchNotify2 2024-06-11T13:31:30.847 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: watch_notify2_test_cb from 24763 notify_id 330712481792 cookie 94562487611440 2024-06-11T13:31:30.847 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchNotify2 (34 ms) 2024-06-11T13:31:30.847 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioNotify 2024-06-11T13:31:30.847 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: watch_notify2_test_cb from 24763 notify_id 330712481794 cookie 94562487611440 2024-06-11T13:31:30.847 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioNotify (6 ms) 2024-06-11T13:31:30.847 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Multi 2024-06-11T13:31:30.847 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: watch_notify2_test_cb from 24763 notify_id 330712481793 cookie 94562487632336 2024-06-11T13:31:30.848 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: watch_notify2_test_cb from 24763 notify_id 330712481793 cookie 94562487654624 2024-06-11T13:31:30.848 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Multi (7 ms) 2024-06-11T13:31:30.848 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Timeout 2024-06-11T13:31:30.848 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: watch_notify2_test_cb from 24763 notify_id 330712481792 cookie 94562487656752 2024-06-11T13:31:30.848 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: watch_notify2_test_cb from 24763 notify_id 335007449089 cookie 94562487656752 2024-06-11T13:31:30.848 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Timeout (3061 ms) 2024-06-11T13:31:30.848 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch3Timeout 2024-06-11T13:31:30.848 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: waiting up to 1024 for osd to time us out ... 2024-06-11T13:31:30.848 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94562487656752 err -107 2024-06-11T13:31:30.848 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: watch_notify2_test_cb from 24763 notify_id 356482285573 cookie 94562487656752 2024-06-11T13:31:30.848 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch3Timeout (5034 ms) 2024-06-11T13:31:30.848 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete2 2024-06-11T13:31:30.848 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: waiting up to 30 for disconnect notification ... 2024-06-11T13:31:30.848 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94562487656752 err -107 2024-06-11T13:31:30.848 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete2 (1005 ms) 2024-06-11T13:31:30.849 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify (10417 ms total) 2024-06-11T13:31:30.849 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: 2024-06-11T13:31:30.849 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC 2024-06-11T13:31:30.849 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotifyEC.WatchNotify 2024-06-11T13:31:30.849 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: watch_notify_test_cb 2024-06-11T13:31:30.849 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: [ OK ] LibRadosWatchNotifyEC.WatchNotify (2798 ms) 2024-06-11T13:31:30.849 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC (2798 ms total) 2024-06-11T13:31:30.849 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: 2024-06-11T13:31:30.849 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: [----------] Global test environment tear-down 2024-06-11T13:31:30.849 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: [==========] 11 tests from 2 test suites ran. (47039 ms total) 2024-06-11T13:31:30.849 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify: [ PASSED ] 11 tests. 2024-06-11T13:31:31.118 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:30 smithi107 ceph-mon[33178]: pgmap v109: 848 pgs: 192 unknown, 72 creating+peering, 584 active+clean; 145 MiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 3.3 KiB/s wr, 3 op/s 2024-06-11T13:31:31.118 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:30 smithi107 ceph-mon[33178]: from='client.35414 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi057-78571-2"}]': finished 2024-06-11T13:31:31.118 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:30 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2194111342' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:31.118 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:30 smithi107 ceph-mon[33178]: from='client.25279 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi057-77506-10"}]': finished 2024-06-11T13:31:31.118 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:30 smithi107 ceph-mon[33178]: from='client.35402 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi057-77874-12"}]': finished 2024-06-11T13:31:31.118 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:30 smithi107 ceph-mon[33178]: osdmap e98: 8 total, 8 up, 8 in 2024-06-11T13:31:31.118 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:30 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3799830007' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi057-77874-12"}]: dispatch 2024-06-11T13:31:31.118 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:30 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2475936860' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi057-77799-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:31.118 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:30 smithi107 ceph-mon[33178]: from='client.35402 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi057-77874-12"}]: dispatch 2024-06-11T13:31:31.118 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:30 smithi107 ceph-mon[33178]: from='client.35528 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi057-77799-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:31.119 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:30 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3998606324' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi057-78571-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:31.119 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:30 smithi107 ceph-mon[33178]: from='client.35534 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi057-78571-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:31.119 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:30 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:31.119 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:30 smithi107 ceph-mon[33178]: pgmap v112: 712 pgs: 288 unknown, 424 active+clean; 145 MiB data, 774 MiB used, 714 GiB / 715 GiB avail 2024-06-11T13:31:31.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:30 smithi057 ceph-mon[29805]: pgmap v109: 848 pgs: 192 unknown, 72 creating+peering, 584 active+clean; 145 MiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 3.3 KiB/s wr, 3 op/s 2024-06-11T13:31:31.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:30 smithi057 ceph-mon[29805]: from='client.35414 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi057-78571-2"}]': finished 2024-06-11T13:31:31.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:30 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2194111342' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:31.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:30 smithi057 ceph-mon[29805]: from='client.25279 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi057-77506-10"}]': finished 2024-06-11T13:31:31.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:30 smithi057 ceph-mon[29805]: from='client.35402 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi057-77874-12"}]': finished 2024-06-11T13:31:31.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:30 smithi057 ceph-mon[29805]: osdmap e98: 8 total, 8 up, 8 in 2024-06-11T13:31:31.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:30 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3799830007' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi057-77874-12"}]: dispatch 2024-06-11T13:31:31.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:30 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2475936860' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi057-77799-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:31.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:30 smithi057 ceph-mon[29805]: from='client.35402 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi057-77874-12"}]: dispatch 2024-06-11T13:31:31.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:30 smithi057 ceph-mon[29805]: from='client.35528 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi057-77799-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:31.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:30 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3998606324' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi057-78571-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:31.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:30 smithi057 ceph-mon[29805]: from='client.35534 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi057-78571-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:31.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:30 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:31.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:30 smithi057 ceph-mon[29805]: pgmap v112: 712 pgs: 288 unknown, 424 active+clean; 145 MiB data, 774 MiB used, 714 GiB / 715 GiB avail 2024-06-11T13:31:31.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:30 smithi057 ceph-mon[40588]: pgmap v109: 848 pgs: 192 unknown, 72 creating+peering, 584 active+clean; 145 MiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 3.3 KiB/s wr, 3 op/s 2024-06-11T13:31:31.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:30 smithi057 ceph-mon[40588]: from='client.35414 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi057-78571-2"}]': finished 2024-06-11T13:31:31.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:30 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2194111342' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:31.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:30 smithi057 ceph-mon[40588]: from='client.25279 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi057-77506-10"}]': finished 2024-06-11T13:31:31.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:30 smithi057 ceph-mon[40588]: from='client.35402 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi057-77874-12"}]': finished 2024-06-11T13:31:31.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:30 smithi057 ceph-mon[40588]: osdmap e98: 8 total, 8 up, 8 in 2024-06-11T13:31:31.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:30 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3799830007' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi057-77874-12"}]: dispatch 2024-06-11T13:31:31.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:30 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2475936860' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi057-77799-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:31.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:30 smithi057 ceph-mon[40588]: from='client.35402 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi057-77874-12"}]: dispatch 2024-06-11T13:31:31.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:30 smithi057 ceph-mon[40588]: from='client.35528 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi057-77799-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:31.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:30 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3998606324' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi057-78571-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:31.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:30 smithi057 ceph-mon[40588]: from='client.35534 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi057-78571-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:31.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:30 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:31.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:30 smithi057 ceph-mon[40588]: pgmap v112: 712 pgs: 288 unknown, 424 active+clean; 145 MiB data, 774 MiB used, 714 GiB / 715 GiB avail 2024-06-11T13:31:31.547 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:31:31 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:31:32.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:32 smithi057 ceph-mon[29805]: Health check update: 10 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:31:32.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:32 smithi057 ceph-mon[29805]: from='client.35402 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi057-77874-12"}]': finished 2024-06-11T13:31:32.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:32 smithi057 ceph-mon[29805]: from='client.35528 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi057-77799-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:32.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:32 smithi057 ceph-mon[29805]: from='client.35534 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi057-78571-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:32.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:32 smithi057 ceph-mon[29805]: osdmap e99: 8 total, 8 up, 8 in 2024-06-11T13:31:32.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:32 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3094030839' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi057-77362-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:32.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:32 smithi057 ceph-mon[29805]: from='client.25558 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi057-77362-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:32.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:32 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3998606324' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi057-78571-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi057-78571-3"}]: dispatch 2024-06-11T13:31:32.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:32 smithi057 ceph-mon[29805]: from='client.35534 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi057-78571-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi057-78571-3"}]: dispatch 2024-06-11T13:31:32.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:32 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:32.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:32 smithi057 ceph-mon[40588]: Health check update: 10 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:31:32.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:32 smithi057 ceph-mon[40588]: from='client.35402 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi057-77874-12"}]': finished 2024-06-11T13:31:32.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:32 smithi057 ceph-mon[40588]: from='client.35528 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi057-77799-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:32.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:32 smithi057 ceph-mon[40588]: from='client.35534 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi057-78571-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:32.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:32 smithi057 ceph-mon[40588]: osdmap e99: 8 total, 8 up, 8 in 2024-06-11T13:31:32.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:32 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3094030839' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi057-77362-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:32.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:32 smithi057 ceph-mon[40588]: from='client.25558 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi057-77362-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:32.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:32 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3998606324' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi057-78571-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi057-78571-3"}]: dispatch 2024-06-11T13:31:32.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:32 smithi057 ceph-mon[40588]: from='client.35534 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi057-78571-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi057-78571-3"}]: dispatch 2024-06-11T13:31:32.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:32 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:32.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:32 smithi107 ceph-mon[33178]: Health check update: 10 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:31:32.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:32 smithi107 ceph-mon[33178]: from='client.35402 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi057-77874-12"}]': finished 2024-06-11T13:31:32.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:32 smithi107 ceph-mon[33178]: from='client.35528 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi057-77799-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:32.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:32 smithi107 ceph-mon[33178]: from='client.35534 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi057-78571-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:32.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:32 smithi107 ceph-mon[33178]: osdmap e99: 8 total, 8 up, 8 in 2024-06-11T13:31:32.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:32 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3094030839' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi057-77362-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:32.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:32 smithi107 ceph-mon[33178]: from='client.25558 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi057-77362-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:32.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:32 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3998606324' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi057-78571-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi057-78571-3"}]: dispatch 2024-06-11T13:31:32.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:32 smithi107 ceph-mon[33178]: from='client.35534 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi057-78571-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi057-78571-3"}]: dispatch 2024-06-11T13:31:32.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:32 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:33.783 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:31:33 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:31:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:31:34.533 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:34 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:31:34.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:34 smithi057 ceph-mon[29805]: from='client.25558 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi057-77362-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:34.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:34 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:31:34.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:34 smithi057 ceph-mon[40588]: from='client.25558 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi057-77362-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:34.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:34 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:31:34.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:34 smithi107 ceph-mon[33178]: from='client.25558 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi057-77362-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:36.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:36.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: pgmap v115: 616 pgs: 32 creating+peering, 96 unknown, 488 active+clean; 145 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 2.5 KiB/s rd, 3 op/s 2024-06-11T13:31:36.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.0"}]: dispatch 2024-06-11T13:31:36.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.0"}]: dispatch 2024-06-11T13:31:36.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.1"}]: dispatch 2024-06-11T13:31:36.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.1"}]: dispatch 2024-06-11T13:31:36.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.2"}]: dispatch 2024-06-11T13:31:36.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.2"}]: dispatch 2024-06-11T13:31:36.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.3"}]: dispatch 2024-06-11T13:31:36.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.3"}]: dispatch 2024-06-11T13:31:36.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.4"}]: dispatch 2024-06-11T13:31:36.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.4"}]: dispatch 2024-06-11T13:31:36.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.5"}]: dispatch 2024-06-11T13:31:36.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.5"}]: dispatch 2024-06-11T13:31:36.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.6"}]: dispatch 2024-06-11T13:31:36.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.6"}]: dispatch 2024-06-11T13:31:36.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.7"}]: dispatch 2024-06-11T13:31:36.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.7"}]: dispatch 2024-06-11T13:31:36.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.8"}]: dispatch 2024-06-11T13:31:36.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.8"}]: dispatch 2024-06-11T13:31:36.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.9"}]: dispatch 2024-06-11T13:31:36.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.9"}]: dispatch 2024-06-11T13:31:36.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: 19.3 deep-scrub starts 2024-06-11T13:31:36.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: 19.3 deep-scrub ok 2024-06-11T13:31:36.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1065579565' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi057-77351-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:36.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: 19.8 deep-scrub starts 2024-06-11T13:31:36.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: 19.8 deep-scrub ok 2024-06-11T13:31:36.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: 19.0 deep-scrub starts 2024-06-11T13:31:36.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: 19.0 deep-scrub ok 2024-06-11T13:31:36.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:36.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: 19.5 deep-scrub starts 2024-06-11T13:31:36.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: 19.5 deep-scrub ok 2024-06-11T13:31:36.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: 19.7 deep-scrub starts 2024-06-11T13:31:36.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: 19.7 deep-scrub ok 2024-06-11T13:31:36.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: 19.2 deep-scrub starts 2024-06-11T13:31:36.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: 19.2 deep-scrub ok 2024-06-11T13:31:36.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: 19.1 deep-scrub starts 2024-06-11T13:31:36.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: 19.1 deep-scrub ok 2024-06-11T13:31:36.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: osdmap e100: 8 total, 8 up, 8 in 2024-06-11T13:31:36.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/302765405' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:36.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: from='client.35534 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi057-78571-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi057-78571-3"}]': finished 2024-06-11T13:31:36.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: osdmap e101: 8 total, 8 up, 8 in 2024-06-11T13:31:36.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/197838560' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:36.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:36.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: from='client.35570 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi057-77351-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:36.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: from='client.25588 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:36.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:36.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:36.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: 19.6 deep-scrub starts 2024-06-11T13:31:36.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: 19.6 deep-scrub ok 2024-06-11T13:31:36.288 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: 19.9 deep-scrub starts 2024-06-11T13:31:36.288 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[40588]: 19.9 deep-scrub ok 2024-06-11T13:31:36.288 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:36.288 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: pgmap v115: 616 pgs: 32 creating+peering, 96 unknown, 488 active+clean; 145 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 2.5 KiB/s rd, 3 op/s 2024-06-11T13:31:36.288 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.0"}]: dispatch 2024-06-11T13:31:36.288 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.0"}]: dispatch 2024-06-11T13:31:36.289 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.1"}]: dispatch 2024-06-11T13:31:36.289 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.1"}]: dispatch 2024-06-11T13:31:36.289 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.2"}]: dispatch 2024-06-11T13:31:36.289 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.2"}]: dispatch 2024-06-11T13:31:36.289 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.3"}]: dispatch 2024-06-11T13:31:36.289 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.3"}]: dispatch 2024-06-11T13:31:36.289 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.4"}]: dispatch 2024-06-11T13:31:36.289 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.4"}]: dispatch 2024-06-11T13:31:36.289 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.5"}]: dispatch 2024-06-11T13:31:36.289 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.5"}]: dispatch 2024-06-11T13:31:36.289 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.6"}]: dispatch 2024-06-11T13:31:36.289 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.6"}]: dispatch 2024-06-11T13:31:36.289 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.7"}]: dispatch 2024-06-11T13:31:36.289 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.7"}]: dispatch 2024-06-11T13:31:36.290 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.8"}]: dispatch 2024-06-11T13:31:36.290 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.8"}]: dispatch 2024-06-11T13:31:36.290 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.9"}]: dispatch 2024-06-11T13:31:36.290 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.9"}]: dispatch 2024-06-11T13:31:36.290 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: 19.3 deep-scrub starts 2024-06-11T13:31:36.290 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: 19.3 deep-scrub ok 2024-06-11T13:31:36.290 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1065579565' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi057-77351-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:36.290 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: 19.8 deep-scrub starts 2024-06-11T13:31:36.290 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: 19.8 deep-scrub ok 2024-06-11T13:31:36.290 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: 19.0 deep-scrub starts 2024-06-11T13:31:36.290 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: 19.0 deep-scrub ok 2024-06-11T13:31:36.291 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:36.291 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: 19.5 deep-scrub starts 2024-06-11T13:31:36.291 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: 19.5 deep-scrub ok 2024-06-11T13:31:36.291 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: 19.7 deep-scrub starts 2024-06-11T13:31:36.291 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: 19.7 deep-scrub ok 2024-06-11T13:31:36.291 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: 19.2 deep-scrub starts 2024-06-11T13:31:36.291 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: 19.2 deep-scrub ok 2024-06-11T13:31:36.291 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: 19.1 deep-scrub starts 2024-06-11T13:31:36.291 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: 19.1 deep-scrub ok 2024-06-11T13:31:36.291 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: osdmap e100: 8 total, 8 up, 8 in 2024-06-11T13:31:36.291 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/302765405' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:36.291 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: from='client.35534 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi057-78571-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi057-78571-3"}]': finished 2024-06-11T13:31:36.291 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: osdmap e101: 8 total, 8 up, 8 in 2024-06-11T13:31:36.291 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/197838560' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:36.292 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:36.292 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: from='client.35570 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi057-77351-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:36.292 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: from='client.25588 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:36.292 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:36.292 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:36.292 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: 19.6 deep-scrub starts 2024-06-11T13:31:36.292 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: 19.6 deep-scrub ok 2024-06-11T13:31:36.292 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: 19.9 deep-scrub starts 2024-06-11T13:31:36.292 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:36 smithi057 ceph-mon[29805]: 19.9 deep-scrub ok 2024-06-11T13:31:36.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:36.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: pgmap v115: 616 pgs: 32 creating+peering, 96 unknown, 488 active+clean; 145 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 2.5 KiB/s rd, 3 op/s 2024-06-11T13:31:36.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.0"}]: dispatch 2024-06-11T13:31:36.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.0"}]: dispatch 2024-06-11T13:31:36.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.1"}]: dispatch 2024-06-11T13:31:36.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.1"}]: dispatch 2024-06-11T13:31:36.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.2"}]: dispatch 2024-06-11T13:31:36.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.2"}]: dispatch 2024-06-11T13:31:36.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.3"}]: dispatch 2024-06-11T13:31:36.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.3"}]: dispatch 2024-06-11T13:31:36.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.4"}]: dispatch 2024-06-11T13:31:36.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.4"}]: dispatch 2024-06-11T13:31:36.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.5"}]: dispatch 2024-06-11T13:31:36.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.5"}]: dispatch 2024-06-11T13:31:36.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.6"}]: dispatch 2024-06-11T13:31:36.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.6"}]: dispatch 2024-06-11T13:31:36.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.7"}]: dispatch 2024-06-11T13:31:36.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.7"}]: dispatch 2024-06-11T13:31:36.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.8"}]: dispatch 2024-06-11T13:31:36.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.8"}]: dispatch 2024-06-11T13:31:36.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.9"}]: dispatch 2024-06-11T13:31:36.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.9"}]: dispatch 2024-06-11T13:31:36.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: 19.3 deep-scrub starts 2024-06-11T13:31:36.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: 19.3 deep-scrub ok 2024-06-11T13:31:36.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1065579565' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi057-77351-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:36.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: 19.8 deep-scrub starts 2024-06-11T13:31:36.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: 19.8 deep-scrub ok 2024-06-11T13:31:36.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: 19.0 deep-scrub starts 2024-06-11T13:31:36.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: 19.0 deep-scrub ok 2024-06-11T13:31:36.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:36.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: 19.5 deep-scrub starts 2024-06-11T13:31:36.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: 19.5 deep-scrub ok 2024-06-11T13:31:36.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: 19.7 deep-scrub starts 2024-06-11T13:31:36.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: 19.7 deep-scrub ok 2024-06-11T13:31:36.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: 19.2 deep-scrub starts 2024-06-11T13:31:36.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: 19.2 deep-scrub ok 2024-06-11T13:31:36.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: 19.1 deep-scrub starts 2024-06-11T13:31:36.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: 19.1 deep-scrub ok 2024-06-11T13:31:36.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: osdmap e100: 8 total, 8 up, 8 in 2024-06-11T13:31:36.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/302765405' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:36.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: from='client.35534 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi057-78571-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi057-78571-3"}]': finished 2024-06-11T13:31:36.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: osdmap e101: 8 total, 8 up, 8 in 2024-06-11T13:31:36.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/197838560' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:36.301 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:36.301 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: from='client.35570 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi057-77351-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:36.301 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: from='client.25588 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:36.301 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:36.301 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:36.301 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: 19.6 deep-scrub starts 2024-06-11T13:31:36.301 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: 19.6 deep-scrub ok 2024-06-11T13:31:36.301 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: 19.9 deep-scrub starts 2024-06-11T13:31:36.301 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:36 smithi107 ceph-mon[33178]: 19.9 deep-scrub ok 2024-06-11T13:31:38.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:37 smithi057 ceph-mon[29805]: pgmap v117: 656 pgs: 32 creating+peering, 200 unknown, 424 active+clean; 145 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.6 KiB/s wr, 5 op/s 2024-06-11T13:31:38.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:37 smithi057 ceph-mon[29805]: 19.4 deep-scrub starts 2024-06-11T13:31:38.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:37 smithi057 ceph-mon[29805]: 19.4 deep-scrub ok 2024-06-11T13:31:38.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:37 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:38.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:37 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/302765405' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:38.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:37 smithi057 ceph-mon[29805]: from='client.35570 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi057-77351-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:38.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:37 smithi057 ceph-mon[29805]: from='client.25588 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:38.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:37 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:38.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:37 smithi057 ceph-mon[29805]: osdmap e102: 8 total, 8 up, 8 in 2024-06-11T13:31:38.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:37 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:31:38.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:37 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:31:38.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:37 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:38.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:37 smithi057 ceph-mon[29805]: osdmap e103: 8 total, 8 up, 8 in 2024-06-11T13:31:38.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:37 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1230309837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi057-77362-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:38.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:37 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:38.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:37 smithi057 ceph-mon[29805]: from='client.35591 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi057-77362-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:38.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:37 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:38.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:37 smithi057 ceph-mon[40588]: pgmap v117: 656 pgs: 32 creating+peering, 200 unknown, 424 active+clean; 145 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.6 KiB/s wr, 5 op/s 2024-06-11T13:31:38.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:37 smithi057 ceph-mon[40588]: 19.4 deep-scrub starts 2024-06-11T13:31:38.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:37 smithi057 ceph-mon[40588]: 19.4 deep-scrub ok 2024-06-11T13:31:38.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:37 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:38.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:37 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/302765405' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:38.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:37 smithi057 ceph-mon[40588]: from='client.35570 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi057-77351-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:38.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:37 smithi057 ceph-mon[40588]: from='client.25588 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:38.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:37 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:38.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:37 smithi057 ceph-mon[40588]: osdmap e102: 8 total, 8 up, 8 in 2024-06-11T13:31:38.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:37 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:31:38.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:37 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:31:38.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:37 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:38.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:37 smithi057 ceph-mon[40588]: osdmap e103: 8 total, 8 up, 8 in 2024-06-11T13:31:38.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:37 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1230309837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi057-77362-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:38.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:37 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:38.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:37 smithi057 ceph-mon[40588]: from='client.35591 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi057-77362-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:38.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:37 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:38.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:37 smithi107 ceph-mon[33178]: pgmap v117: 656 pgs: 32 creating+peering, 200 unknown, 424 active+clean; 145 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.6 KiB/s wr, 5 op/s 2024-06-11T13:31:38.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:37 smithi107 ceph-mon[33178]: 19.4 deep-scrub starts 2024-06-11T13:31:38.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:37 smithi107 ceph-mon[33178]: 19.4 deep-scrub ok 2024-06-11T13:31:38.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:37 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:38.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:37 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/302765405' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:38.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:37 smithi107 ceph-mon[33178]: from='client.35570 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi057-77351-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:38.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:37 smithi107 ceph-mon[33178]: from='client.25588 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:38.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:37 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:38.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:37 smithi107 ceph-mon[33178]: osdmap e102: 8 total, 8 up, 8 in 2024-06-11T13:31:38.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:37 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:31:38.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:37 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:31:38.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:37 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:38.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:37 smithi107 ceph-mon[33178]: osdmap e103: 8 total, 8 up, 8 in 2024-06-11T13:31:38.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:37 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1230309837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi057-77362-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:38.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:37 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:38.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:37 smithi107 ceph-mon[33178]: from='client.35591 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi057-77362-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:38.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:37 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:39.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:39 smithi057 ceph-mon[29805]: pgmap v119: 720 pgs: 96 creating+peering, 624 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 13 MiB/s wr, 9 op/s 2024-06-11T13:31:39.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:39 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:39.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:39 smithi057 ceph-mon[29805]: Health check update: 10 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:31:39.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:39 smithi057 ceph-mon[29805]: from='client.35591 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi057-77362-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:39.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:39 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:39.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:39 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3998606324' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi057-78571-3"}]: dispatch 2024-06-11T13:31:39.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:39 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:31:39.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:39 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/197523128' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4"}]: dispatch 2024-06-11T13:31:39.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:39 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:39.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:39 smithi057 ceph-mon[29805]: osdmap e104: 8 total, 8 up, 8 in 2024-06-11T13:31:39.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:39 smithi057 ceph-mon[29805]: from='client.35534 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi057-78571-3"}]: dispatch 2024-06-11T13:31:39.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:39 smithi057 ceph-mon[40588]: pgmap v119: 720 pgs: 96 creating+peering, 624 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 13 MiB/s wr, 9 op/s 2024-06-11T13:31:39.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:39 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:39.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:39 smithi057 ceph-mon[40588]: Health check update: 10 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:31:39.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:39 smithi057 ceph-mon[40588]: from='client.35591 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi057-77362-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:39.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:39 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:39.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:39 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3998606324' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi057-78571-3"}]: dispatch 2024-06-11T13:31:39.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:39 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:31:39.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:39 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/197523128' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4"}]: dispatch 2024-06-11T13:31:39.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:39 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:39.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:39 smithi057 ceph-mon[40588]: osdmap e104: 8 total, 8 up, 8 in 2024-06-11T13:31:39.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:39 smithi057 ceph-mon[40588]: from='client.35534 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi057-78571-3"}]: dispatch 2024-06-11T13:31:39.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:39 smithi107 ceph-mon[33178]: pgmap v119: 720 pgs: 96 creating+peering, 624 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 13 MiB/s wr, 9 op/s 2024-06-11T13:31:39.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:39 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:39.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:39 smithi107 ceph-mon[33178]: Health check update: 10 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:31:39.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:39 smithi107 ceph-mon[33178]: from='client.35591 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi057-77362-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:39.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:39 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:39.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:39 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3998606324' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi057-78571-3"}]: dispatch 2024-06-11T13:31:39.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:39 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:31:39.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:39 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/197523128' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4"}]: dispatch 2024-06-11T13:31:39.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:39 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:39.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:39 smithi107 ceph-mon[33178]: osdmap e104: 8 total, 8 up, 8 in 2024-06-11T13:31:39.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:39 smithi107 ceph-mon[33178]: from='client.35534 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi057-78571-3"}]: dispatch 2024-06-11T13:31:40.639 INFO:tasks.workunit.client.0.smithi057.stdout: ec_list: Running main() from gmock_main.cc 2024-06-11T13:31:40.639 INFO:tasks.workunit.client.0.smithi057.stdout: ec_list: [==========] Running 3 tests from 1 test suite. 2024-06-11T13:31:40.639 INFO:tasks.workunit.client.0.smithi057.stdout: ec_list: [----------] Global test environment set-up. 2024-06-11T13:31:40.639 INFO:tasks.workunit.client.0.smithi057.stdout: ec_list: [----------] 3 tests from NeoradosECList 2024-06-11T13:31:40.639 INFO:tasks.workunit.client.0.smithi057.stdout: ec_list: [ RUN ] NeoradosECList.ListObjects 2024-06-11T13:31:40.640 INFO:tasks.workunit.client.0.smithi057.stdout: ec_list: [ OK ] NeoradosECList.ListObjects (33088 ms) 2024-06-11T13:31:40.640 INFO:tasks.workunit.client.0.smithi057.stdout: ec_list: [ RUN ] NeoradosECList.ListObjectsNS 2024-06-11T13:31:40.640 INFO:tasks.workunit.client.0.smithi057.stdout: ec_list: [ OK ] NeoradosECList.ListObjectsNS (12667 ms) 2024-06-11T13:31:40.640 INFO:tasks.workunit.client.0.smithi057.stdout: ec_list: [ RUN ] NeoradosECList.ListObjectsMany 2024-06-11T13:31:40.640 INFO:tasks.workunit.client.0.smithi057.stdout: ec_list: [ OK ] NeoradosECList.ListObjectsMany (10820 ms) 2024-06-11T13:31:40.640 INFO:tasks.workunit.client.0.smithi057.stdout: ec_list: [----------] 3 tests from NeoradosECList (56575 ms total) 2024-06-11T13:31:40.640 INFO:tasks.workunit.client.0.smithi057.stdout: ec_list: 2024-06-11T13:31:40.640 INFO:tasks.workunit.client.0.smithi057.stdout: ec_list: [----------] Global test environment tear-down 2024-06-11T13:31:40.640 INFO:tasks.workunit.client.0.smithi057.stdout: ec_list: [==========] 3 tests from 1 test suite ran. (56575 ms total) 2024-06-11T13:31:40.640 INFO:tasks.workunit.client.0.smithi057.stdout: ec_list: [ PASSED ] 3 tests. 2024-06-11T13:31:41.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:40 smithi057 ceph-mon[40588]: pgmap v122: 648 pgs: 160 unknown, 64 creating+peering, 424 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:31:41.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:40 smithi057 ceph-mon[40588]: from='client.35363 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4"}]: dispatch 2024-06-11T13:31:41.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:40 smithi057 ceph-mon[40588]: from='client.35534 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi057-78571-3"}]': finished 2024-06-11T13:31:41.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:40 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1154278633' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:41.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:40 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/4128251910' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi057-77351-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:41.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:40 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3998606324' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi057-78571-3"}]: dispatch 2024-06-11T13:31:41.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:40 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1945277826' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi057-77714-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:41.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:40 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:41.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:40 smithi057 ceph-mon[40588]: osdmap e105: 8 total, 8 up, 8 in 2024-06-11T13:31:41.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:40 smithi057 ceph-mon[40588]: from='client.35621 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi057-77351-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:41.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:40 smithi057 ceph-mon[40588]: from='client.35534 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi057-78571-3"}]: dispatch 2024-06-11T13:31:41.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:40 smithi057 ceph-mon[40588]: from='client.35612 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi057-77714-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:41.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:40 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:31:41.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:40 smithi057 ceph-mon[40588]: from='client.25627 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:41.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:40 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:41.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:40 smithi057 ceph-mon[29805]: pgmap v122: 648 pgs: 160 unknown, 64 creating+peering, 424 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:31:41.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:40 smithi057 ceph-mon[29805]: from='client.35363 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4"}]: dispatch 2024-06-11T13:31:41.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:40 smithi057 ceph-mon[29805]: from='client.35534 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi057-78571-3"}]': finished 2024-06-11T13:31:41.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:40 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1154278633' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:41.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:40 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/4128251910' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi057-77351-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:41.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:40 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3998606324' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi057-78571-3"}]: dispatch 2024-06-11T13:31:41.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:40 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1945277826' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi057-77714-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:41.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:40 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:41.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:40 smithi057 ceph-mon[29805]: osdmap e105: 8 total, 8 up, 8 in 2024-06-11T13:31:41.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:40 smithi057 ceph-mon[29805]: from='client.35621 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi057-77351-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:41.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:40 smithi057 ceph-mon[29805]: from='client.35534 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi057-78571-3"}]: dispatch 2024-06-11T13:31:41.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:40 smithi057 ceph-mon[29805]: from='client.35612 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi057-77714-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:41.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:40 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:31:41.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:40 smithi057 ceph-mon[29805]: from='client.25627 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:41.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:40 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:41.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:40 smithi107 ceph-mon[33178]: pgmap v122: 648 pgs: 160 unknown, 64 creating+peering, 424 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:31:41.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:40 smithi107 ceph-mon[33178]: from='client.35363 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4"}]: dispatch 2024-06-11T13:31:41.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:40 smithi107 ceph-mon[33178]: from='client.35534 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi057-78571-3"}]': finished 2024-06-11T13:31:41.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:40 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1154278633' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:41.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:40 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/4128251910' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi057-77351-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:41.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:40 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3998606324' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi057-78571-3"}]: dispatch 2024-06-11T13:31:41.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:40 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1945277826' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi057-77714-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:41.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:40 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:41.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:40 smithi107 ceph-mon[33178]: osdmap e105: 8 total, 8 up, 8 in 2024-06-11T13:31:41.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:40 smithi107 ceph-mon[33178]: from='client.35621 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi057-77351-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:41.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:40 smithi107 ceph-mon[33178]: from='client.35534 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi057-78571-3"}]: dispatch 2024-06-11T13:31:41.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:40 smithi107 ceph-mon[33178]: from='client.35612 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi057-77714-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:41.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:40 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:31:41.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:40 smithi107 ceph-mon[33178]: from='client.25627 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:41.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:40 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:41.547 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:31:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:31:42.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:42 smithi057 ceph-mon[29805]: pgmap v124: 680 pgs: 224 unknown, 32 creating+peering, 424 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-06-11T13:31:42.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:42 smithi057 ceph-mon[29805]: from='client.35363 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4"}]': finished 2024-06-11T13:31:42.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:42 smithi057 ceph-mon[29805]: from='client.35621 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi057-77351-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:42.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:42 smithi057 ceph-mon[29805]: from='client.35534 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi057-78571-3"}]': finished 2024-06-11T13:31:42.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:42 smithi057 ceph-mon[29805]: from='client.35612 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi057-77714-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:42.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:42 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-7", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:31:42.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:42 smithi057 ceph-mon[29805]: from='client.25627 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:42.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:42 smithi057 ceph-mon[29805]: osdmap e106: 8 total, 8 up, 8 in 2024-06-11T13:31:42.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:42 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/197523128' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4", "tierpool": "test-rados-api-smithi057-77882-6"}]: dispatch 2024-06-11T13:31:42.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:42 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-7"}]: dispatch 2024-06-11T13:31:42.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:42 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/4086579560' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:42.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:42 smithi057 ceph-mon[29805]: from='client.35363 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4", "tierpool": "test-rados-api-smithi057-77882-6"}]: dispatch 2024-06-11T13:31:42.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:42 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-7"}]: dispatch 2024-06-11T13:31:42.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:42 smithi057 ceph-mon[29805]: from='client.25651 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:42.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:42 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:42.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:42 smithi057 ceph-mon[40588]: pgmap v124: 680 pgs: 224 unknown, 32 creating+peering, 424 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-06-11T13:31:42.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:42 smithi057 ceph-mon[40588]: from='client.35363 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4"}]': finished 2024-06-11T13:31:42.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:42 smithi057 ceph-mon[40588]: from='client.35621 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi057-77351-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:42.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:42 smithi057 ceph-mon[40588]: from='client.35534 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi057-78571-3"}]': finished 2024-06-11T13:31:42.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:42 smithi057 ceph-mon[40588]: from='client.35612 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi057-77714-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:42.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:42 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-7", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:31:42.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:42 smithi057 ceph-mon[40588]: from='client.25627 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:42.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:42 smithi057 ceph-mon[40588]: osdmap e106: 8 total, 8 up, 8 in 2024-06-11T13:31:42.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:42 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/197523128' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4", "tierpool": "test-rados-api-smithi057-77882-6"}]: dispatch 2024-06-11T13:31:42.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:42 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-7"}]: dispatch 2024-06-11T13:31:42.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:42 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/4086579560' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:42.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:42 smithi057 ceph-mon[40588]: from='client.35363 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4", "tierpool": "test-rados-api-smithi057-77882-6"}]: dispatch 2024-06-11T13:31:42.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:42 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-7"}]: dispatch 2024-06-11T13:31:42.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:42 smithi057 ceph-mon[40588]: from='client.25651 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:42.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:42 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:42.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:42 smithi107 ceph-mon[33178]: pgmap v124: 680 pgs: 224 unknown, 32 creating+peering, 424 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-06-11T13:31:42.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:42 smithi107 ceph-mon[33178]: from='client.35363 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4"}]': finished 2024-06-11T13:31:42.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:42 smithi107 ceph-mon[33178]: from='client.35621 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi057-77351-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:42.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:42 smithi107 ceph-mon[33178]: from='client.35534 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi057-78571-3"}]': finished 2024-06-11T13:31:42.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:42 smithi107 ceph-mon[33178]: from='client.35612 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi057-77714-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:42.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:42 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-7", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:31:42.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:42 smithi107 ceph-mon[33178]: from='client.25627 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:42.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:42 smithi107 ceph-mon[33178]: osdmap e106: 8 total, 8 up, 8 in 2024-06-11T13:31:42.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:42 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/197523128' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4", "tierpool": "test-rados-api-smithi057-77882-6"}]: dispatch 2024-06-11T13:31:42.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:42 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-7"}]: dispatch 2024-06-11T13:31:42.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:42 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/4086579560' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:42.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:42 smithi107 ceph-mon[33178]: from='client.35363 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4", "tierpool": "test-rados-api-smithi057-77882-6"}]: dispatch 2024-06-11T13:31:42.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:42 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-7"}]: dispatch 2024-06-11T13:31:42.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:42 smithi107 ceph-mon[33178]: from='client.25651 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:42.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:42 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:44.033 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:31:43 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:31:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:31:44.417 INFO:tasks.workunit.client.0.smithi057.stdout:tch_notify_pp: flushed 2024-06-11T13:31:44.417 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 (3004 ms) 2024-06-11T13:31:44.417 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 2024-06-11T13:31:44.417 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: trying... 2024-06-11T13:31:44.418 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: handle_notify cookie 94514182267952 notify_id 416611827715 notifier_gid 35363 2024-06-11T13:31:44.418 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: timed out 2024-06-11T13:31:44.418 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: flushing 2024-06-11T13:31:44.418 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: flushed 2024-06-11T13:31:44.418 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 (3004 ms) 2024-06-11T13:31:44.418 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 2024-06-11T13:31:44.418 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: List watches 2024-06-11T13:31:44.418 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: notify2 2024-06-11T13:31:44.418 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: handle_notify cookie 94514182267952 notify_id 425201762312 notifier_gid 35363 2024-06-11T13:31:44.418 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: notify2 done 2024-06-11T13:31:44.418 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: watch_check 2024-06-11T13:31:44.418 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: unwatch2 2024-06-11T13:31:44.419 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: flushing 2024-06-11T13:31:44.419 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: done 2024-06-11T13:31:44.419 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 (3007 ms) 2024-06-11T13:31:44.419 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 2024-06-11T13:31:44.419 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: List watches 2024-06-11T13:31:44.419 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: notify2 2024-06-11T13:31:44.419 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: handle_notify cookie 94514182265616 notify_id 433791696902 notifier_gid 35363 2024-06-11T13:31:44.419 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: notify2 done 2024-06-11T13:31:44.419 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: watch_check 2024-06-11T13:31:44.419 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: unwatch2 2024-06-11T13:31:44.419 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: flushing 2024-06-11T13:31:44.420 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: done 2024-06-11T13:31:44.420 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 (3005 ms) 2024-06-11T13:31:44.420 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP (20968 ms total) 2024-06-11T13:31:44.420 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: 2024-06-11T13:31:44.420 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: [----------] Global test environment tear-down 2024-06-11T13:31:44.420 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: [==========] 16 tests from 2 test suites ran. (60630 ms total) 2024-06-11T13:31:44.420 INFO:tasks.workunit.client.0.smithi057.stdout: api_watch_notify_pp: [ PASSED ] 16 tests. 2024-06-11T13:31:44.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:44 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:31:44.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:44 smithi057 ceph-mon[29805]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:31:44.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:44 smithi057 ceph-mon[29805]: from='client.35363 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4", "tierpool": "test-rados-api-smithi057-77882-6"}]': finished 2024-06-11T13:31:44.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:44 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-7"}]': finished 2024-06-11T13:31:44.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:44 smithi057 ceph-mon[29805]: from='client.25651 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:44.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:44 smithi057 ceph-mon[29805]: osdmap e107: 8 total, 8 up, 8 in 2024-06-11T13:31:44.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:44 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/197523128' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi057-77882-6", "pool2": "test-rados-api-smithi057-77882-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-11T13:31:44.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:44 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3749870526' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi057-77362-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:44.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:44 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:31:44.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:44 smithi057 ceph-mon[29805]: from='client.35363 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi057-77882-6", "pool2": "test-rados-api-smithi057-77882-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-11T13:31:44.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:44 smithi057 ceph-mon[29805]: from='client.35642 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi057-77362-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:44.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:44 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:31:44.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:44 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:44.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:44 smithi057 ceph-mon[29805]: pgmap v127: 680 pgs: 32 unknown, 120 creating+peering, 528 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 685 B/s wr, 2 op/s 2024-06-11T13:31:44.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:44 smithi057 ceph-mon[29805]: from='client.35363 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi057-77882-6", "pool2": "test-rados-api-smithi057-77882-6", "yes_i_really_really_mean_it": true}]': finished 2024-06-11T13:31:44.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:44 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:31:44.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:44 smithi057 ceph-mon[40588]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:31:44.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:44 smithi057 ceph-mon[40588]: from='client.35363 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4", "tierpool": "test-rados-api-smithi057-77882-6"}]': finished 2024-06-11T13:31:44.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:44 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-7"}]': finished 2024-06-11T13:31:44.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:44 smithi057 ceph-mon[40588]: from='client.25651 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:44.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:44 smithi057 ceph-mon[40588]: osdmap e107: 8 total, 8 up, 8 in 2024-06-11T13:31:44.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:44 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/197523128' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi057-77882-6", "pool2": "test-rados-api-smithi057-77882-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-11T13:31:44.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:44 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3749870526' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi057-77362-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:44.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:44 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:31:44.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:44 smithi057 ceph-mon[40588]: from='client.35363 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi057-77882-6", "pool2": "test-rados-api-smithi057-77882-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-11T13:31:44.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:44 smithi057 ceph-mon[40588]: from='client.35642 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi057-77362-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:44.787 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:44 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:31:44.787 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:44 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:44.787 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:44 smithi057 ceph-mon[40588]: pgmap v127: 680 pgs: 32 unknown, 120 creating+peering, 528 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 685 B/s wr, 2 op/s 2024-06-11T13:31:44.787 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:44 smithi057 ceph-mon[40588]: from='client.35363 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi057-77882-6", "pool2": "test-rados-api-smithi057-77882-6", "yes_i_really_really_mean_it": true}]': finished 2024-06-11T13:31:44.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:44 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:31:44.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:44 smithi107 ceph-mon[33178]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:31:44.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:44 smithi107 ceph-mon[33178]: from='client.35363 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi057-77882-4", "tierpool": "test-rados-api-smithi057-77882-6"}]': finished 2024-06-11T13:31:44.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:44 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-7"}]': finished 2024-06-11T13:31:44.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:44 smithi107 ceph-mon[33178]: from='client.25651 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:44.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:44 smithi107 ceph-mon[33178]: osdmap e107: 8 total, 8 up, 8 in 2024-06-11T13:31:44.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:44 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/197523128' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi057-77882-6", "pool2": "test-rados-api-smithi057-77882-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-11T13:31:44.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:44 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3749870526' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi057-77362-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:44.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:44 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:31:44.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:44 smithi107 ceph-mon[33178]: from='client.35363 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi057-77882-6", "pool2": "test-rados-api-smithi057-77882-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-11T13:31:44.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:44 smithi107 ceph-mon[33178]: from='client.35642 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi057-77362-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:44.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:44 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:31:44.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:44 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:44.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:44 smithi107 ceph-mon[33178]: pgmap v127: 680 pgs: 32 unknown, 120 creating+peering, 528 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 685 B/s wr, 2 op/s 2024-06-11T13:31:44.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:44 smithi107 ceph-mon[33178]: from='client.35363 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi057-77882-6", "pool2": "test-rados-api-smithi057-77882-6", "yes_i_really_really_mean_it": true}]': finished 2024-06-11T13:31:45.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:45 smithi057 ceph-mon[29805]: from='client.35642 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi057-77362-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:45.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:45 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:31:45.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:45 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-7"}]: dispatch 2024-06-11T13:31:45.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:45 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:45.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:45 smithi057 ceph-mon[29805]: osdmap e108: 8 total, 8 up, 8 in 2024-06-11T13:31:45.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:45 smithi057 ceph-mon[29805]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:31:45.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:45 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-7"}]: dispatch 2024-06-11T13:31:45.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:45 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:45.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:45 smithi057 ceph-mon[29805]: pgmap v129: 644 pgs: 128 unknown, 40 creating+peering, 476 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-06-11T13:31:45.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:45 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-7"}]': finished 2024-06-11T13:31:45.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:45 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1903952725' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi057-77351-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:45.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:45 smithi057 ceph-mon[29805]: osdmap e109: 8 total, 8 up, 8 in 2024-06-11T13:31:45.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:45 smithi057 ceph-mon[29805]: from='client.35681 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi057-77351-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:45.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:45 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:45.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:45 smithi057 ceph-mon[40588]: from='client.35642 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi057-77362-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:45.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:45 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:31:45.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:45 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-7"}]: dispatch 2024-06-11T13:31:45.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:45 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:45.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:45 smithi057 ceph-mon[40588]: osdmap e108: 8 total, 8 up, 8 in 2024-06-11T13:31:45.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:45 smithi057 ceph-mon[40588]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:31:45.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:45 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-7"}]: dispatch 2024-06-11T13:31:45.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:45 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:45.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:45 smithi057 ceph-mon[40588]: pgmap v129: 644 pgs: 128 unknown, 40 creating+peering, 476 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-06-11T13:31:45.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:45 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-7"}]': finished 2024-06-11T13:31:45.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:45 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1903952725' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi057-77351-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:45.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:45 smithi057 ceph-mon[40588]: osdmap e109: 8 total, 8 up, 8 in 2024-06-11T13:31:45.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:45 smithi057 ceph-mon[40588]: from='client.35681 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi057-77351-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:45.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:45 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:45.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:45 smithi107 ceph-mon[33178]: from='client.35642 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi057-77362-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:45.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:45 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:31:45.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:45 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-7"}]: dispatch 2024-06-11T13:31:45.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:45 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:45.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:45 smithi107 ceph-mon[33178]: osdmap e108: 8 total, 8 up, 8 in 2024-06-11T13:31:45.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:45 smithi107 ceph-mon[33178]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:31:45.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:45 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-7"}]: dispatch 2024-06-11T13:31:45.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:45 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:45.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:45 smithi107 ceph-mon[33178]: pgmap v129: 644 pgs: 128 unknown, 40 creating+peering, 476 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-06-11T13:31:45.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:45 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-7"}]': finished 2024-06-11T13:31:45.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:45 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1903952725' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi057-77351-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:45.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:45 smithi107 ceph-mon[33178]: osdmap e109: 8 total, 8 up, 8 in 2024-06-11T13:31:45.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:45 smithi107 ceph-mon[33178]: from='client.35681 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi057-77351-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:45.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:45 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:47.283 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:46 smithi057 ceph-mon[40588]: from='client.35681 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi057-77351-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:47.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:46 smithi057 ceph-mon[40588]: osdmap e110: 8 total, 8 up, 8 in 2024-06-11T13:31:47.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:46 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/125888143' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:47.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:46 smithi057 ceph-mon[40588]: from='client.35678 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:47.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:46 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:47.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:46 smithi057 ceph-mon[29805]: from='client.35681 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi057-77351-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:47.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:46 smithi057 ceph-mon[29805]: osdmap e110: 8 total, 8 up, 8 in 2024-06-11T13:31:47.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:46 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/125888143' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:47.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:46 smithi057 ceph-mon[29805]: from='client.35678 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:47.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:46 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:47.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:46 smithi107 ceph-mon[33178]: from='client.35681 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi057-77351-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:47.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:46 smithi107 ceph-mon[33178]: osdmap e110: 8 total, 8 up, 8 in 2024-06-11T13:31:47.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:46 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/125888143' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:47.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:46 smithi107 ceph-mon[33178]: from='client.35678 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:47.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:46 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:47.855 INFO:tasks.workunit.client.0.smithi057.stdout: write_operations: Running main() from gmock_main.cc 2024-06-11T13:31:47.855 INFO:tasks.workunit.client.0.smithi057.stdout: write_operations: [==========] Running 7 tests from 1 test suite. 2024-06-11T13:31:47.855 INFO:tasks.workunit.client.0.smithi057.stdout: write_operations: [----------] Global test environment set-up. 2024-06-11T13:31:47.855 INFO:tasks.workunit.client.0.smithi057.stdout: write_operations: [----------] 7 tests from NeoRadosWriteOps 2024-06-11T13:31:47.855 INFO:tasks.workunit.client.0.smithi057.stdout: write_operations: [ RUN ] NeoRadosWriteOps.AssertExists 2024-06-11T13:31:47.855 INFO:tasks.workunit.client.0.smithi057.stdout: write_operations: [ OK ] NeoRadosWriteOps.AssertExists (19561 ms) 2024-06-11T13:31:47.856 INFO:tasks.workunit.client.0.smithi057.stdout: write_operations: [ RUN ] NeoRadosWriteOps.AssertVersion 2024-06-11T13:31:47.856 INFO:tasks.workunit.client.0.smithi057.stdout: write_operations: [ OK ] NeoRadosWriteOps.AssertVersion (12123 ms) 2024-06-11T13:31:47.856 INFO:tasks.workunit.client.0.smithi057.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Xattrs 2024-06-11T13:31:47.856 INFO:tasks.workunit.client.0.smithi057.stdout: write_operations: [ OK ] NeoRadosWriteOps.Xattrs (7925 ms) 2024-06-11T13:31:47.856 INFO:tasks.workunit.client.0.smithi057.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Write 2024-06-11T13:31:47.856 INFO:tasks.workunit.client.0.smithi057.stdout: write_operations: [ OK ] NeoRadosWriteOps.Write (7100 ms) 2024-06-11T13:31:47.856 INFO:tasks.workunit.client.0.smithi057.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Exec 2024-06-11T13:31:47.856 INFO:tasks.workunit.client.0.smithi057.stdout: write_operations: [ OK ] NeoRadosWriteOps.Exec (5689 ms) 2024-06-11T13:31:47.856 INFO:tasks.workunit.client.0.smithi057.stdout: write_operations: [ RUN ] NeoRadosWriteOps.WriteSame 2024-06-11T13:31:47.856 INFO:tasks.workunit.client.0.smithi057.stdout: write_operations: [ OK ] NeoRadosWriteOps.WriteSame (5148 ms) 2024-06-11T13:31:47.856 INFO:tasks.workunit.client.0.smithi057.stdout: write_operations: [ RUN ] NeoRadosWriteOps.CmpExt 2024-06-11T13:31:47.856 INFO:tasks.workunit.client.0.smithi057.stdout: write_operations: [ OK ] NeoRadosWriteOps.CmpExt (6193 ms) 2024-06-11T13:31:47.856 INFO:tasks.workunit.client.0.smithi057.stdout: write_operations: [----------] 7 tests from NeoRadosWriteOps (63739 ms total) 2024-06-11T13:31:47.857 INFO:tasks.workunit.client.0.smithi057.stdout: write_operations: 2024-06-11T13:31:47.857 INFO:tasks.workunit.client.0.smithi057.stdout: write_operations: [----------] Global test environment tear-down 2024-06-11T13:31:47.857 INFO:tasks.workunit.client.0.smithi057.stdout: write_operations: [==========] 7 tests from 1 test suite ran. (63739 ms total) 2024-06-11T13:31:47.857 INFO:tasks.workunit.client.0.smithi057.stdout: write_operations: [ PASSED ] 7 tests. 2024-06-11T13:31:48.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:47 smithi057 ceph-mon[40588]: pgmap v132: 644 pgs: 64 creating+peering, 580 active+clean; 217 MiB data, 1021 MiB used, 714 GiB / 715 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 0 op/s 2024-06-11T13:31:48.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:47 smithi057 ceph-mon[40588]: from='client.35678 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:48.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:47 smithi057 ceph-mon[40588]: osdmap e111: 8 total, 8 up, 8 in 2024-06-11T13:31:48.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:47 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:48.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:47 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3373550751' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:48.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:47 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/981345639' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi057-77362-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:48.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:47 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:48.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:47 smithi057 ceph-mon[40588]: from='client.35693 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:48.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:47 smithi057 ceph-mon[40588]: from='client.35690 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi057-77362-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:48.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:47 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:48.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:47 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:48.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:47 smithi057 ceph-mon[40588]: from='client.35693 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:48.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:47 smithi057 ceph-mon[40588]: from='client.35690 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi057-77362-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:48.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:47 smithi057 ceph-mon[40588]: osdmap e112: 8 total, 8 up, 8 in 2024-06-11T13:31:48.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:47 smithi057 ceph-mon[29805]: pgmap v132: 644 pgs: 64 creating+peering, 580 active+clean; 217 MiB data, 1021 MiB used, 714 GiB / 715 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 0 op/s 2024-06-11T13:31:48.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:47 smithi057 ceph-mon[29805]: from='client.35678 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:48.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:47 smithi057 ceph-mon[29805]: osdmap e111: 8 total, 8 up, 8 in 2024-06-11T13:31:48.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:47 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:48.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:47 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3373550751' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:48.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:47 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/981345639' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi057-77362-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:48.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:47 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:48.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:47 smithi057 ceph-mon[29805]: from='client.35693 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:48.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:47 smithi057 ceph-mon[29805]: from='client.35690 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi057-77362-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:48.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:47 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:48.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:47 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:48.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:47 smithi057 ceph-mon[29805]: from='client.35693 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:48.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:47 smithi057 ceph-mon[29805]: from='client.35690 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi057-77362-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:48.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:47 smithi057 ceph-mon[29805]: osdmap e112: 8 total, 8 up, 8 in 2024-06-11T13:31:48.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:47 smithi107 ceph-mon[33178]: pgmap v132: 644 pgs: 64 creating+peering, 580 active+clean; 217 MiB data, 1021 MiB used, 714 GiB / 715 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 0 op/s 2024-06-11T13:31:48.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:47 smithi107 ceph-mon[33178]: from='client.35678 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-78107-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:48.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:47 smithi107 ceph-mon[33178]: osdmap e111: 8 total, 8 up, 8 in 2024-06-11T13:31:48.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:47 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:48.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:47 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3373550751' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:48.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:47 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/981345639' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi057-77362-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:48.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:47 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:48.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:47 smithi107 ceph-mon[33178]: from='client.35693 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:48.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:47 smithi107 ceph-mon[33178]: from='client.35690 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi057-77362-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:48.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:47 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:48.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:47 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:48.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:47 smithi107 ceph-mon[33178]: from='client.35693 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:48.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:47 smithi107 ceph-mon[33178]: from='client.35690 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi057-77362-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:48.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:47 smithi107 ceph-mon[33178]: osdmap e112: 8 total, 8 up, 8 in 2024-06-11T13:31:49.740 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_write_operations: Running main() from gmock_main.cc 2024-06-11T13:31:49.740 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_write_operations: [==========] Running 8 tests from 2 test suites. 2024-06-11T13:31:49.740 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_write_operations: [----------] Global test environment set-up. 2024-06-11T13:31:49.740 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps 2024-06-11T13:31:49.740 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_write_operations: [ RUN ] LibradosCWriteOps.NewDelete 2024-06-11T13:31:49.741 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_write_operations: [ OK ] LibradosCWriteOps.NewDelete (0 ms) 2024-06-11T13:31:49.741 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps (0 ms total) 2024-06-11T13:31:49.741 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_write_operations: 2024-06-11T13:31:49.741 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps 2024-06-11T13:31:49.741 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.assertExists 2024-06-11T13:31:49.741 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.assertExists (22605 ms) 2024-06-11T13:31:49.741 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteOpAssertVersion 2024-06-11T13:31:49.741 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteOpAssertVersion (12082 ms) 2024-06-11T13:31:49.741 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Xattrs 2024-06-11T13:31:49.742 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Xattrs (6093 ms) 2024-06-11T13:31:49.742 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Write 2024-06-11T13:31:49.742 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Write (6163 ms) 2024-06-11T13:31:49.742 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Exec 2024-06-11T13:31:49.742 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Exec (5715 ms) 2024-06-11T13:31:49.742 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteSame 2024-06-11T13:31:49.742 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteSame (6492 ms) 2024-06-11T13:31:49.742 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.CmpExt 2024-06-11T13:31:49.742 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.CmpExt (6707 ms) 2024-06-11T13:31:49.742 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps (65857 ms total) 2024-06-11T13:31:49.742 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_write_operations: 2024-06-11T13:31:49.743 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_write_operations: [----------] Global test environment tear-down 2024-06-11T13:31:49.743 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_write_operations: [==========] 8 tests from 2 test suites ran. (65857 ms total) 2024-06-11T13:31:49.743 INFO:tasks.workunit.client.0.smithi057.stdout: api_c_write_operations: [ PASSED ] 8 tests. 2024-06-11T13:31:50.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:49 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:31:50.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:49 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:50.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:49 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:31:50.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:49 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:50.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:49 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:31:50.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:49 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:51.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:50 smithi057 ceph-mon[29805]: pgmap v135: 580 pgs: 96 unknown, 32 creating+peering, 452 active+clean; 217 MiB data, 1021 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 767 B/s wr, 2 op/s 2024-06-11T13:31:51.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:50 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:31:51.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:50 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1716144744' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi057-77351-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:51.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:50 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:51.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:50 smithi057 ceph-mon[29805]: osdmap e113: 8 total, 8 up, 8 in 2024-06-11T13:31:51.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:50 smithi057 ceph-mon[29805]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:31:51.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:50 smithi057 ceph-mon[29805]: from='client.35711 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi057-77351-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:51.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:50 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-9", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:31:51.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:50 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-9"}]: dispatch 2024-06-11T13:31:51.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:50 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:51.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:50 smithi057 ceph-mon[29805]: osdmap e114: 8 total, 8 up, 8 in 2024-06-11T13:31:51.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:50 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-9"}]: dispatch 2024-06-11T13:31:51.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:50 smithi057 ceph-mon[40588]: pgmap v135: 580 pgs: 96 unknown, 32 creating+peering, 452 active+clean; 217 MiB data, 1021 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 767 B/s wr, 2 op/s 2024-06-11T13:31:51.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:50 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:31:51.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:50 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1716144744' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi057-77351-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:51.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:50 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:51.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:50 smithi057 ceph-mon[40588]: osdmap e113: 8 total, 8 up, 8 in 2024-06-11T13:31:51.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:50 smithi057 ceph-mon[40588]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:31:51.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:50 smithi057 ceph-mon[40588]: from='client.35711 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi057-77351-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:51.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:50 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-9", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:31:51.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:50 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-9"}]: dispatch 2024-06-11T13:31:51.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:50 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:51.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:50 smithi057 ceph-mon[40588]: osdmap e114: 8 total, 8 up, 8 in 2024-06-11T13:31:51.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:50 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-9"}]: dispatch 2024-06-11T13:31:51.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:50 smithi107 ceph-mon[33178]: pgmap v135: 580 pgs: 96 unknown, 32 creating+peering, 452 active+clean; 217 MiB data, 1021 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 767 B/s wr, 2 op/s 2024-06-11T13:31:51.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:50 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:31:51.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:50 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1716144744' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi057-77351-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:51.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:50 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:51.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:50 smithi107 ceph-mon[33178]: osdmap e113: 8 total, 8 up, 8 in 2024-06-11T13:31:51.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:50 smithi107 ceph-mon[33178]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:31:51.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:50 smithi107 ceph-mon[33178]: from='client.35711 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi057-77351-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:51.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:50 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-9", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:31:51.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:50 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-9"}]: dispatch 2024-06-11T13:31:51.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:50 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:51.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:50 smithi107 ceph-mon[33178]: osdmap e114: 8 total, 8 up, 8 in 2024-06-11T13:31:51.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:50 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-9"}]: dispatch 2024-06-11T13:31:51.550 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:31:51 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:31:52.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:51 smithi057 ceph-mon[29805]: pgmap v138: 644 pgs: 192 unknown, 452 active+clean; 217 MiB data, 1021 MiB used, 714 GiB / 715 GiB avail 2024-06-11T13:31:52.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:51 smithi057 ceph-mon[29805]: from='client.35711 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi057-77351-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:52.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:51 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-9"}]': finished 2024-06-11T13:31:52.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:51 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-9", "mode": "writeback"}]: dispatch 2024-06-11T13:31:52.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:51 smithi057 ceph-mon[29805]: osdmap e115: 8 total, 8 up, 8 in 2024-06-11T13:31:52.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:51 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-9", "mode": "writeback"}]: dispatch 2024-06-11T13:31:52.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:51 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2542924549' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi057-77362-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:52.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:51 smithi057 ceph-mon[29805]: from='client.35723 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi057-77362-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:52.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:51 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:31:52.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:51 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:31:52.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:51 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:52.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:51 smithi057 ceph-mon[40588]: pgmap v138: 644 pgs: 192 unknown, 452 active+clean; 217 MiB data, 1021 MiB used, 714 GiB / 715 GiB avail 2024-06-11T13:31:52.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:51 smithi057 ceph-mon[40588]: from='client.35711 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi057-77351-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:52.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:51 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-9"}]': finished 2024-06-11T13:31:52.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:51 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-9", "mode": "writeback"}]: dispatch 2024-06-11T13:31:52.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:51 smithi057 ceph-mon[40588]: osdmap e115: 8 total, 8 up, 8 in 2024-06-11T13:31:52.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:51 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-9", "mode": "writeback"}]: dispatch 2024-06-11T13:31:52.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:51 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2542924549' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi057-77362-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:52.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:51 smithi057 ceph-mon[40588]: from='client.35723 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi057-77362-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:52.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:51 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:31:52.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:51 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:31:52.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:51 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:52.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:51 smithi107 ceph-mon[33178]: pgmap v138: 644 pgs: 192 unknown, 452 active+clean; 217 MiB data, 1021 MiB used, 714 GiB / 715 GiB avail 2024-06-11T13:31:52.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:51 smithi107 ceph-mon[33178]: from='client.35711 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi057-77351-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:52.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:51 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-9"}]': finished 2024-06-11T13:31:52.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:51 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-9", "mode": "writeback"}]: dispatch 2024-06-11T13:31:52.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:51 smithi107 ceph-mon[33178]: osdmap e115: 8 total, 8 up, 8 in 2024-06-11T13:31:52.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:51 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-9", "mode": "writeback"}]: dispatch 2024-06-11T13:31:52.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:51 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2542924549' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi057-77362-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:52.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:51 smithi107 ceph-mon[33178]: from='client.35723 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi057-77362-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:52.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:51 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:31:52.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:51 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:31:52.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:51 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:53.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:53 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:31:53.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:53 smithi057 ceph-mon[29805]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:31:53.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:53 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-9", "mode": "writeback"}]': finished 2024-06-11T13:31:53.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:53 smithi057 ceph-mon[29805]: from='client.35723 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi057-77362-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:53.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:53 smithi057 ceph-mon[29805]: osdmap e116: 8 total, 8 up, 8 in 2024-06-11T13:31:53.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:53 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:31:53.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:53 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:53.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:53 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:31:53.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:53 smithi057 ceph-mon[40588]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:31:53.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:53 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-9", "mode": "writeback"}]': finished 2024-06-11T13:31:53.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:53 smithi057 ceph-mon[40588]: from='client.35723 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi057-77362-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:53.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:53 smithi057 ceph-mon[40588]: osdmap e116: 8 total, 8 up, 8 in 2024-06-11T13:31:53.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:53 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:31:53.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:53 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:53.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:53 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:31:53.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:53 smithi107 ceph-mon[33178]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:31:53.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:53 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-9", "mode": "writeback"}]': finished 2024-06-11T13:31:53.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:53 smithi107 ceph-mon[33178]: from='client.35723 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi057-77362-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:53.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:53 smithi107 ceph-mon[33178]: osdmap e116: 8 total, 8 up, 8 in 2024-06-11T13:31:53.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:53 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:31:53.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:53 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:53.727 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: [==========] Running 12 tests from 4 test suites. 2024-06-11T13:31:53.728 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: [----------] Global test environment set-up. 2024-06-11T13:31:53.728 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion 2024-06-11T13:31:53.728 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: [ RUN ] LibRadosMiscVersion.Version 2024-06-11T13:31:53.728 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: [ OK ] LibRadosMiscVersion.Version (0 ms) 2024-06-11T13:31:53.728 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-06-11T13:31:53.728 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: 2024-06-11T13:31:53.728 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure 2024-06-11T13:31:53.728 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectFailure 2024-06-11T13:31:53.728 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: unable to get monitor info from DNS SRV with service name: ceph-mon 2024-06-11T13:31:53.728 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: 2024-06-11T13:30:43.709+0000 7f20e6c39780 -1 failed for service _ceph-mon._tcp 2024-06-11T13:31:53.729 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: 2024-06-11T13:30:43.709+0000 7f20e6c39780 -1 monclient: get_monmap_and_config cannot identify monitors to contact 2024-06-11T13:31:53.729 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectFailure (58 ms) 2024-06-11T13:31:53.729 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectTimeout 2024-06-11T13:31:53.729 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectTimeout (2013 ms) 2024-06-11T13:31:53.729 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure (2071 ms total) 2024-06-11T13:31:53.729 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: 2024-06-11T13:31:53.729 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: [----------] 1 test from LibRadosMiscPool 2024-06-11T13:31:53.729 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: [ RUN ] LibRadosMiscPool.PoolCreationRace 2024-06-11T13:31:53.729 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x56407869d660 2024-06-11T13:31:53.729 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a64d70 2024-06-11T13:31:53.729 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a65790 2024-06-11T13:31:53.729 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6a5c0 2024-06-11T13:31:53.729 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6b050 2024-06-11T13:31:53.730 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6a750 2024-06-11T13:31:53.730 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6b5f0 2024-06-11T13:31:53.730 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6a8c0 2024-06-11T13:31:53.730 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6bb00 2024-06-11T13:31:53.730 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6aa30 2024-06-11T13:31:53.730 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6c0c0 2024-06-11T13:31:53.730 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6bc10 2024-06-11T13:31:53.730 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6b760 2024-06-11T13:31:53.730 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6b870 2024-06-11T13:31:53.730 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6b980 2024-06-11T13:31:53.730 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a65260 2024-06-11T13:31:53.730 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6c1d0 2024-06-11T13:31:53.730 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6c2e0 2024-06-11T13:31:53.731 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6c8a0 2024-06-11T13:31:53.731 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6c3f0 2024-06-11T13:31:53.731 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6bd20 2024-06-11T13:31:53.731 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x5640788c63b0 2024-06-11T13:31:53.731 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6be30 2024-06-11T13:31:53.731 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6bf40 2024-06-11T13:31:53.731 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6c9b0 2024-06-11T13:31:53.731 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6cea0 2024-06-11T13:31:53.731 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6cfb0 2024-06-11T13:31:53.731 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6cb30 2024-06-11T13:31:53.731 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6cc80 2024-06-11T13:31:53.731 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6d570 2024-06-11T13:31:53.731 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6d0c0 2024-06-11T13:31:53.731 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6c500 2024-06-11T13:31:53.732 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6c630 2024-06-11T13:31:53.732 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6c740 2024-06-11T13:31:53.732 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6db30 2024-06-11T13:31:53.732 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6d680 2024-06-11T13:31:53.732 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6d1d0 2024-06-11T13:31:53.732 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6d2e0 2024-06-11T13:31:53.732 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6d450 2024-06-11T13:31:53.732 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6e110 2024-06-11T13:31:53.732 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6dc60 2024-06-11T13:31:53.732 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6e220 2024-06-11T13:31:53.732 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6e3b0 2024-06-11T13:31:53.732 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6e970 2024-06-11T13:31:53.732 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6dd70 2024-06-11T13:31:53.732 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6de80 2024-06-11T13:31:53.733 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6dfd0 2024-06-11T13:31:53.733 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6eac0 2024-06-11T13:31:53.733 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6e4c0 2024-06-11T13:31:53.733 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6e5f0 2024-06-11T13:31:53.733 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6e740 2024-06-11T13:31:54.033 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:31:53 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:31:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:31:54.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:54 smithi057 ceph-mon[40588]: pgmap v141: 548 pgs: 64 creating+peering, 484 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-06-11T13:31:54.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:54 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:31:54.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:54 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3301304531' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi057-77351-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:54.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:54 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1661984905' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:54.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:54 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:54.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:54 smithi057 ceph-mon[40588]: osdmap e117: 8 total, 8 up, 8 in 2024-06-11T13:31:54.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:54 smithi057 ceph-mon[40588]: from='client.35735 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:54.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:54 smithi057 ceph-mon[40588]: from='client.25741 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi057-77351-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:54.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:54 smithi057 ceph-mon[29805]: pgmap v141: 548 pgs: 64 creating+peering, 484 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-06-11T13:31:54.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:54 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:31:54.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:54 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3301304531' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi057-77351-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:54.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:54 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1661984905' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:54.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:54 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:54.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:54 smithi057 ceph-mon[29805]: osdmap e117: 8 total, 8 up, 8 in 2024-06-11T13:31:54.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:54 smithi057 ceph-mon[29805]: from='client.35735 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:54.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:54 smithi057 ceph-mon[29805]: from='client.25741 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi057-77351-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:54.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:54 smithi107 ceph-mon[33178]: pgmap v141: 548 pgs: 64 creating+peering, 484 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-06-11T13:31:54.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:54 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:31:54.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:54 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3301304531' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi057-77351-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:54.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:54 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1661984905' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:54.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:54 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:54.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:54 smithi107 ceph-mon[33178]: osdmap e117: 8 total, 8 up, 8 in 2024-06-11T13:31:54.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:54 smithi107 ceph-mon[33178]: from='client.35735 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:54.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:54 smithi107 ceph-mon[33178]: from='client.25741 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi057-77351-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:56.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:55 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:31:56.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:55 smithi057 ceph-mon[29805]: from='client.35735 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:56.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:55 smithi057 ceph-mon[29805]: from='client.25741 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi057-77351-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:56.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:55 smithi057 ceph-mon[29805]: osdmap e118: 8 total, 8 up, 8 in 2024-06-11T13:31:56.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:55 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-9"}]: dispatch 2024-06-11T13:31:56.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:55 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-9"}]: dispatch 2024-06-11T13:31:56.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:55 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1493964945' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi057-77799-10"}]: dispatch 2024-06-11T13:31:56.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:55 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:56.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:55 smithi057 ceph-mon[29805]: pgmap v144: 580 pgs: 160 unknown, 420 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 2 op/s 2024-06-11T13:31:56.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:55 smithi057 ceph-mon[29805]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:31:56.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:55 smithi057 ceph-mon[29805]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:31:56.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:55 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-9"}]': finished 2024-06-11T13:31:56.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:55 smithi057 ceph-mon[29805]: osdmap e119: 8 total, 8 up, 8 in 2024-06-11T13:31:56.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:55 smithi057 ceph-mon[29805]: from='client.35759 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi057-77799-10"}]: dispatch 2024-06-11T13:31:56.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:55 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1493964945' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi057-77799-10"}]: dispatch 2024-06-11T13:31:56.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:55 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1661984905' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77632-10", "tierpool":"test-rados-api-smithi057-77632-10-cache", "force_nonempty":""}]: dispatch 2024-06-11T13:31:56.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:55 smithi057 ceph-mon[29805]: from='client.35759 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi057-77799-10"}]: dispatch 2024-06-11T13:31:56.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:55 smithi057 ceph-mon[29805]: from='client.35735 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77632-10", "tierpool":"test-rados-api-smithi057-77632-10-cache", "force_nonempty":""}]: dispatch 2024-06-11T13:31:56.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:55 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1493964945' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi057-77799-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:56.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:55 smithi057 ceph-mon[29805]: from='client.35759 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi057-77799-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:56.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:55 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:31:56.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:55 smithi057 ceph-mon[40588]: from='client.35735 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:56.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:55 smithi057 ceph-mon[40588]: from='client.25741 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi057-77351-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:56.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:55 smithi057 ceph-mon[40588]: osdmap e118: 8 total, 8 up, 8 in 2024-06-11T13:31:56.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:55 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-9"}]: dispatch 2024-06-11T13:31:56.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:55 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-9"}]: dispatch 2024-06-11T13:31:56.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:55 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1493964945' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi057-77799-10"}]: dispatch 2024-06-11T13:31:56.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:55 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:56.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:55 smithi057 ceph-mon[40588]: pgmap v144: 580 pgs: 160 unknown, 420 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 2 op/s 2024-06-11T13:31:56.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:55 smithi057 ceph-mon[40588]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:31:56.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:55 smithi057 ceph-mon[40588]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:31:56.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:55 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-9"}]': finished 2024-06-11T13:31:56.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:55 smithi057 ceph-mon[40588]: osdmap e119: 8 total, 8 up, 8 in 2024-06-11T13:31:56.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:55 smithi057 ceph-mon[40588]: from='client.35759 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi057-77799-10"}]: dispatch 2024-06-11T13:31:56.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:55 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1493964945' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi057-77799-10"}]: dispatch 2024-06-11T13:31:56.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:55 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1661984905' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77632-10", "tierpool":"test-rados-api-smithi057-77632-10-cache", "force_nonempty":""}]: dispatch 2024-06-11T13:31:56.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:55 smithi057 ceph-mon[40588]: from='client.35759 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi057-77799-10"}]: dispatch 2024-06-11T13:31:56.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:55 smithi057 ceph-mon[40588]: from='client.35735 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77632-10", "tierpool":"test-rados-api-smithi057-77632-10-cache", "force_nonempty":""}]: dispatch 2024-06-11T13:31:56.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:55 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1493964945' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi057-77799-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:56.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:55 smithi057 ceph-mon[40588]: from='client.35759 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi057-77799-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:56.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:55 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:31:56.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:55 smithi107 ceph-mon[33178]: from='client.35735 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77632-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:56.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:55 smithi107 ceph-mon[33178]: from='client.25741 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi057-77351-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:56.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:55 smithi107 ceph-mon[33178]: osdmap e118: 8 total, 8 up, 8 in 2024-06-11T13:31:56.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:55 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-9"}]: dispatch 2024-06-11T13:31:56.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:55 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-9"}]: dispatch 2024-06-11T13:31:56.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:55 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1493964945' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi057-77799-10"}]: dispatch 2024-06-11T13:31:56.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:55 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:56.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:55 smithi107 ceph-mon[33178]: pgmap v144: 580 pgs: 160 unknown, 420 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 2 op/s 2024-06-11T13:31:56.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:55 smithi107 ceph-mon[33178]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:31:56.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:55 smithi107 ceph-mon[33178]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:31:56.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:55 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-9"}]': finished 2024-06-11T13:31:56.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:55 smithi107 ceph-mon[33178]: osdmap e119: 8 total, 8 up, 8 in 2024-06-11T13:31:56.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:55 smithi107 ceph-mon[33178]: from='client.35759 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi057-77799-10"}]: dispatch 2024-06-11T13:31:56.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:55 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1493964945' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi057-77799-10"}]: dispatch 2024-06-11T13:31:56.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:55 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1661984905' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77632-10", "tierpool":"test-rados-api-smithi057-77632-10-cache", "force_nonempty":""}]: dispatch 2024-06-11T13:31:56.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:55 smithi107 ceph-mon[33178]: from='client.35759 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi057-77799-10"}]: dispatch 2024-06-11T13:31:56.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:55 smithi107 ceph-mon[33178]: from='client.35735 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77632-10", "tierpool":"test-rados-api-smithi057-77632-10-cache", "force_nonempty":""}]: dispatch 2024-06-11T13:31:56.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:55 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1493964945' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi057-77799-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:56.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:55 smithi107 ceph-mon[33178]: from='client.35759 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi057-77799-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:31:57.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:57 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:57.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:57 smithi057 ceph-mon[29805]: from='client.35735 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77632-10", "tierpool":"test-rados-api-smithi057-77632-10-cache", "force_nonempty":""}]': finished 2024-06-11T13:31:57.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:57 smithi057 ceph-mon[29805]: from='client.35759 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi057-77799-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:57.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:57 smithi057 ceph-mon[29805]: osdmap e120: 8 total, 8 up, 8 in 2024-06-11T13:31:57.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:57 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1661984905' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77632-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:57.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:57 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1493964945' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi057-77799-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi057-77799-10"}]: dispatch 2024-06-11T13:31:57.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:57 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1778129485' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi057-77362-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:57.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:57 smithi057 ceph-mon[29805]: from='client.35735 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77632-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:57.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:57 smithi057 ceph-mon[29805]: from='client.35759 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi057-77799-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi057-77799-10"}]: dispatch 2024-06-11T13:31:57.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:57 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:57.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:57 smithi057 ceph-mon[40588]: from='client.35735 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77632-10", "tierpool":"test-rados-api-smithi057-77632-10-cache", "force_nonempty":""}]': finished 2024-06-11T13:31:57.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:57 smithi057 ceph-mon[40588]: from='client.35759 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi057-77799-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:57.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:57 smithi057 ceph-mon[40588]: osdmap e120: 8 total, 8 up, 8 in 2024-06-11T13:31:57.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:57 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1661984905' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77632-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:57.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:57 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1493964945' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi057-77799-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi057-77799-10"}]: dispatch 2024-06-11T13:31:57.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:57 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1778129485' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi057-77362-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:57.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:57 smithi057 ceph-mon[40588]: from='client.35735 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77632-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:57.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:57 smithi057 ceph-mon[40588]: from='client.35759 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi057-77799-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi057-77799-10"}]: dispatch 2024-06-11T13:31:57.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:57 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:57.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:57 smithi107 ceph-mon[33178]: from='client.35735 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77632-10", "tierpool":"test-rados-api-smithi057-77632-10-cache", "force_nonempty":""}]': finished 2024-06-11T13:31:57.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:57 smithi107 ceph-mon[33178]: from='client.35759 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi057-77799-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:31:57.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:57 smithi107 ceph-mon[33178]: osdmap e120: 8 total, 8 up, 8 in 2024-06-11T13:31:57.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:57 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1661984905' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77632-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:57.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:57 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1493964945' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi057-77799-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi057-77799-10"}]: dispatch 2024-06-11T13:31:57.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:57 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1778129485' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi057-77362-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:57.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:57 smithi107 ceph-mon[33178]: from='client.35735 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77632-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:57.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:57 smithi107 ceph-mon[33178]: from='client.35759 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi057-77799-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi057-77799-10"}]: dispatch 2024-06-11T13:31:58.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:58 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:58.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:58 smithi057 ceph-mon[29805]: pgmap v147: 548 pgs: 64 creating+peering, 32 unknown, 452 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 1 op/s 2024-06-11T13:31:58.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:58 smithi057 ceph-mon[29805]: from='client.25768 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi057-77362-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:58.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:58 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:58.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:58 smithi057 ceph-mon[29805]: from='client.35735 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77632-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:58.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:58 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:58.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:58 smithi057 ceph-mon[40588]: pgmap v147: 548 pgs: 64 creating+peering, 32 unknown, 452 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 1 op/s 2024-06-11T13:31:58.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:58 smithi057 ceph-mon[40588]: from='client.25768 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi057-77362-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:58.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:58 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:58.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:58 smithi057 ceph-mon[40588]: from='client.35735 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77632-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:58.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:58 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:58.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:58 smithi107 ceph-mon[33178]: pgmap v147: 548 pgs: 64 creating+peering, 32 unknown, 452 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 1 op/s 2024-06-11T13:31:58.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:58 smithi107 ceph-mon[33178]: from='client.25768 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi057-77362-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:58.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:58 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:58.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:58 smithi107 ceph-mon[33178]: from='client.35735 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77632-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:59.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:59 smithi057 ceph-mon[29805]: from='client.25768 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi057-77362-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:59.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:59 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1661984905' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77632-10", "tierpool":"test-rados-api-smithi057-77632-10-cache"}]: dispatch 2024-06-11T13:31:59.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:59 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:59.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:59 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:59.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:59 smithi057 ceph-mon[29805]: pgmap v149: 612 pgs: 32 creating+peering, 128 unknown, 452 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.6 KiB/s wr, 4 op/s 2024-06-11T13:31:59.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:59 smithi057 ceph-mon[29805]: osdmap e121: 8 total, 8 up, 8 in 2024-06-11T13:31:59.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:59 smithi057 ceph-mon[29805]: from='client.35735 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77632-10", "tierpool":"test-rados-api-smithi057-77632-10-cache"}]: dispatch 2024-06-11T13:31:59.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:59 smithi057 ceph-mon[29805]: from='client.35759 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi057-77799-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi057-77799-10"}]': finished 2024-06-11T13:31:59.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:59 smithi057 ceph-mon[29805]: from='client.35735 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77632-10", "tierpool":"test-rados-api-smithi057-77632-10-cache"}]': finished 2024-06-11T13:31:59.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:59 smithi057 ceph-mon[29805]: osdmap e122: 8 total, 8 up, 8 in 2024-06-11T13:31:59.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:59 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/584374032' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi057-77351-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:59.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:59 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:59.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:59 smithi057 ceph-mon[29805]: from='client.25780 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi057-77351-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:59.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:31:59 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:59.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:59 smithi057 ceph-mon[40588]: from='client.25768 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi057-77362-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:59.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:59 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1661984905' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77632-10", "tierpool":"test-rados-api-smithi057-77632-10-cache"}]: dispatch 2024-06-11T13:31:59.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:59 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:59.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:59 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:59.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:59 smithi057 ceph-mon[40588]: pgmap v149: 612 pgs: 32 creating+peering, 128 unknown, 452 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.6 KiB/s wr, 4 op/s 2024-06-11T13:31:59.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:59 smithi057 ceph-mon[40588]: osdmap e121: 8 total, 8 up, 8 in 2024-06-11T13:31:59.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:59 smithi057 ceph-mon[40588]: from='client.35735 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77632-10", "tierpool":"test-rados-api-smithi057-77632-10-cache"}]: dispatch 2024-06-11T13:31:59.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:59 smithi057 ceph-mon[40588]: from='client.35759 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi057-77799-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi057-77799-10"}]': finished 2024-06-11T13:31:59.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:59 smithi057 ceph-mon[40588]: from='client.35735 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77632-10", "tierpool":"test-rados-api-smithi057-77632-10-cache"}]': finished 2024-06-11T13:31:59.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:59 smithi057 ceph-mon[40588]: osdmap e122: 8 total, 8 up, 8 in 2024-06-11T13:31:59.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:59 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/584374032' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi057-77351-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:59.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:59 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:59.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:59 smithi057 ceph-mon[40588]: from='client.25780 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi057-77351-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:59.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:31:59 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:59.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:59 smithi107 ceph-mon[33178]: from='client.25768 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi057-77362-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:31:59.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:59 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1661984905' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77632-10", "tierpool":"test-rados-api-smithi057-77632-10-cache"}]: dispatch 2024-06-11T13:31:59.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:59 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:59.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:59 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:31:59.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:59 smithi107 ceph-mon[33178]: pgmap v149: 612 pgs: 32 creating+peering, 128 unknown, 452 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.6 KiB/s wr, 4 op/s 2024-06-11T13:31:59.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:59 smithi107 ceph-mon[33178]: osdmap e121: 8 total, 8 up, 8 in 2024-06-11T13:31:59.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:59 smithi107 ceph-mon[33178]: from='client.35735 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77632-10", "tierpool":"test-rados-api-smithi057-77632-10-cache"}]: dispatch 2024-06-11T13:31:59.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:59 smithi107 ceph-mon[33178]: from='client.35759 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi057-77799-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi057-77799-10"}]': finished 2024-06-11T13:31:59.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:59 smithi107 ceph-mon[33178]: from='client.35735 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77632-10", "tierpool":"test-rados-api-smithi057-77632-10-cache"}]': finished 2024-06-11T13:31:59.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:59 smithi107 ceph-mon[33178]: osdmap e122: 8 total, 8 up, 8 in 2024-06-11T13:31:59.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:59 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/584374032' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi057-77351-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:59.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:59 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:59.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:59 smithi107 ceph-mon[33178]: from='client.25780 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi057-77351-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:31:59.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:31:59 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:00.431 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started api_pool: Running main() from gmock_main.cc 2024-06-11T13:32:00.431 INFO:tasks.workunit.client.0.smithi057.stdout: api_pool: [==========] Running 9 tests from 1 test suite. 2024-06-11T13:32:00.431 INFO:tasks.workunit.client.0.smithi057.stdout: api_pool: [----------] Global test environment set-up. 2024-06-11T13:32:00.431 INFO:tasks.workunit.client.0.smithi057.stdout: api_pool: [----------] 9 tests from LibRadosPools 2024-06-11T13:32:00.431 INFO:tasks.workunit.client.0.smithi057.stdout: api_pool: [ RUN ] LibRadosPools.PoolList 2024-06-11T13:32:00.431 INFO:tasks.workunit.client.0.smithi057.stdout: api_pool: [ OK ] LibRadosPools.PoolList (22812 ms) 2024-06-11T13:32:00.431 INFO:tasks.workunit.client.0.smithi057.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup 2024-06-11T13:32:00.432 INFO:tasks.workunit.client.0.smithi057.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup (9290 ms) 2024-06-11T13:32:00.432 INFO:tasks.workunit.client.0.smithi057.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup2 2024-06-11T13:32:00.432 INFO:tasks.workunit.client.0.smithi057.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup2 (6896 ms) 2024-06-11T13:32:00.432 INFO:tasks.workunit.client.0.smithi057.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookupOtherInstance 2024-06-11T13:32:00.432 INFO:tasks.workunit.client.0.smithi057.stdout: api_pool: [ OK ] LibRadosPools.PoolLookupOtherInstance (4709 ms) 2024-06-11T13:32:00.432 INFO:tasks.workunit.client.0.smithi057.stdout: api_pool: [ RUN ] LibRadosPools.PoolReverseLookupOtherInstance 2024-06-11T13:32:00.432 INFO:tasks.workunit.client.0.smithi057.stdout: api_pool: [ OK ] LibRadosPools.PoolReverseLookupOtherInstance (3448 ms) 2024-06-11T13:32:00.432 INFO:tasks.workunit.client.0.smithi057.stdout: api_pool: [ RUN ] LibRadosPools.PoolDelete 2024-06-11T13:32:00.432 INFO:tasks.workunit.client.0.smithi057.stdout: api_pool: [ OK ] LibRadosPools.PoolDelete (8419 ms) 2024-06-11T13:32:00.432 INFO:tasks.workunit.client.0.smithi057.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateDelete 2024-06-11T13:32:00.432 INFO:tasks.workunit.client.0.smithi057.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateDelete (6175 ms) 2024-06-11T13:32:00.432 INFO:tasks.workunit.client.0.smithi057.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateWithCrushRule 2024-06-11T13:32:00.432 INFO:tasks.workunit.client.0.smithi057.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateWithCrushRule (6312 ms) 2024-06-11T13:32:00.432 INFO:tasks.workunit.client.0.smithi057.stdout: api_pool: [ RUN ] LibRadosPools.PoolGetBaseTier 2024-06-11T13:32:00.433 INFO:tasks.workunit.client.0.smithi057.stdout: api_pool: [ OK ] LibRadosPools.PoolGetBaseTier (8677 ms) 2024-06-11T13:32:00.433 INFO:tasks.workunit.client.0.smithi057.stdout: api_pool: [----------] 9 tests from LibRadosPools (76739 ms total) 2024-06-11T13:32:00.433 INFO:tasks.workunit.client.0.smithi057.stdout: api_pool: 2024-06-11T13:32:00.433 INFO:tasks.workunit.client.0.smithi057.stdout: api_pool: [----------] Global test environment tear-down 2024-06-11T13:32:00.433 INFO:tasks.workunit.client.0.smithi057.stdout: api_pool: [==========] 9 tests from 1 test suite ran. (76739 ms total) 2024-06-11T13:32:00.433 INFO:tasks.workunit.client.0.smithi057.stdout: api_pool: [ PASSED ] 9 tests. 2024-06-11T13:32:01.547 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:32:01 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:32:02.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:02 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:02.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:02 smithi057 ceph-mon[29805]: from='client.25780 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi057-77351-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:02.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:02 smithi057 ceph-mon[29805]: osdmap e123: 8 total, 8 up, 8 in 2024-06-11T13:32:02.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:02 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:02.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:02 smithi057 ceph-mon[29805]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:32:02.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:02 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:02.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:02 smithi057 ceph-mon[40588]: from='client.25780 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi057-77351-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:02.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:02 smithi057 ceph-mon[40588]: osdmap e123: 8 total, 8 up, 8 in 2024-06-11T13:32:02.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:02 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:02.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:02 smithi057 ceph-mon[40588]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:32:02.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:02 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:02.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:02 smithi107 ceph-mon[33178]: from='client.25780 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi057-77351-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:02.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:02 smithi107 ceph-mon[33178]: osdmap e123: 8 total, 8 up, 8 in 2024-06-11T13:32:02.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:02 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:02.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:02 smithi107 ceph-mon[33178]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:32:03.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:03 smithi057 ceph-mon[29805]: pgmap v152: 492 pgs: 72 unknown, 420 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-06-11T13:32:03.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:03 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:32:03.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:03 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:03.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:03 smithi057 ceph-mon[29805]: osdmap e124: 8 total, 8 up, 8 in 2024-06-11T13:32:03.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:03 smithi057 ceph-mon[29805]: osdmap e125: 8 total, 8 up, 8 in 2024-06-11T13:32:03.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:03 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1901873280' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi057-77362-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:03.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:03 smithi057 ceph-mon[29805]: from='client.35804 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi057-77362-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:03.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:03 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:32:03.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:03 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:03.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:03 smithi057 ceph-mon[29805]: pgmap v155: 588 pgs: 21 creating+peering, 139 unknown, 428 active+clean; 218 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 258 KiB/s wr, 3 op/s 2024-06-11T13:32:03.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:03 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-11T13:32:03.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:03 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi057-77576-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:03.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:03 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:32:03.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:03 smithi057 ceph-mon[29805]: from='client.24721 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi057-77576-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:03.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:03 smithi057 ceph-mon[40588]: pgmap v152: 492 pgs: 72 unknown, 420 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-06-11T13:32:03.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:03 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:32:03.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:03 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:03.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:03 smithi057 ceph-mon[40588]: osdmap e124: 8 total, 8 up, 8 in 2024-06-11T13:32:03.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:03 smithi057 ceph-mon[40588]: osdmap e125: 8 total, 8 up, 8 in 2024-06-11T13:32:03.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:03 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1901873280' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi057-77362-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:03.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:03 smithi057 ceph-mon[40588]: from='client.35804 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi057-77362-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:03.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:03 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:32:03.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:03 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:03.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:03 smithi057 ceph-mon[40588]: pgmap v155: 588 pgs: 21 creating+peering, 139 unknown, 428 active+clean; 218 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 258 KiB/s wr, 3 op/s 2024-06-11T13:32:03.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:03 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-11T13:32:03.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:03 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi057-77576-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:03.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:03 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:32:03.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:03 smithi057 ceph-mon[40588]: from='client.24721 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi057-77576-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:03.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:03 smithi107 ceph-mon[33178]: pgmap v152: 492 pgs: 72 unknown, 420 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-06-11T13:32:03.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:03 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:32:03.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:03 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:03.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:03 smithi107 ceph-mon[33178]: osdmap e124: 8 total, 8 up, 8 in 2024-06-11T13:32:03.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:03 smithi107 ceph-mon[33178]: osdmap e125: 8 total, 8 up, 8 in 2024-06-11T13:32:03.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:03 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1901873280' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi057-77362-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:03.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:03 smithi107 ceph-mon[33178]: from='client.35804 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi057-77362-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:03.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:03 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:32:03.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:03 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:03.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:03 smithi107 ceph-mon[33178]: pgmap v155: 588 pgs: 21 creating+peering, 139 unknown, 428 active+clean; 218 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 258 KiB/s wr, 3 op/s 2024-06-11T13:32:03.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:03 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-11T13:32:03.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:03 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi057-77576-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:03.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:03 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:32:03.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:03 smithi107 ceph-mon[33178]: from='client.24721 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi057-77576-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:04.034 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:32:03 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:32:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:32:04.225 INFO:tasks.workunit.client.0.smithi057.stdout: misc: Running main() from gmock_main.cc 2024-06-11T13:32:04.225 INFO:tasks.workunit.client.0.smithi057.stdout: misc: [==========] Running 12 tests from 1 test suite. 2024-06-11T13:32:04.225 INFO:tasks.workunit.client.0.smithi057.stdout: misc: [----------] Global test environment set-up. 2024-06-11T13:32:04.226 INFO:tasks.workunit.client.0.smithi057.stdout: misc: [----------] 12 tests from NeoRadosMisc 2024-06-11T13:32:04.226 INFO:tasks.workunit.client.0.smithi057.stdout: misc: [ RUN ] NeoRadosMisc.Version 2024-06-11T13:32:04.226 INFO:tasks.workunit.client.0.smithi057.stdout: misc: [ OK ] NeoRadosMisc.Version (16152 ms) 2024-06-11T13:32:04.226 INFO:tasks.workunit.client.0.smithi057.stdout: misc: [ RUN ] NeoRadosMisc.WaitOSDMap 2024-06-11T13:32:04.226 INFO:tasks.workunit.client.0.smithi057.stdout: misc: [ OK ] NeoRadosMisc.WaitOSDMap (4832 ms) 2024-06-11T13:32:04.226 INFO:tasks.workunit.client.0.smithi057.stdout: misc: [ RUN ] NeoRadosMisc.LongName 2024-06-11T13:32:04.226 INFO:tasks.workunit.client.0.smithi057.stdout: misc: [ OK ] NeoRadosMisc.LongName (10747 ms) 2024-06-11T13:32:04.226 INFO:tasks.workunit.client.0.smithi057.stdout: misc: [ RUN ] NeoRadosMisc.LongLocator 2024-06-11T13:32:04.226 INFO:tasks.workunit.client.0.smithi057.stdout: misc: [ OK ] NeoRadosMisc.LongLocator (8871 ms) 2024-06-11T13:32:04.226 INFO:tasks.workunit.client.0.smithi057.stdout: misc: [ RUN ] NeoRadosMisc.LongNamespace 2024-06-11T13:32:04.226 INFO:tasks.workunit.client.0.smithi057.stdout: misc: [ OK ] NeoRadosMisc.LongNamespace (6166 ms) 2024-06-11T13:32:04.226 INFO:tasks.workunit.client.0.smithi057.stdout: misc: [ RUN ] NeoRadosMisc.LongAttrName 2024-06-11T13:32:04.230 INFO:tasks.workunit.client.0.smithi057.stdout: misc: [ OK ] NeoRadosMisc.LongAttrName (5682 ms) 2024-06-11T13:32:04.230 INFO:tasks.workunit.client.0.smithi057.stdout: misc: [ RUN ] NeoRadosMisc.Exec 2024-06-11T13:32:04.231 INFO:tasks.workunit.client.0.smithi057.stdout: misc: [ OK ] NeoRadosMisc.Exec (5135 ms) 2024-06-11T13:32:04.231 INFO:tasks.workunit.client.0.smithi057.stdout: misc: [ RUN ] NeoRadosMisc.Operate1 2024-06-11T13:32:04.231 INFO:tasks.workunit.client.0.smithi057.stdout: misc: [ OK ] NeoRadosMisc.Operate1 (6221 ms) 2024-06-11T13:32:04.231 INFO:tasks.workunit.client.0.smithi057.stdout: misc: [ RUN ] NeoRadosMisc.Operate2 2024-06-11T13:32:04.231 INFO:tasks.workunit.client.0.smithi057.stdout: misc: [ OK ] NeoRadosMisc.Operate2 (6322 ms) 2024-06-11T13:32:04.231 INFO:tasks.workunit.client.0.smithi057.stdout: misc: [ RUN ] NeoRadosMisc.BigObject 2024-06-11T13:32:04.231 INFO:tasks.workunit.client.0.smithi057.stdout: misc: [ OK ] NeoRadosMisc.BigObject (3785 ms) 2024-06-11T13:32:04.231 INFO:tasks.workunit.client.0.smithi057.stdout: misc: [ RUN ] NeoRadosMisc.BigAttr 2024-06-11T13:32:04.231 INFO:tasks.workunit.client.0.smithi057.stdout: misc: [ OK ] NeoRadosMisc.BigAttr (1937 ms) 2024-06-11T13:32:04.231 INFO:tasks.workunit.client.0.smithi057.stdout: misc: [ RUN ] NeoRadosMisc.WriteSame 2024-06-11T13:32:04.231 INFO:tasks.workunit.client.0.smithi057.stdout: misc: [ OK ] NeoRadosMisc.WriteSame (4309 ms) 2024-06-11T13:32:04.231 INFO:tasks.workunit.client.0.smithi057.stdout: misc: [----------] 12 tests from NeoRadosMisc (80159 ms total) 2024-06-11T13:32:04.231 INFO:tasks.workunit.client.0.smithi057.stdout: misc: 2024-06-11T13:32:04.231 INFO:tasks.workunit.client.0.smithi057.stdout: misc: [----------] Global test environment tear-down 2024-06-11T13:32:04.232 INFO:tasks.workunit.client.0.smithi057.stdout: misc: [==========] 12 tests from 1 test suite ran. (80159 ms total) 2024-06-11T13:32:04.232 INFO:tasks.workunit.client.0.smithi057.stdout: misc: [ PASSED ] 12 tests. 2024-06-11T13:32:04.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:04 smithi057 ceph-mon[29805]: from='client.35804 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi057-77362-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:04.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:04 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-11", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:32:04.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:04 smithi057 ceph-mon[29805]: from='client.24721 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi057-77576-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:04.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:04 smithi057 ceph-mon[29805]: osdmap e126: 8 total, 8 up, 8 in 2024-06-11T13:32:04.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:04 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-11"}]: dispatch 2024-06-11T13:32:04.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:04 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi057-77576-1","app": "app2"}]: dispatch 2024-06-11T13:32:04.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:04 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi057-77576-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:04.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:04 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-11"}]: dispatch 2024-06-11T13:32:04.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:04 smithi057 ceph-mon[29805]: from='client.24721 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi057-77576-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:04.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:04 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2201752990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi057-77351-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:04.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:04 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:04.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:04 smithi057 ceph-mon[40588]: from='client.35804 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi057-77362-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:04.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:04 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-11", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:32:04.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:04 smithi057 ceph-mon[40588]: from='client.24721 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi057-77576-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:04.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:04 smithi057 ceph-mon[40588]: osdmap e126: 8 total, 8 up, 8 in 2024-06-11T13:32:04.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:04 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-11"}]: dispatch 2024-06-11T13:32:04.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:04 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi057-77576-1","app": "app2"}]: dispatch 2024-06-11T13:32:04.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:04 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi057-77576-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:04.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:04 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-11"}]: dispatch 2024-06-11T13:32:04.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:04 smithi057 ceph-mon[40588]: from='client.24721 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi057-77576-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:04.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:04 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2201752990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi057-77351-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:04.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:04 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:04.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:04 smithi107 ceph-mon[33178]: from='client.35804 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi057-77362-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:04.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:04 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-11", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:32:04.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:04 smithi107 ceph-mon[33178]: from='client.24721 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi057-77576-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:04.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:04 smithi107 ceph-mon[33178]: osdmap e126: 8 total, 8 up, 8 in 2024-06-11T13:32:04.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:04 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-11"}]: dispatch 2024-06-11T13:32:04.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:04 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi057-77576-1","app": "app2"}]: dispatch 2024-06-11T13:32:04.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:04 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi057-77576-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:04.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:04 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-11"}]: dispatch 2024-06-11T13:32:04.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:04 smithi107 ceph-mon[33178]: from='client.24721 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi057-77576-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:04.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:04 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2201752990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi057-77351-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:04.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:04 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:06.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:05 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-11"}]': finished 2024-06-11T13:32:06.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:05 smithi057 ceph-mon[40588]: from='client.24721 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi057-77576-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:06.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:05 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2201752990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi057-77351-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:06.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:05 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-11", "mode": "writeback"}]: dispatch 2024-06-11T13:32:06.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:05 smithi057 ceph-mon[40588]: osdmap e127: 8 total, 8 up, 8 in 2024-06-11T13:32:06.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:05 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi057-77576-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-06-11T13:32:06.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:05 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi057-77576-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-06-11T13:32:06.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:05 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-11", "mode": "writeback"}]: dispatch 2024-06-11T13:32:06.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:05 smithi057 ceph-mon[40588]: from='client.24721 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi057-77576-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-06-11T13:32:06.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:05 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:06.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:05 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-11"}]': finished 2024-06-11T13:32:06.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:05 smithi057 ceph-mon[29805]: from='client.24721 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi057-77576-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:06.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:05 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2201752990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi057-77351-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:06.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:05 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-11", "mode": "writeback"}]: dispatch 2024-06-11T13:32:06.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:05 smithi057 ceph-mon[29805]: osdmap e127: 8 total, 8 up, 8 in 2024-06-11T13:32:06.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:05 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi057-77576-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-06-11T13:32:06.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:05 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi057-77576-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-06-11T13:32:06.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:05 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-11", "mode": "writeback"}]: dispatch 2024-06-11T13:32:06.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:05 smithi057 ceph-mon[29805]: from='client.24721 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi057-77576-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-06-11T13:32:06.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:05 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:06.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:05 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-11"}]': finished 2024-06-11T13:32:06.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:05 smithi107 ceph-mon[33178]: from='client.24721 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi057-77576-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:06.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:05 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2201752990' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi057-77351-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:06.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:05 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-11", "mode": "writeback"}]: dispatch 2024-06-11T13:32:06.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:05 smithi107 ceph-mon[33178]: osdmap e127: 8 total, 8 up, 8 in 2024-06-11T13:32:06.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:05 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi057-77576-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-06-11T13:32:06.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:05 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi057-77576-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-06-11T13:32:06.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:05 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-11", "mode": "writeback"}]: dispatch 2024-06-11T13:32:06.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:05 smithi107 ceph-mon[33178]: from='client.24721 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi057-77576-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-06-11T13:32:06.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:05 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:07.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:06 smithi057 ceph-mon[29805]: pgmap v158: 492 pgs: 64 unknown, 428 active+clean; 218 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 258 KiB/s wr, 3 op/s 2024-06-11T13:32:07.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:06 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:07.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:06 smithi057 ceph-mon[29805]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:32:07.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:06 smithi057 ceph-mon[29805]: Health check update: 10 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:32:07.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:06 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-11", "mode": "writeback"}]': finished 2024-06-11T13:32:07.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:06 smithi057 ceph-mon[29805]: from='client.24721 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi057-77576-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-06-11T13:32:07.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:06 smithi057 ceph-mon[29805]: osdmap e128: 8 total, 8 up, 8 in 2024-06-11T13:32:07.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:06 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi057-77576-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-06-11T13:32:07.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:06 smithi057 ceph-mon[29805]: from='client.24721 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi057-77576-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-06-11T13:32:07.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:06 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3443594309' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi057-77362-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:07.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:06 smithi057 ceph-mon[29805]: from='client.35834 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi057-77362-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:07.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:06 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:32:07.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:06 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:32:07.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:06 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:07.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:06 smithi057 ceph-mon[29805]: from='client.24721 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi057-77576-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-06-11T13:32:07.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:06 smithi057 ceph-mon[40588]: pgmap v158: 492 pgs: 64 unknown, 428 active+clean; 218 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 258 KiB/s wr, 3 op/s 2024-06-11T13:32:07.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:06 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:07.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:06 smithi057 ceph-mon[40588]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:32:07.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:06 smithi057 ceph-mon[40588]: Health check update: 10 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:32:07.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:06 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-11", "mode": "writeback"}]': finished 2024-06-11T13:32:07.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:06 smithi057 ceph-mon[40588]: from='client.24721 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi057-77576-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-06-11T13:32:07.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:06 smithi057 ceph-mon[40588]: osdmap e128: 8 total, 8 up, 8 in 2024-06-11T13:32:07.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:06 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi057-77576-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-06-11T13:32:07.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:06 smithi057 ceph-mon[40588]: from='client.24721 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi057-77576-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-06-11T13:32:07.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:06 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3443594309' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi057-77362-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:07.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:06 smithi057 ceph-mon[40588]: from='client.35834 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi057-77362-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:07.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:06 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:32:07.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:06 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:32:07.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:06 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:07.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:06 smithi057 ceph-mon[40588]: from='client.24721 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi057-77576-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-06-11T13:32:07.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:06 smithi107 ceph-mon[33178]: pgmap v158: 492 pgs: 64 unknown, 428 active+clean; 218 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 258 KiB/s wr, 3 op/s 2024-06-11T13:32:07.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:06 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:07.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:06 smithi107 ceph-mon[33178]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:32:07.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:06 smithi107 ceph-mon[33178]: Health check update: 10 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:32:07.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:06 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-11", "mode": "writeback"}]': finished 2024-06-11T13:32:07.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:06 smithi107 ceph-mon[33178]: from='client.24721 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi057-77576-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-06-11T13:32:07.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:06 smithi107 ceph-mon[33178]: osdmap e128: 8 total, 8 up, 8 in 2024-06-11T13:32:07.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:06 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi057-77576-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-06-11T13:32:07.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:06 smithi107 ceph-mon[33178]: from='client.24721 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi057-77576-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-06-11T13:32:07.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:06 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3443594309' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi057-77362-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:07.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:06 smithi107 ceph-mon[33178]: from='client.35834 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi057-77362-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:07.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:06 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:32:07.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:06 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:32:07.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:06 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:07.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:06 smithi107 ceph-mon[33178]: from='client.24721 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi057-77576-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-06-11T13:32:07.633 INFO:tasks.workunit.client.0.smithi057.stdout: read_operations: Running main() from gmock_main.cc 2024-06-11T13:32:07.633 INFO:tasks.workunit.client.0.smithi057.stdout: read_operations: [==========] Running 14 tests from 1 test suite. 2024-06-11T13:32:07.633 INFO:tasks.workunit.client.0.smithi057.stdout: read_operations: [----------] Global test environment set-up. 2024-06-11T13:32:07.633 INFO:tasks.workunit.client.0.smithi057.stdout: read_operations: [----------] 14 tests from NeoRadosReadOps 2024-06-11T13:32:07.633 INFO:tasks.workunit.client.0.smithi057.stdout: read_operations: [ RUN ] NeoRadosReadOps.SetOpFlags 2024-06-11T13:32:07.633 INFO:tasks.workunit.client.0.smithi057.stdout: read_operations: [ OK ] NeoRadosReadOps.SetOpFlags (17611 ms) 2024-06-11T13:32:07.633 INFO:tasks.workunit.client.0.smithi057.stdout: read_operations: [ RUN ] NeoRadosReadOps.AssertExists 2024-06-11T13:32:07.633 INFO:tasks.workunit.client.0.smithi057.stdout: read_operations: [ OK ] NeoRadosReadOps.AssertExists (8957 ms) 2024-06-11T13:32:07.633 INFO:tasks.workunit.client.0.smithi057.stdout: read_operations: [ RUN ] NeoRadosReadOps.AssertVersion 2024-06-11T13:32:07.633 INFO:tasks.workunit.client.0.smithi057.stdout: read_operations: [ OK ] NeoRadosReadOps.AssertVersion (6496 ms) 2024-06-11T13:32:07.633 INFO:tasks.workunit.client.0.smithi057.stdout: read_operations: [ RUN ] NeoRadosReadOps.CmpXattr 2024-06-11T13:32:07.633 INFO:tasks.workunit.client.0.smithi057.stdout: read_operations: [ OK ] NeoRadosReadOps.CmpXattr (6561 ms) 2024-06-11T13:32:07.634 INFO:tasks.workunit.client.0.smithi057.stdout: read_operations: [ RUN ] NeoRadosReadOps.Read 2024-06-11T13:32:07.634 INFO:tasks.workunit.client.0.smithi057.stdout: read_operations: [ OK ] NeoRadosReadOps.Read (4679 ms) 2024-06-11T13:32:07.634 INFO:tasks.workunit.client.0.smithi057.stdout: read_operations: [ RUN ] NeoRadosReadOps.Checksum 2024-06-11T13:32:07.634 INFO:tasks.workunit.client.0.smithi057.stdout: read_operations: [ OK ] NeoRadosReadOps.Checksum (5698 ms) 2024-06-11T13:32:07.634 INFO:tasks.workunit.client.0.smithi057.stdout: read_operations: [ RUN ] NeoRadosReadOps.RWOrderedRead 2024-06-11T13:32:07.634 INFO:tasks.workunit.client.0.smithi057.stdout: read_operations: [ OK ] NeoRadosReadOps.RWOrderedRead (5172 ms) 2024-06-11T13:32:07.634 INFO:tasks.workunit.client.0.smithi057.stdout: read_operations: [ RUN ] NeoRadosReadOps.ShortRead 2024-06-11T13:32:07.634 INFO:tasks.workunit.client.0.smithi057.stdout: read_operations: [ OK ] NeoRadosReadOps.ShortRead (3759 ms) 2024-06-11T13:32:07.634 INFO:tasks.workunit.client.0.smithi057.stdout: read_operations: [ RUN ] NeoRadosReadOps.Exec 2024-06-11T13:32:07.634 INFO:tasks.workunit.client.0.smithi057.stdout: read_operations: [ OK ] NeoRadosReadOps.Exec (4833 ms) 2024-06-11T13:32:07.634 INFO:tasks.workunit.client.0.smithi057.stdout: read_operations: [ RUN ] NeoRadosReadOps.Stat 2024-06-11T13:32:07.634 INFO:tasks.workunit.client.0.smithi057.stdout: read_operations: [ OK ] NeoRadosReadOps.Stat (3902 ms) 2024-06-11T13:32:07.634 INFO:tasks.workunit.client.0.smithi057.stdout: read_operations: [ RUN ] NeoRadosReadOps.Omap 2024-06-11T13:32:07.634 INFO:tasks.workunit.client.0.smithi057.stdout: read_operations: [ OK ] NeoRadosReadOps.Omap (4366 ms) 2024-06-11T13:32:07.635 INFO:tasks.workunit.client.0.smithi057.stdout: read_operations: [ RUN ] NeoRadosReadOps.OmapNuls 2024-06-11T13:32:07.635 INFO:tasks.workunit.client.0.smithi057.stdout: read_operations: [ OK ] NeoRadosReadOps.OmapNuls (3772 ms) 2024-06-11T13:32:07.635 INFO:tasks.workunit.client.0.smithi057.stdout: read_operations: [ RUN ] NeoRadosReadOps.GetXattrs 2024-06-11T13:32:07.635 INFO:tasks.workunit.client.0.smithi057.stdout: read_operations: [ OK ] NeoRadosReadOps.GetXattrs (4312 ms) 2024-06-11T13:32:07.635 INFO:tasks.workunit.client.0.smithi057.stdout: read_operations: [ RUN ] NeoRadosReadOps.CmpExt 2024-06-11T13:32:07.635 INFO:tasks.workunit.client.0.smithi057.stdout: read_operations: [ OK ] NeoRadosReadOps.CmpExt (3413 ms) 2024-06-11T13:32:07.635 INFO:tasks.workunit.client.0.smithi057.stdout: read_operations: [----------] 14 tests from NeoRadosReadOps (83531 ms total) 2024-06-11T13:32:07.635 INFO:tasks.workunit.client.0.smithi057.stdout: read_operations: 2024-06-11T13:32:07.635 INFO:tasks.workunit.client.0.smithi057.stdout: read_operations: [----------] Global test environment tear-down 2024-06-11T13:32:07.635 INFO:tasks.workunit.client.0.smithi057.stdout: read_operations: [==========] 14 tests from 1 test suite ran. (83531 ms total) 2024-06-11T13:32:07.635 INFO:tasks.workunit.client.0.smithi057.stdout: read_operations: [ PASSED ] 14 tests. 2024-06-11T13:32:08.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:07 smithi057 ceph-mon[40588]: pgmap v160: 556 pgs: 96 creating+peering, 460 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 4.1 KiB/s rd, 3.3 KiB/s wr, 9 op/s 2024-06-11T13:32:08.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:07 smithi057 ceph-mon[40588]: from='client.35834 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi057-77362-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:08.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:07 smithi057 ceph-mon[40588]: osdmap e129: 8 total, 8 up, 8 in 2024-06-11T13:32:08.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:07 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi057-77576-1","app":"app1","key":"key1"}]: dispatch 2024-06-11T13:32:08.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:07 smithi057 ceph-mon[40588]: from='client.24721 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi057-77576-1","app":"app1","key":"key1"}]: dispatch 2024-06-11T13:32:08.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:07 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2279355734' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi057-77351-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:08.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:07 smithi057 ceph-mon[40588]: from='client.35837 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi057-77351-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:08.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:07 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:32:08.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:07 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:32:08.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:07 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:08.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:07 smithi057 ceph-mon[40588]: from='client.24721 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi057-77576-1","app":"app1","key":"key1"}]': finished 2024-06-11T13:32:08.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:07 smithi057 ceph-mon[40588]: from='client.35837 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi057-77351-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:08.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:07 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:32:08.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:07 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-11"}]: dispatch 2024-06-11T13:32:08.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:07 smithi057 ceph-mon[40588]: osdmap e130: 8 total, 8 up, 8 in 2024-06-11T13:32:08.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:07 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-11"}]: dispatch 2024-06-11T13:32:08.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:07 smithi057 ceph-mon[29805]: pgmap v160: 556 pgs: 96 creating+peering, 460 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 4.1 KiB/s rd, 3.3 KiB/s wr, 9 op/s 2024-06-11T13:32:08.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:07 smithi057 ceph-mon[29805]: from='client.35834 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi057-77362-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:08.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:07 smithi057 ceph-mon[29805]: osdmap e129: 8 total, 8 up, 8 in 2024-06-11T13:32:08.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:07 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi057-77576-1","app":"app1","key":"key1"}]: dispatch 2024-06-11T13:32:08.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:07 smithi057 ceph-mon[29805]: from='client.24721 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi057-77576-1","app":"app1","key":"key1"}]: dispatch 2024-06-11T13:32:08.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:07 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2279355734' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi057-77351-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:08.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:07 smithi057 ceph-mon[29805]: from='client.35837 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi057-77351-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:08.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:07 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:32:08.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:07 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:32:08.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:07 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:08.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:07 smithi057 ceph-mon[29805]: from='client.24721 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi057-77576-1","app":"app1","key":"key1"}]': finished 2024-06-11T13:32:08.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:07 smithi057 ceph-mon[29805]: from='client.35837 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi057-77351-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:08.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:07 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:32:08.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:07 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-11"}]: dispatch 2024-06-11T13:32:08.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:07 smithi057 ceph-mon[29805]: osdmap e130: 8 total, 8 up, 8 in 2024-06-11T13:32:08.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:07 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-11"}]: dispatch 2024-06-11T13:32:08.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:07 smithi107 ceph-mon[33178]: pgmap v160: 556 pgs: 96 creating+peering, 460 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 4.1 KiB/s rd, 3.3 KiB/s wr, 9 op/s 2024-06-11T13:32:08.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:07 smithi107 ceph-mon[33178]: from='client.35834 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi057-77362-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:08.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:07 smithi107 ceph-mon[33178]: osdmap e129: 8 total, 8 up, 8 in 2024-06-11T13:32:08.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:07 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2341478199' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi057-77576-1","app":"app1","key":"key1"}]: dispatch 2024-06-11T13:32:08.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:07 smithi107 ceph-mon[33178]: from='client.24721 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi057-77576-1","app":"app1","key":"key1"}]: dispatch 2024-06-11T13:32:08.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:07 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2279355734' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi057-77351-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:08.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:07 smithi107 ceph-mon[33178]: from='client.35837 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi057-77351-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:08.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:07 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:32:08.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:07 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:32:08.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:07 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:08.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:07 smithi107 ceph-mon[33178]: from='client.24721 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi057-77576-1","app":"app1","key":"key1"}]': finished 2024-06-11T13:32:08.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:07 smithi107 ceph-mon[33178]: from='client.35837 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi057-77351-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:08.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:07 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:32:08.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:07 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-11"}]: dispatch 2024-06-11T13:32:08.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:07 smithi107 ceph-mon[33178]: osdmap e130: 8 total, 8 up, 8 in 2024-06-11T13:32:08.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:07 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-11"}]: dispatch 2024-06-11T13:32:08.632 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [==========] Running 31 tests from 7 test suites. 2024-06-11T13:32:08.632 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [----------] Global test environment set-up. 2024-06-11T13:32:08.632 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion 2024-06-11T13:32:08.632 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ RUN ] LibRadosMiscVersion.VersionPP 2024-06-11T13:32:08.633 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ OK ] LibRadosMiscVersion.VersionPP (0 ms) 2024-06-11T13:32:08.633 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-06-11T13:32:08.633 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: 2024-06-11T13:32:08.633 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP 2024-06-11T13:32:08.633 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: seed 77576 2024-06-11T13:32:08.633 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WaitOSDMapPP 2024-06-11T13:32:08.633 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WaitOSDMapPP (2 ms) 2024-06-11T13:32:08.633 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNamePP 2024-06-11T13:32:08.633 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNamePP (167 ms) 2024-06-11T13:32:08.633 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongLocatorPP 2024-06-11T13:32:08.633 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongLocatorPP (17 ms) 2024-06-11T13:32:08.633 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNSpacePP 2024-06-11T13:32:08.633 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNSpacePP (6 ms) 2024-06-11T13:32:08.633 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongAttrNamePP 2024-06-11T13:32:08.634 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongAttrNamePP (6 ms) 2024-06-11T13:32:08.634 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.ExecPP 2024-06-11T13:32:08.634 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.ExecPP (2 ms) 2024-06-11T13:32:08.634 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BadFlagsPP 2024-06-11T13:32:08.634 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BadFlagsPP (2 ms) 2024-06-11T13:32:08.634 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate1PP 2024-06-11T13:32:08.634 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate1PP (3 ms) 2024-06-11T13:32:08.634 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate2PP 2024-06-11T13:32:08.634 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate2PP (2 ms) 2024-06-11T13:32:08.634 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigObjectPP 2024-06-11T13:32:08.634 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigObjectPP (6 ms) 2024-06-11T13:32:08.634 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AioOperatePP 2024-06-11T13:32:08.634 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AioOperatePP (2 ms) 2024-06-11T13:32:08.635 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertExistsPP 2024-06-11T13:32:08.635 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertExistsPP (3 ms) 2024-06-11T13:32:08.635 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertVersionPP 2024-06-11T13:32:08.635 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertVersionPP (5 ms) 2024-06-11T13:32:08.635 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigAttrPP 2024-06-11T13:32:08.635 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: osd_max_attr_size = 0 2024-06-11T13:32:08.635 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: osd_max_attr_size == 0; skipping test 2024-06-11T13:32:08.635 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigAttrPP (1671 ms) 2024-06-11T13:32:08.635 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyPP 2024-06-11T13:32:08.635 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyPP (309 ms) 2024-06-11T13:32:08.635 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyScrubPP 2024-06-11T13:32:08.635 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: waiting for initial deep scrubs... 2024-06-11T13:32:08.635 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: done waiting, doing copies 2024-06-11T13:32:08.635 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: waiting for final deep scrubs... 2024-06-11T13:32:08.635 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: done waiting 2024-06-11T13:32:08.636 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyScrubPP (60660 ms) 2024-06-11T13:32:08.636 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WriteSamePP 2024-06-11T13:32:08.636 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WriteSamePP (4 ms) 2024-06-11T13:32:08.636 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CmpExtPP 2024-06-11T13:32:08.636 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CmpExtPP (4 ms) 2024-06-11T13:32:08.636 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Applications 2024-06-11T13:32:08.636 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Applications (5047 ms) 2024-06-11T13:32:08.636 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatOSD 2024-06-11T13:32:08.636 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatOSD (0 ms) 2024-06-11T13:32:08.636 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatClient 2024-06-11T13:32:08.636 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatClient (0 ms) 2024-06-11T13:32:08.636 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Conf 2024-06-11T13:32:08.636 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Conf (0 ms) 2024-06-11T13:32:09.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:08 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:09.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:08 smithi057 ceph-mon[29805]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:32:09.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:08 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-11"}]': finished 2024-06-11T13:32:09.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:08 smithi057 ceph-mon[29805]: osdmap e131: 8 total, 8 up, 8 in 2024-06-11T13:32:09.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:08 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:09.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:08 smithi057 ceph-mon[40588]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:32:09.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:08 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-11"}]': finished 2024-06-11T13:32:09.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:08 smithi057 ceph-mon[40588]: osdmap e131: 8 total, 8 up, 8 in 2024-06-11T13:32:09.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:08 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:09.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:08 smithi107 ceph-mon[33178]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:32:09.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:08 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-11"}]': finished 2024-06-11T13:32:09.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:08 smithi107 ceph-mon[33178]: osdmap e131: 8 total, 8 up, 8 in 2024-06-11T13:32:10.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:10 smithi057 ceph-mon[40588]: pgmap v163: 492 pgs: 32 unknown, 460 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 4.1 KiB/s rd, 3.4 KiB/s wr, 9 op/s 2024-06-11T13:32:10.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:10 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/943465910' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi057-77362-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:10.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:10 smithi057 ceph-mon[40588]: from='client.25843 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi057-77362-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:10.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:10 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1824182913' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi057-77576-24"}]: dispatch 2024-06-11T13:32:10.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:10 smithi057 ceph-mon[40588]: from='client.25849 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi057-77576-24"}]: dispatch 2024-06-11T13:32:10.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:10 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1824182913' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi057-77576-24"}]: dispatch 2024-06-11T13:32:10.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:10 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:10.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:10 smithi057 ceph-mon[40588]: from='client.25849 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi057-77576-24"}]: dispatch 2024-06-11T13:32:10.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:10 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1824182913' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi057-77576-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:32:10.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:10 smithi057 ceph-mon[40588]: from='client.25849 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi057-77576-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:32:10.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:10 smithi057 ceph-mon[40588]: from='client.25843 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi057-77362-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:10.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:10 smithi057 ceph-mon[40588]: from='client.25849 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi057-77576-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:32:10.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:10 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1824182913' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi057-77576-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi057-77576-24"}]: dispatch 2024-06-11T13:32:10.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:10 smithi057 ceph-mon[40588]: osdmap e132: 8 total, 8 up, 8 in 2024-06-11T13:32:10.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:10 smithi057 ceph-mon[29805]: pgmap v163: 492 pgs: 32 unknown, 460 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 4.1 KiB/s rd, 3.4 KiB/s wr, 9 op/s 2024-06-11T13:32:10.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:10 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/943465910' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi057-77362-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:10.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:10 smithi057 ceph-mon[29805]: from='client.25843 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi057-77362-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:10.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:10 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1824182913' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi057-77576-24"}]: dispatch 2024-06-11T13:32:10.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:10 smithi057 ceph-mon[29805]: from='client.25849 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi057-77576-24"}]: dispatch 2024-06-11T13:32:10.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:10 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1824182913' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi057-77576-24"}]: dispatch 2024-06-11T13:32:10.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:10 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:10.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:10 smithi057 ceph-mon[29805]: from='client.25849 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi057-77576-24"}]: dispatch 2024-06-11T13:32:10.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:10 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1824182913' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi057-77576-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:32:10.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:10 smithi057 ceph-mon[29805]: from='client.25849 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi057-77576-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:32:10.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:10 smithi057 ceph-mon[29805]: from='client.25843 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi057-77362-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:10.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:10 smithi057 ceph-mon[29805]: from='client.25849 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi057-77576-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:32:10.538 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:10 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1824182913' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi057-77576-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi057-77576-24"}]: dispatch 2024-06-11T13:32:10.538 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:10 smithi057 ceph-mon[29805]: osdmap e132: 8 total, 8 up, 8 in 2024-06-11T13:32:10.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:10 smithi107 ceph-mon[33178]: pgmap v163: 492 pgs: 32 unknown, 460 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 4.1 KiB/s rd, 3.4 KiB/s wr, 9 op/s 2024-06-11T13:32:10.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:10 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/943465910' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi057-77362-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:10.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:10 smithi107 ceph-mon[33178]: from='client.25843 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi057-77362-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:10.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:10 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1824182913' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi057-77576-24"}]: dispatch 2024-06-11T13:32:10.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:10 smithi107 ceph-mon[33178]: from='client.25849 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi057-77576-24"}]: dispatch 2024-06-11T13:32:10.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:10 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1824182913' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi057-77576-24"}]: dispatch 2024-06-11T13:32:10.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:10 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:10.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:10 smithi107 ceph-mon[33178]: from='client.25849 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi057-77576-24"}]: dispatch 2024-06-11T13:32:10.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:10 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1824182913' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi057-77576-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:32:10.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:10 smithi107 ceph-mon[33178]: from='client.25849 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi057-77576-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:32:10.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:10 smithi107 ceph-mon[33178]: from='client.25843 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi057-77362-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:10.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:10 smithi107 ceph-mon[33178]: from='client.25849 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi057-77576-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:32:10.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:10 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1824182913' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi057-77576-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi057-77576-24"}]: dispatch 2024-06-11T13:32:10.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:10 smithi107 ceph-mon[33178]: osdmap e132: 8 total, 8 up, 8 in 2024-06-11T13:32:11.074 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP (6 io: Running main() from gmock_main.cc 2024-06-11T13:32:11.074 INFO:tasks.workunit.client.0.smithi057.stdout: io: [==========] Running 14 tests from 1 test suite. 2024-06-11T13:32:11.074 INFO:tasks.workunit.client.0.smithi057.stdout: io: [----------] Global test environment set-up. 2024-06-11T13:32:11.074 INFO:tasks.workunit.client.0.smithi057.stdout: io: [----------] 14 tests from NeoRadosIo 2024-06-11T13:32:11.074 INFO:tasks.workunit.client.0.smithi057.stdout: io: [ RUN ] NeoRadosIo.Limits 2024-06-11T13:32:11.074 INFO:tasks.workunit.client.0.smithi057.stdout: io: [ OK ] NeoRadosIo.Limits (22471 ms) 2024-06-11T13:32:11.074 INFO:tasks.workunit.client.0.smithi057.stdout: io: [ RUN ] NeoRadosIo.SimpleWrite 2024-06-11T13:32:11.074 INFO:tasks.workunit.client.0.smithi057.stdout: io: [ OK ] NeoRadosIo.SimpleWrite (8827 ms) 2024-06-11T13:32:11.074 INFO:tasks.workunit.client.0.smithi057.stdout: io: [ RUN ] NeoRadosIo.ReadOp 2024-06-11T13:32:11.074 INFO:tasks.workunit.client.0.smithi057.stdout: io: [ OK ] NeoRadosIo.ReadOp (8401 ms) 2024-06-11T13:32:11.074 INFO:tasks.workunit.client.0.smithi057.stdout: io: [ RUN ] NeoRadosIo.SparseRead 2024-06-11T13:32:11.074 INFO:tasks.workunit.client.0.smithi057.stdout: io: [ OK ] NeoRadosIo.SparseRead (4679 ms) 2024-06-11T13:32:11.075 INFO:tasks.workunit.client.0.smithi057.stdout: io: [ RUN ] NeoRadosIo.RoundTrip 2024-06-11T13:32:11.075 INFO:tasks.workunit.client.0.smithi057.stdout: io: [ OK ] NeoRadosIo.RoundTrip (5681 ms) 2024-06-11T13:32:11.075 INFO:tasks.workunit.client.0.smithi057.stdout: io: [ RUN ] NeoRadosIo.ReadIntoBuufferlist 2024-06-11T13:32:11.075 INFO:tasks.workunit.client.0.smithi057.stdout: io: [ OK ] NeoRadosIo.ReadIntoBuufferlist (4712 ms) 2024-06-11T13:32:11.075 INFO:tasks.workunit.client.0.smithi057.stdout: io: [ RUN ] NeoRadosIo.OverlappingWriteRoundTrip 2024-06-11T13:32:11.075 INFO:tasks.workunit.client.0.smithi057.stdout: io: [ OK ] NeoRadosIo.OverlappingWriteRoundTrip (4247 ms) 2024-06-11T13:32:11.075 INFO:tasks.workunit.client.0.smithi057.stdout: io: [ RUN ] NeoRadosIo.WriteFullRoundTrip 2024-06-11T13:32:11.075 INFO:tasks.workunit.client.0.smithi057.stdout: io: [ OK ] NeoRadosIo.WriteFullRoundTrip (4822 ms) 2024-06-11T13:32:11.075 INFO:tasks.workunit.client.0.smithi057.stdout: io: [ RUN ] NeoRadosIo.AppendRoundTrip 2024-06-11T13:32:11.075 INFO:tasks.workunit.client.0.smithi057.stdout: io: [ OK ] NeoRadosIo.AppendRoundTrip (3907 ms) 2024-06-11T13:32:11.075 INFO:tasks.workunit.client.0.smithi057.stdout: io: [ RUN ] NeoRadosIo.Trunc 2024-06-11T13:32:11.075 INFO:tasks.workunit.client.0.smithi057.stdout: io: [ OK ] NeoRadosIo.Trunc (4354 ms) 2024-06-11T13:32:11.075 INFO:tasks.workunit.client.0.smithi057.stdout: io: [ RUN ] NeoRadosIo.Remove 2024-06-11T13:32:11.075 INFO:tasks.workunit.client.0.smithi057.stdout: io: [ OK ] NeoRadosIo.Remove (3779 ms) 2024-06-11T13:32:11.076 INFO:tasks.workunit.client.0.smithi057.stdout: io: [ RUN ] NeoRadosIo.XattrsRoundTrip 2024-06-11T13:32:11.076 INFO:tasks.workunit.client.0.smithi057.stdout: io: [ OK ] NeoRadosIo.XattrsRoundTrip (4317 ms) 2024-06-11T13:32:11.076 INFO:tasks.workunit.client.0.smithi057.stdout: io: [ RUN ] NeoRadosIo.RmXattr 2024-06-11T13:32:11.076 INFO:tasks.workunit.client.0.smithi057.stdout: io: [ OK ] NeoRadosIo.RmXattr (3404 ms) 2024-06-11T13:32:11.076 INFO:tasks.workunit.client.0.smithi057.stdout: io: [ RUN ] NeoRadosIo.GetXattrs 2024-06-11T13:32:11.076 INFO:tasks.workunit.client.0.smithi057.stdout: io: [ OK ] NeoRadosIo.GetXattrs (3446 ms) 2024-06-11T13:32:11.076 INFO:tasks.workunit.client.0.smithi057.stdout: io: [----------] 14 tests from NeoRadosIo (87047 ms total) 2024-06-11T13:32:11.076 INFO:tasks.workunit.client.0.smithi057.stdout: io: 2024-06-11T13:32:11.076 INFO:tasks.workunit.client.0.smithi057.stdout: io: [----------] Global test environment tear-down 2024-06-11T13:32:11.076 INFO:tasks.workunit.client.0.smithi057.stdout: io: [==========] 14 tests from 1 test suite ran. (87047 ms total) 2024-06-11T13:32:11.076 INFO:tasks.workunit.client.0.smithi057.stdout: io: [ PASSED ] 14 tests. 2024-06-11T13:32:11.547 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:32:11 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:32:12.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:11 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/925541334' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi057-77351-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:12.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:11 smithi057 ceph-mon[40588]: from='client.25849 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi057-77576-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi057-77576-24"}]: dispatch 2024-06-11T13:32:12.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:11 smithi057 ceph-mon[40588]: from='client.35852 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi057-77351-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:12.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:11 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:12.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:11 smithi057 ceph-mon[40588]: pgmap v166: 492 pgs: 128 unknown, 364 active+clean; 587 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-06-11T13:32:12.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:11 smithi057 ceph-mon[40588]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:32:12.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:11 smithi057 ceph-mon[40588]: from='client.35852 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi057-77351-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:12.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:11 smithi057 ceph-mon[40588]: osdmap e133: 8 total, 8 up, 8 in 2024-06-11T13:32:12.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:11 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1493964945' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi057-77799-10"}]: dispatch 2024-06-11T13:32:12.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:11 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:12.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:11 smithi057 ceph-mon[40588]: from='client.35759 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi057-77799-10"}]: dispatch 2024-06-11T13:32:12.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:11 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:12.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:11 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/925541334' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi057-77351-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:12.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:11 smithi057 ceph-mon[29805]: from='client.25849 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi057-77576-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi057-77576-24"}]: dispatch 2024-06-11T13:32:12.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:11 smithi057 ceph-mon[29805]: from='client.35852 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi057-77351-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:12.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:11 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:12.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:11 smithi057 ceph-mon[29805]: pgmap v166: 492 pgs: 128 unknown, 364 active+clean; 587 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-06-11T13:32:12.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:11 smithi057 ceph-mon[29805]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:32:12.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:11 smithi057 ceph-mon[29805]: from='client.35852 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi057-77351-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:12.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:11 smithi057 ceph-mon[29805]: osdmap e133: 8 total, 8 up, 8 in 2024-06-11T13:32:12.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:11 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1493964945' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi057-77799-10"}]: dispatch 2024-06-11T13:32:12.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:11 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:12.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:11 smithi057 ceph-mon[29805]: from='client.35759 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi057-77799-10"}]: dispatch 2024-06-11T13:32:12.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:11 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:12.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:11 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/925541334' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi057-77351-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:12.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:11 smithi107 ceph-mon[33178]: from='client.25849 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi057-77576-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi057-77576-24"}]: dispatch 2024-06-11T13:32:12.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:11 smithi107 ceph-mon[33178]: from='client.35852 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi057-77351-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:12.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:11 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:12.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:11 smithi107 ceph-mon[33178]: pgmap v166: 492 pgs: 128 unknown, 364 active+clean; 587 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-06-11T13:32:12.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:11 smithi107 ceph-mon[33178]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:32:12.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:11 smithi107 ceph-mon[33178]: from='client.35852 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi057-77351-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:12.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:11 smithi107 ceph-mon[33178]: osdmap e133: 8 total, 8 up, 8 in 2024-06-11T13:32:12.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:11 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1493964945' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi057-77799-10"}]: dispatch 2024-06-11T13:32:12.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:11 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:12.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:11 smithi107 ceph-mon[33178]: from='client.35759 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi057-77799-10"}]: dispatch 2024-06-11T13:32:12.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:11 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:13.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:12 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:32:13.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:12 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:13.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:12 smithi057 ceph-mon[40588]: from='client.25849 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi057-77576-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi057-77576-24"}]': finished 2024-06-11T13:32:13.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:12 smithi057 ceph-mon[40588]: from='client.35759 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi057-77799-10"}]': finished 2024-06-11T13:32:13.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:12 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:13.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:12 smithi057 ceph-mon[40588]: osdmap e134: 8 total, 8 up, 8 in 2024-06-11T13:32:13.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:12 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1493964945' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi057-77799-10"}]: dispatch 2024-06-11T13:32:13.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:12 smithi057 ceph-mon[40588]: from='client.35759 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi057-77799-10"}]: dispatch 2024-06-11T13:32:13.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:12 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1093780975' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi057-77362-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:13.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:12 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:13.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:12 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:32:13.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:12 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:13.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:12 smithi057 ceph-mon[29805]: from='client.25849 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi057-77576-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi057-77576-24"}]': finished 2024-06-11T13:32:13.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:12 smithi057 ceph-mon[29805]: from='client.35759 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi057-77799-10"}]': finished 2024-06-11T13:32:13.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:12 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:13.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:12 smithi057 ceph-mon[29805]: osdmap e134: 8 total, 8 up, 8 in 2024-06-11T13:32:13.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:12 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1493964945' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi057-77799-10"}]: dispatch 2024-06-11T13:32:13.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:12 smithi057 ceph-mon[29805]: from='client.35759 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi057-77799-10"}]: dispatch 2024-06-11T13:32:13.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:12 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1093780975' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi057-77362-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:13.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:12 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:13.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:12 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:32:13.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:12 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:13.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:12 smithi107 ceph-mon[33178]: from='client.25849 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi057-77576-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi057-77576-24"}]': finished 2024-06-11T13:32:13.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:12 smithi107 ceph-mon[33178]: from='client.35759 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi057-77799-10"}]': finished 2024-06-11T13:32:13.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:12 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:13.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:12 smithi107 ceph-mon[33178]: osdmap e134: 8 total, 8 up, 8 in 2024-06-11T13:32:13.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:12 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1493964945' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi057-77799-10"}]: dispatch 2024-06-11T13:32:13.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:12 smithi107 ceph-mon[33178]: from='client.35759 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi057-77799-10"}]: dispatch 2024-06-11T13:32:13.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:12 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1093780975' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi057-77362-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:13.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:12 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:14.033 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:32:13 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:32:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:32:14.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:13 smithi057 ceph-mon[40588]: pgmap v169: 460 pgs: 21 creating+peering, 51 unknown, 388 active+clean; 588 KiB data, 475 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:32:14.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:13 smithi057 ceph-mon[40588]: from='client.35759 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi057-77799-10"}]': finished 2024-06-11T13:32:14.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:13 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1093780975' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi057-77362-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:14.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:13 smithi057 ceph-mon[40588]: osdmap e135: 8 total, 8 up, 8 in 2024-06-11T13:32:14.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:13 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1824182913' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77576-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:14.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:13 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/4225682890' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi057-77351-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:14.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:13 smithi057 ceph-mon[40588]: from='client.25849 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77576-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:14.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:13 smithi057 ceph-mon[40588]: from='client.25858 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi057-77351-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:14.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:13 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3693143549' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi057-77799-15"}]: dispatch 2024-06-11T13:32:14.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:13 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3693143549' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi057-77799-15"}]: dispatch 2024-06-11T13:32:14.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:13 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3693143549' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi057-77799-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:32:14.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:13 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:14.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:13 smithi057 ceph-mon[29805]: pgmap v169: 460 pgs: 21 creating+peering, 51 unknown, 388 active+clean; 588 KiB data, 475 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:32:14.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:13 smithi057 ceph-mon[29805]: from='client.35759 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi057-77799-10"}]': finished 2024-06-11T13:32:14.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:13 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1093780975' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi057-77362-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:14.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:13 smithi057 ceph-mon[29805]: osdmap e135: 8 total, 8 up, 8 in 2024-06-11T13:32:14.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:13 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1824182913' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77576-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:14.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:13 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/4225682890' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi057-77351-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:14.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:13 smithi057 ceph-mon[29805]: from='client.25849 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77576-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:14.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:13 smithi057 ceph-mon[29805]: from='client.25858 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi057-77351-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:14.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:13 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3693143549' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi057-77799-15"}]: dispatch 2024-06-11T13:32:14.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:13 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3693143549' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi057-77799-15"}]: dispatch 2024-06-11T13:32:14.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:13 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3693143549' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi057-77799-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:32:14.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:13 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:14.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:13 smithi107 ceph-mon[33178]: pgmap v169: 460 pgs: 21 creating+peering, 51 unknown, 388 active+clean; 588 KiB data, 475 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:32:14.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:13 smithi107 ceph-mon[33178]: from='client.35759 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi057-77799-10"}]': finished 2024-06-11T13:32:14.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:13 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1093780975' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi057-77362-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:14.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:13 smithi107 ceph-mon[33178]: osdmap e135: 8 total, 8 up, 8 in 2024-06-11T13:32:14.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:13 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1824182913' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77576-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:14.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:13 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/4225682890' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi057-77351-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:14.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:13 smithi107 ceph-mon[33178]: from='client.25849 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77576-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:14.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:13 smithi107 ceph-mon[33178]: from='client.25858 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi057-77351-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:14.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:13 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3693143549' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi057-77799-15"}]: dispatch 2024-06-11T13:32:14.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:13 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3693143549' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi057-77799-15"}]: dispatch 2024-06-11T13:32:14.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:13 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3693143549' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi057-77799-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:32:14.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:13 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:15.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:14 smithi057 ceph-mon[29805]: from='client.25849 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77576-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:15.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:14 smithi057 ceph-mon[29805]: from='client.25858 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi057-77351-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:15.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:14 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3693143549' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi057-77799-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:32:15.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:14 smithi057 ceph-mon[29805]: osdmap e136: 8 total, 8 up, 8 in 2024-06-11T13:32:15.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:14 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1824182913' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77576-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:15.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:14 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3693143549' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi057-77799-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi057-77799-15"}]: dispatch 2024-06-11T13:32:15.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:14 smithi057 ceph-mon[29805]: from='client.25849 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77576-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:15.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:14 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:15.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:14 smithi057 ceph-mon[40588]: from='client.25849 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77576-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:15.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:14 smithi057 ceph-mon[40588]: from='client.25858 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi057-77351-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:15.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:14 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3693143549' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi057-77799-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:32:15.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:14 smithi057 ceph-mon[40588]: osdmap e136: 8 total, 8 up, 8 in 2024-06-11T13:32:15.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:14 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1824182913' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77576-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:15.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:14 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3693143549' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi057-77799-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi057-77799-15"}]: dispatch 2024-06-11T13:32:15.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:14 smithi057 ceph-mon[40588]: from='client.25849 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77576-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:15.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:14 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:15.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:14 smithi107 ceph-mon[33178]: from='client.25849 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77576-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:15.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:14 smithi107 ceph-mon[33178]: from='client.25858 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi057-77351-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:15.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:14 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3693143549' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi057-77799-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:32:15.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:14 smithi107 ceph-mon[33178]: osdmap e136: 8 total, 8 up, 8 in 2024-06-11T13:32:15.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:14 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1824182913' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77576-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:15.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:14 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3693143549' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi057-77799-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi057-77799-15"}]: dispatch 2024-06-11T13:32:15.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:14 smithi107 ceph-mon[33178]: from='client.25849 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77576-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:15.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:14 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:16.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:15 smithi057 ceph-mon[29805]: pgmap v172: 492 pgs: 21 creating+peering, 83 unknown, 388 active+clean; 588 KiB data, 475 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-06-11T13:32:16.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:15 smithi057 ceph-mon[29805]: from='client.25849 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77576-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:16.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:15 smithi057 ceph-mon[29805]: osdmap e137: 8 total, 8 up, 8 in 2024-06-11T13:32:16.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:15 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/4021667148' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi057-77362-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:16.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:15 smithi057 ceph-mon[29805]: from='client.25867 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi057-77362-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:16.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:15 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:16.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:15 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3693143549' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi057-77799-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi057-77799-15"}]': finished 2024-06-11T13:32:16.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:15 smithi057 ceph-mon[29805]: from='client.25867 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi057-77362-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:16.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:15 smithi057 ceph-mon[29805]: osdmap e138: 8 total, 8 up, 8 in 2024-06-11T13:32:16.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:15 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1860541746' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi057-77351-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:16.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:15 smithi057 ceph-mon[29805]: from='client.35885 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi057-77351-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:16.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:15 smithi057 ceph-mon[40588]: pgmap v172: 492 pgs: 21 creating+peering, 83 unknown, 388 active+clean; 588 KiB data, 475 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-06-11T13:32:16.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:15 smithi057 ceph-mon[40588]: from='client.25849 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77576-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:16.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:15 smithi057 ceph-mon[40588]: osdmap e137: 8 total, 8 up, 8 in 2024-06-11T13:32:16.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:15 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/4021667148' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi057-77362-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:16.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:15 smithi057 ceph-mon[40588]: from='client.25867 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi057-77362-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:16.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:15 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:16.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:15 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3693143549' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi057-77799-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi057-77799-15"}]': finished 2024-06-11T13:32:16.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:15 smithi057 ceph-mon[40588]: from='client.25867 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi057-77362-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:16.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:15 smithi057 ceph-mon[40588]: osdmap e138: 8 total, 8 up, 8 in 2024-06-11T13:32:16.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:15 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1860541746' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi057-77351-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:16.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:15 smithi057 ceph-mon[40588]: from='client.35885 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi057-77351-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:16.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:15 smithi107 ceph-mon[33178]: pgmap v172: 492 pgs: 21 creating+peering, 83 unknown, 388 active+clean; 588 KiB data, 475 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-06-11T13:32:16.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:15 smithi107 ceph-mon[33178]: from='client.25849 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77576-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:16.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:15 smithi107 ceph-mon[33178]: osdmap e137: 8 total, 8 up, 8 in 2024-06-11T13:32:16.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:15 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/4021667148' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi057-77362-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:16.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:15 smithi107 ceph-mon[33178]: from='client.25867 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi057-77362-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:16.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:15 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:16.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:15 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3693143549' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi057-77799-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi057-77799-15"}]': finished 2024-06-11T13:32:16.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:15 smithi107 ceph-mon[33178]: from='client.25867 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi057-77362-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:16.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:15 smithi107 ceph-mon[33178]: osdmap e138: 8 total, 8 up, 8 in 2024-06-11T13:32:16.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:15 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1860541746' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi057-77351-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:16.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:15 smithi107 ceph-mon[33178]: from='client.35885 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi057-77351-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:17.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:16 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:32:17.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:16 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:32:17.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:16 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:17.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:16 smithi057 ceph-mon[40588]: from='client.35885 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi057-77351-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:17.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:16 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-13", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:32:17.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:16 smithi057 ceph-mon[40588]: osdmap e139: 8 total, 8 up, 8 in 2024-06-11T13:32:17.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:16 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1824182913' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi057-77576-24"}]: dispatch 2024-06-11T13:32:17.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:16 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-13"}]: dispatch 2024-06-11T13:32:17.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:16 smithi057 ceph-mon[40588]: from='client.25849 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi057-77576-24"}]: dispatch 2024-06-11T13:32:17.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:16 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-13"}]: dispatch 2024-06-11T13:32:17.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:16 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:32:17.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:16 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:32:17.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:16 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:17.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:16 smithi057 ceph-mon[29805]: from='client.35885 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi057-77351-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:17.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:16 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-13", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:32:17.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:16 smithi057 ceph-mon[29805]: osdmap e139: 8 total, 8 up, 8 in 2024-06-11T13:32:17.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:16 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1824182913' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi057-77576-24"}]: dispatch 2024-06-11T13:32:17.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:16 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-13"}]: dispatch 2024-06-11T13:32:17.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:16 smithi057 ceph-mon[29805]: from='client.25849 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi057-77576-24"}]: dispatch 2024-06-11T13:32:17.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:16 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-13"}]: dispatch 2024-06-11T13:32:17.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:16 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:32:17.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:16 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:32:17.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:16 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:17.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:16 smithi107 ceph-mon[33178]: from='client.35885 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi057-77351-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:17.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:16 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-13", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:32:17.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:16 smithi107 ceph-mon[33178]: osdmap e139: 8 total, 8 up, 8 in 2024-06-11T13:32:17.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:16 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1824182913' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi057-77576-24"}]: dispatch 2024-06-11T13:32:17.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:16 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-13"}]: dispatch 2024-06-11T13:32:17.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:16 smithi107 ceph-mon[33178]: from='client.25849 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi057-77576-24"}]: dispatch 2024-06-11T13:32:17.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:16 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-13"}]: dispatch 2024-06-11T13:32:18.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:17 smithi057 ceph-mon[40588]: pgmap v175: 500 pgs: 40 creating+peering, 460 active+clean; 593 KiB data, 520 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 127 KiB/s wr, 129 op/s 2024-06-11T13:32:18.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:17 smithi057 ceph-mon[40588]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:32:18.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:17 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:18.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:17 smithi057 ceph-mon[40588]: from='client.25849 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi057-77576-24"}]': finished 2024-06-11T13:32:18.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:17 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-13"}]': finished 2024-06-11T13:32:18.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:17 smithi057 ceph-mon[40588]: osdmap e140: 8 total, 8 up, 8 in 2024-06-11T13:32:18.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:17 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3557990031' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi057-77362-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:18.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:17 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1824182913' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi057-77576-24"}]: dispatch 2024-06-11T13:32:18.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:17 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-13", "mode": "writeback"}]: dispatch 2024-06-11T13:32:18.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:17 smithi057 ceph-mon[40588]: from='client.25849 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi057-77576-24"}]: dispatch 2024-06-11T13:32:18.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:17 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-13", "mode": "writeback"}]: dispatch 2024-06-11T13:32:18.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:17 smithi057 ceph-mon[29805]: pgmap v175: 500 pgs: 40 creating+peering, 460 active+clean; 593 KiB data, 520 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 127 KiB/s wr, 129 op/s 2024-06-11T13:32:18.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:17 smithi057 ceph-mon[29805]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:32:18.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:17 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:18.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:17 smithi057 ceph-mon[29805]: from='client.25849 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi057-77576-24"}]': finished 2024-06-11T13:32:18.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:17 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-13"}]': finished 2024-06-11T13:32:18.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:17 smithi057 ceph-mon[29805]: osdmap e140: 8 total, 8 up, 8 in 2024-06-11T13:32:18.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:17 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3557990031' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi057-77362-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:18.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:17 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1824182913' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi057-77576-24"}]: dispatch 2024-06-11T13:32:18.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:17 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-13", "mode": "writeback"}]: dispatch 2024-06-11T13:32:18.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:17 smithi057 ceph-mon[29805]: from='client.25849 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi057-77576-24"}]: dispatch 2024-06-11T13:32:18.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:17 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-13", "mode": "writeback"}]: dispatch 2024-06-11T13:32:18.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:17 smithi107 ceph-mon[33178]: pgmap v175: 500 pgs: 40 creating+peering, 460 active+clean; 593 KiB data, 520 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 127 KiB/s wr, 129 op/s 2024-06-11T13:32:18.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:17 smithi107 ceph-mon[33178]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:32:18.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:17 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:18.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:17 smithi107 ceph-mon[33178]: from='client.25849 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi057-77576-24"}]': finished 2024-06-11T13:32:18.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:17 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-13"}]': finished 2024-06-11T13:32:18.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:17 smithi107 ceph-mon[33178]: osdmap e140: 8 total, 8 up, 8 in 2024-06-11T13:32:18.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:17 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3557990031' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi057-77362-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:18.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:17 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1824182913' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi057-77576-24"}]: dispatch 2024-06-11T13:32:18.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:17 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-13", "mode": "writeback"}]: dispatch 2024-06-11T13:32:18.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:17 smithi107 ceph-mon[33178]: from='client.25849 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi057-77576-24"}]: dispatch 2024-06-11T13:32:18.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:17 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-13", "mode": "writeback"}]: dispatch 2024-06-11T13:32:19.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:18 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:19.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:18 smithi057 ceph-mon[29805]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:32:19.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:18 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3557990031' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi057-77362-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:19.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:18 smithi057 ceph-mon[29805]: from='client.25849 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi057-77576-24"}]': finished 2024-06-11T13:32:19.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:18 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-13", "mode": "writeback"}]': finished 2024-06-11T13:32:19.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:18 smithi057 ceph-mon[29805]: osdmap e141: 8 total, 8 up, 8 in 2024-06-11T13:32:19.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:18 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3321705142' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi057-77351-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:19.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:18 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:19.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:18 smithi057 ceph-mon[40588]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:32:19.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:18 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3557990031' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi057-77362-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:19.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:18 smithi057 ceph-mon[40588]: from='client.25849 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi057-77576-24"}]': finished 2024-06-11T13:32:19.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:18 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-13", "mode": "writeback"}]': finished 2024-06-11T13:32:19.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:18 smithi057 ceph-mon[40588]: osdmap e141: 8 total, 8 up, 8 in 2024-06-11T13:32:19.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:18 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3321705142' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi057-77351-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:19.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:18 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:19.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:18 smithi107 ceph-mon[33178]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:32:19.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:18 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3557990031' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi057-77362-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:19.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:18 smithi107 ceph-mon[33178]: from='client.25849 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi057-77576-24"}]': finished 2024-06-11T13:32:19.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:18 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-13", "mode": "writeback"}]': finished 2024-06-11T13:32:19.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:18 smithi107 ceph-mon[33178]: osdmap e141: 8 total, 8 up, 8 in 2024-06-11T13:32:19.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:18 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3321705142' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi057-77351-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:20.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:19 smithi057 ceph-mon[29805]: pgmap v178: 460 pgs: 64 unknown, 8 creating+peering, 388 active+clean; 592 KiB data, 520 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 124 KiB/s wr, 124 op/s 2024-06-11T13:32:20.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:19 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:20.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:19 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.0"}]: dispatch 2024-06-11T13:32:20.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:19 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.1"}]: dispatch 2024-06-11T13:32:20.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:19 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.2"}]: dispatch 2024-06-11T13:32:20.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:19 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.3"}]: dispatch 2024-06-11T13:32:20.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:19 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.4"}]: dispatch 2024-06-11T13:32:20.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:19 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.5"}]: dispatch 2024-06-11T13:32:20.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:19 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.6"}]: dispatch 2024-06-11T13:32:20.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:19 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.7"}]: dispatch 2024-06-11T13:32:20.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:19 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.8"}]: dispatch 2024-06-11T13:32:20.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:19 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.9"}]: dispatch 2024-06-11T13:32:20.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:19 smithi057 ceph-mon[29805]: 174.3 deep-scrub starts 2024-06-11T13:32:20.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:19 smithi057 ceph-mon[29805]: 174.3 deep-scrub ok 2024-06-11T13:32:20.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:19 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3321705142' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi057-77351-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:20.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:19 smithi057 ceph-mon[29805]: osdmap e142: 8 total, 8 up, 8 in 2024-06-11T13:32:20.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:19 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1475768091' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi057-77576-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:20.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:19 smithi057 ceph-mon[40588]: pgmap v178: 460 pgs: 64 unknown, 8 creating+peering, 388 active+clean; 592 KiB data, 520 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 124 KiB/s wr, 124 op/s 2024-06-11T13:32:20.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:19 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:20.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:19 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.0"}]: dispatch 2024-06-11T13:32:20.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:19 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.1"}]: dispatch 2024-06-11T13:32:20.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:19 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.2"}]: dispatch 2024-06-11T13:32:20.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:19 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.3"}]: dispatch 2024-06-11T13:32:20.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:19 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.4"}]: dispatch 2024-06-11T13:32:20.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:19 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.5"}]: dispatch 2024-06-11T13:32:20.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:19 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.6"}]: dispatch 2024-06-11T13:32:20.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:19 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.7"}]: dispatch 2024-06-11T13:32:20.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:19 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.8"}]: dispatch 2024-06-11T13:32:20.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:19 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.9"}]: dispatch 2024-06-11T13:32:20.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:19 smithi057 ceph-mon[40588]: 174.3 deep-scrub starts 2024-06-11T13:32:20.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:19 smithi057 ceph-mon[40588]: 174.3 deep-scrub ok 2024-06-11T13:32:20.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:19 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3321705142' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi057-77351-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:20.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:19 smithi057 ceph-mon[40588]: osdmap e142: 8 total, 8 up, 8 in 2024-06-11T13:32:20.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:19 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1475768091' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi057-77576-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:20.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:19 smithi107 ceph-mon[33178]: pgmap v178: 460 pgs: 64 unknown, 8 creating+peering, 388 active+clean; 592 KiB data, 520 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 124 KiB/s wr, 124 op/s 2024-06-11T13:32:20.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:19 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:20.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:19 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.0"}]: dispatch 2024-06-11T13:32:20.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:19 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.1"}]: dispatch 2024-06-11T13:32:20.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:19 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.2"}]: dispatch 2024-06-11T13:32:20.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:19 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.3"}]: dispatch 2024-06-11T13:32:20.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:19 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.4"}]: dispatch 2024-06-11T13:32:20.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:19 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.5"}]: dispatch 2024-06-11T13:32:20.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:19 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.6"}]: dispatch 2024-06-11T13:32:20.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:19 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.7"}]: dispatch 2024-06-11T13:32:20.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:19 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.8"}]: dispatch 2024-06-11T13:32:20.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:19 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.9"}]: dispatch 2024-06-11T13:32:20.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:19 smithi107 ceph-mon[33178]: 174.3 deep-scrub starts 2024-06-11T13:32:20.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:19 smithi107 ceph-mon[33178]: 174.3 deep-scrub ok 2024-06-11T13:32:20.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:19 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3321705142' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi057-77351-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:20.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:19 smithi107 ceph-mon[33178]: osdmap e142: 8 total, 8 up, 8 in 2024-06-11T13:32:20.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:19 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1475768091' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi057-77576-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:21.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:20 smithi057 ceph-mon[29805]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.0"}]: dispatch 2024-06-11T13:32:21.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:20 smithi057 ceph-mon[29805]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.1"}]: dispatch 2024-06-11T13:32:21.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:20 smithi057 ceph-mon[29805]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.2"}]: dispatch 2024-06-11T13:32:21.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:20 smithi057 ceph-mon[29805]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.3"}]: dispatch 2024-06-11T13:32:21.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:20 smithi057 ceph-mon[29805]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.4"}]: dispatch 2024-06-11T13:32:21.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:20 smithi057 ceph-mon[29805]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.5"}]: dispatch 2024-06-11T13:32:21.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:20 smithi057 ceph-mon[29805]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.6"}]: dispatch 2024-06-11T13:32:21.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:20 smithi057 ceph-mon[29805]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.7"}]: dispatch 2024-06-11T13:32:21.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:20 smithi057 ceph-mon[29805]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.8"}]: dispatch 2024-06-11T13:32:21.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:20 smithi057 ceph-mon[29805]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.9"}]: dispatch 2024-06-11T13:32:21.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:20 smithi057 ceph-mon[29805]: 174.8 scrub starts 2024-06-11T13:32:21.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:20 smithi057 ceph-mon[29805]: 174.8 scrub ok 2024-06-11T13:32:21.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:20 smithi057 ceph-mon[29805]: 174.7 scrub starts 2024-06-11T13:32:21.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:20 smithi057 ceph-mon[29805]: 174.7 scrub ok 2024-06-11T13:32:21.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:20 smithi057 ceph-mon[29805]: 174.2 scrub starts 2024-06-11T13:32:21.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:20 smithi057 ceph-mon[29805]: 174.2 scrub ok 2024-06-11T13:32:21.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:20 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:21.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:20 smithi057 ceph-mon[29805]: 174.1 scrub starts 2024-06-11T13:32:21.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:20 smithi057 ceph-mon[29805]: 174.1 scrub ok 2024-06-11T13:32:21.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:20 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1475768091' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi057-77576-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:21.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:20 smithi057 ceph-mon[29805]: osdmap e143: 8 total, 8 up, 8 in 2024-06-11T13:32:21.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:20 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1124651616' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi057-77362-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:21.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:20 smithi057 ceph-mon[29805]: from='client.35912 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi057-77362-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:21.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:20 smithi057 ceph-mon[40588]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.0"}]: dispatch 2024-06-11T13:32:21.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:20 smithi057 ceph-mon[40588]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.1"}]: dispatch 2024-06-11T13:32:21.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:20 smithi057 ceph-mon[40588]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.2"}]: dispatch 2024-06-11T13:32:21.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:20 smithi057 ceph-mon[40588]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.3"}]: dispatch 2024-06-11T13:32:21.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:20 smithi057 ceph-mon[40588]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.4"}]: dispatch 2024-06-11T13:32:21.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:20 smithi057 ceph-mon[40588]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.5"}]: dispatch 2024-06-11T13:32:21.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:20 smithi057 ceph-mon[40588]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.6"}]: dispatch 2024-06-11T13:32:21.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:20 smithi057 ceph-mon[40588]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.7"}]: dispatch 2024-06-11T13:32:21.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:20 smithi057 ceph-mon[40588]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.8"}]: dispatch 2024-06-11T13:32:21.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:20 smithi057 ceph-mon[40588]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.9"}]: dispatch 2024-06-11T13:32:21.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:20 smithi057 ceph-mon[40588]: 174.8 scrub starts 2024-06-11T13:32:21.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:20 smithi057 ceph-mon[40588]: 174.8 scrub ok 2024-06-11T13:32:21.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:20 smithi057 ceph-mon[40588]: 174.7 scrub starts 2024-06-11T13:32:21.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:20 smithi057 ceph-mon[40588]: 174.7 scrub ok 2024-06-11T13:32:21.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:20 smithi057 ceph-mon[40588]: 174.2 scrub starts 2024-06-11T13:32:21.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:20 smithi057 ceph-mon[40588]: 174.2 scrub ok 2024-06-11T13:32:21.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:20 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:21.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:20 smithi057 ceph-mon[40588]: 174.1 scrub starts 2024-06-11T13:32:21.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:20 smithi057 ceph-mon[40588]: 174.1 scrub ok 2024-06-11T13:32:21.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:20 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1475768091' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi057-77576-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:21.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:20 smithi057 ceph-mon[40588]: osdmap e143: 8 total, 8 up, 8 in 2024-06-11T13:32:21.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:20 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1124651616' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi057-77362-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:21.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:20 smithi057 ceph-mon[40588]: from='client.35912 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi057-77362-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:21.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:20 smithi107 ceph-mon[33178]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.0"}]: dispatch 2024-06-11T13:32:21.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:20 smithi107 ceph-mon[33178]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.1"}]: dispatch 2024-06-11T13:32:21.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:20 smithi107 ceph-mon[33178]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.2"}]: dispatch 2024-06-11T13:32:21.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:20 smithi107 ceph-mon[33178]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.3"}]: dispatch 2024-06-11T13:32:21.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:20 smithi107 ceph-mon[33178]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.4"}]: dispatch 2024-06-11T13:32:21.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:20 smithi107 ceph-mon[33178]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.5"}]: dispatch 2024-06-11T13:32:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:20 smithi107 ceph-mon[33178]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.6"}]: dispatch 2024-06-11T13:32:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:20 smithi107 ceph-mon[33178]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.7"}]: dispatch 2024-06-11T13:32:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:20 smithi107 ceph-mon[33178]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.8"}]: dispatch 2024-06-11T13:32:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:20 smithi107 ceph-mon[33178]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.9"}]: dispatch 2024-06-11T13:32:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:20 smithi107 ceph-mon[33178]: 174.8 scrub starts 2024-06-11T13:32:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:20 smithi107 ceph-mon[33178]: 174.8 scrub ok 2024-06-11T13:32:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:20 smithi107 ceph-mon[33178]: 174.7 scrub starts 2024-06-11T13:32:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:20 smithi107 ceph-mon[33178]: 174.7 scrub ok 2024-06-11T13:32:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:20 smithi107 ceph-mon[33178]: 174.2 scrub starts 2024-06-11T13:32:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:20 smithi107 ceph-mon[33178]: 174.2 scrub ok 2024-06-11T13:32:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:20 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:20 smithi107 ceph-mon[33178]: 174.1 scrub starts 2024-06-11T13:32:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:20 smithi107 ceph-mon[33178]: 174.1 scrub ok 2024-06-11T13:32:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:20 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1475768091' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi057-77576-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:21.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:20 smithi107 ceph-mon[33178]: osdmap e143: 8 total, 8 up, 8 in 2024-06-11T13:32:21.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:20 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1124651616' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi057-77362-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:21.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:20 smithi107 ceph-mon[33178]: from='client.35912 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi057-77362-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:21.547 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:32:21 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:32:22.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:21 smithi057 ceph-mon[29805]: 174.6 scrub starts 2024-06-11T13:32:22.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:21 smithi057 ceph-mon[29805]: 174.6 scrub ok 2024-06-11T13:32:22.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:21 smithi057 ceph-mon[29805]: pgmap v181: 492 pgs: 96 unknown, 8 creating+peering, 388 active+clean; 592 KiB data, 520 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:32:22.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:21 smithi057 ceph-mon[29805]: 174.9 scrub starts 2024-06-11T13:32:22.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:21 smithi057 ceph-mon[29805]: 174.9 scrub ok 2024-06-11T13:32:22.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:21 smithi057 ceph-mon[29805]: 174.0 scrub starts 2024-06-11T13:32:22.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:21 smithi057 ceph-mon[29805]: 174.0 scrub ok 2024-06-11T13:32:22.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:21 smithi057 ceph-mon[29805]: 174.4 scrub starts 2024-06-11T13:32:22.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:21 smithi057 ceph-mon[29805]: 174.4 scrub ok 2024-06-11T13:32:22.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:21 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:32:22.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:21 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:22.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:21 smithi057 ceph-mon[29805]: 174.5 scrub starts 2024-06-11T13:32:22.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:21 smithi057 ceph-mon[29805]: 174.5 scrub ok 2024-06-11T13:32:22.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:21 smithi057 ceph-mon[29805]: from='client.35912 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi057-77362-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:22.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:21 smithi057 ceph-mon[29805]: osdmap e144: 8 total, 8 up, 8 in 2024-06-11T13:32:22.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:21 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3599688193' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi057-77351-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:22.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:21 smithi057 ceph-mon[29805]: from='client.35918 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi057-77351-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:22.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:21 smithi057 ceph-mon[40588]: 174.6 scrub starts 2024-06-11T13:32:22.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:21 smithi057 ceph-mon[40588]: 174.6 scrub ok 2024-06-11T13:32:22.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:21 smithi057 ceph-mon[40588]: pgmap v181: 492 pgs: 96 unknown, 8 creating+peering, 388 active+clean; 592 KiB data, 520 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:32:22.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:21 smithi057 ceph-mon[40588]: 174.9 scrub starts 2024-06-11T13:32:22.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:21 smithi057 ceph-mon[40588]: 174.9 scrub ok 2024-06-11T13:32:22.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:21 smithi057 ceph-mon[40588]: 174.0 scrub starts 2024-06-11T13:32:22.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:21 smithi057 ceph-mon[40588]: 174.0 scrub ok 2024-06-11T13:32:22.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:21 smithi057 ceph-mon[40588]: 174.4 scrub starts 2024-06-11T13:32:22.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:21 smithi057 ceph-mon[40588]: 174.4 scrub ok 2024-06-11T13:32:22.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:21 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:32:22.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:21 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:22.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:21 smithi057 ceph-mon[40588]: 174.5 scrub starts 2024-06-11T13:32:22.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:21 smithi057 ceph-mon[40588]: 174.5 scrub ok 2024-06-11T13:32:22.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:21 smithi057 ceph-mon[40588]: from='client.35912 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi057-77362-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:22.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:21 smithi057 ceph-mon[40588]: osdmap e144: 8 total, 8 up, 8 in 2024-06-11T13:32:22.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:21 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3599688193' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi057-77351-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:22.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:21 smithi057 ceph-mon[40588]: from='client.35918 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi057-77351-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:22.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:21 smithi107 ceph-mon[33178]: 174.6 scrub starts 2024-06-11T13:32:22.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:21 smithi107 ceph-mon[33178]: 174.6 scrub ok 2024-06-11T13:32:22.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:21 smithi107 ceph-mon[33178]: pgmap v181: 492 pgs: 96 unknown, 8 creating+peering, 388 active+clean; 592 KiB data, 520 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:32:22.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:21 smithi107 ceph-mon[33178]: 174.9 scrub starts 2024-06-11T13:32:22.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:21 smithi107 ceph-mon[33178]: 174.9 scrub ok 2024-06-11T13:32:22.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:21 smithi107 ceph-mon[33178]: 174.0 scrub starts 2024-06-11T13:32:22.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:21 smithi107 ceph-mon[33178]: 174.0 scrub ok 2024-06-11T13:32:22.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:21 smithi107 ceph-mon[33178]: 174.4 scrub starts 2024-06-11T13:32:22.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:21 smithi107 ceph-mon[33178]: 174.4 scrub ok 2024-06-11T13:32:22.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:21 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:32:22.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:21 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:22.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:21 smithi107 ceph-mon[33178]: 174.5 scrub starts 2024-06-11T13:32:22.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:21 smithi107 ceph-mon[33178]: 174.5 scrub ok 2024-06-11T13:32:22.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:21 smithi107 ceph-mon[33178]: from='client.35912 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi057-77362-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:22.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:21 smithi107 ceph-mon[33178]: osdmap e144: 8 total, 8 up, 8 in 2024-06-11T13:32:22.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:21 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3599688193' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi057-77351-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:22.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:21 smithi107 ceph-mon[33178]: from='client.35918 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi057-77351-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:23.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:22 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:32:23.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:22 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:23.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:22 smithi057 ceph-mon[40588]: from='client.35918 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi057-77351-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:23.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:22 smithi057 ceph-mon[40588]: osdmap e145: 8 total, 8 up, 8 in 2024-06-11T13:32:23.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:22 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/215959717' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi057-77576-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:23.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:22 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:32:23.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:22 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:23.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:22 smithi057 ceph-mon[29805]: from='client.35918 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi057-77351-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:23.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:22 smithi057 ceph-mon[29805]: osdmap e145: 8 total, 8 up, 8 in 2024-06-11T13:32:23.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:22 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/215959717' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi057-77576-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:23.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:22 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:32:23.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:22 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:23.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:22 smithi107 ceph-mon[33178]: from='client.35918 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi057-77351-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:23.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:22 smithi107 ceph-mon[33178]: osdmap e145: 8 total, 8 up, 8 in 2024-06-11T13:32:23.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:22 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/215959717' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi057-77576-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:24.034 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:32:23 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:32:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:32:24.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:24 smithi057 ceph-mon[40588]: pgmap v184: 492 pgs: 32 creating+peering, 32 unknown, 428 active+clean; 602 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s rd, 262 KiB/s wr, 78 op/s 2024-06-11T13:32:24.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:24 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:32:24.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:24 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:32:24.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:24 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:32:24.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:24 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:24.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:24 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:32:24.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:24 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/215959717' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi057-77576-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:24.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:24 smithi057 ceph-mon[40588]: osdmap e146: 8 total, 8 up, 8 in 2024-06-11T13:32:24.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:24 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/413331880' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi057-77362-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:24.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:24 smithi057 ceph-mon[29805]: pgmap v184: 492 pgs: 32 creating+peering, 32 unknown, 428 active+clean; 602 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s rd, 262 KiB/s wr, 78 op/s 2024-06-11T13:32:24.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:24 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:32:24.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:24 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:32:24.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:24 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:32:24.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:24 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:24.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:24 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:32:24.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:24 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/215959717' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi057-77576-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:24.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:24 smithi057 ceph-mon[29805]: osdmap e146: 8 total, 8 up, 8 in 2024-06-11T13:32:24.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:24 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/413331880' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi057-77362-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:24.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:24 smithi107 ceph-mon[33178]: pgmap v184: 492 pgs: 32 creating+peering, 32 unknown, 428 active+clean; 602 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s rd, 262 KiB/s wr, 78 op/s 2024-06-11T13:32:24.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:24 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:32:24.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:24 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:32:24.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:24 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:32:24.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:24 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:24.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:24 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:32:24.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:24 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/215959717' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi057-77576-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:24.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:24 smithi107 ceph-mon[33178]: osdmap e146: 8 total, 8 up, 8 in 2024-06-11T13:32:24.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:24 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/413331880' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi057-77362-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:25.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:25 smithi057 ceph-mon[29805]: pgmap v186: 460 pgs: 64 unknown, 396 active+clean; 602 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 66 KiB/s rd, 6.7 KiB/s wr, 87 op/s 2024-06-11T13:32:25.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:25 smithi057 ceph-mon[29805]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:32:25.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:25 smithi057 ceph-mon[29805]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 2 stray daemon(s) not managed by cephadm) 2024-06-11T13:32:25.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:25 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:25.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:25 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/413331880' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi057-77362-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:25.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:25 smithi057 ceph-mon[29805]: osdmap e147: 8 total, 8 up, 8 in 2024-06-11T13:32:25.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:25 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1929035411' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi057-77351-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:25.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:25 smithi057 ceph-mon[29805]: from='client.25927 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi057-77351-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:25.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:25 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3693143549' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi057-77799-15"}]: dispatch 2024-06-11T13:32:25.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:25 smithi057 ceph-mon[40588]: pgmap v186: 460 pgs: 64 unknown, 396 active+clean; 602 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 66 KiB/s rd, 6.7 KiB/s wr, 87 op/s 2024-06-11T13:32:25.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:25 smithi057 ceph-mon[40588]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:32:25.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:25 smithi057 ceph-mon[40588]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 2 stray daemon(s) not managed by cephadm) 2024-06-11T13:32:25.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:25 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:25.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:25 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/413331880' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi057-77362-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:25.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:25 smithi057 ceph-mon[40588]: osdmap e147: 8 total, 8 up, 8 in 2024-06-11T13:32:25.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:25 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1929035411' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi057-77351-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:25.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:25 smithi057 ceph-mon[40588]: from='client.25927 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi057-77351-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:25.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:25 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3693143549' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi057-77799-15"}]: dispatch 2024-06-11T13:32:25.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:25 smithi107 ceph-mon[33178]: pgmap v186: 460 pgs: 64 unknown, 396 active+clean; 602 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 66 KiB/s rd, 6.7 KiB/s wr, 87 op/s 2024-06-11T13:32:25.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:25 smithi107 ceph-mon[33178]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:32:25.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:25 smithi107 ceph-mon[33178]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 2 stray daemon(s) not managed by cephadm) 2024-06-11T13:32:25.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:25 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:25.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:25 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/413331880' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi057-77362-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:25.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:25 smithi107 ceph-mon[33178]: osdmap e147: 8 total, 8 up, 8 in 2024-06-11T13:32:25.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:25 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1929035411' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi057-77351-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:25.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:25 smithi107 ceph-mon[33178]: from='client.25927 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi057-77351-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:25.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:25 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3693143549' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi057-77799-15"}]: dispatch 2024-06-11T13:32:26.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:26 smithi057 ceph-mon[40588]: pgmap v189: 484 pgs: 96 unknown, 388 active+clean; 594 KiB data, 604 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:32:26.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:26 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:26.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:26 smithi057 ceph-mon[40588]: from='client.25927 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi057-77351-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:26.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:26 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3693143549' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi057-77799-15"}]': finished 2024-06-11T13:32:26.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:26 smithi057 ceph-mon[40588]: osdmap e148: 8 total, 8 up, 8 in 2024-06-11T13:32:26.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:26 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3693143549' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi057-77799-15"}]: dispatch 2024-06-11T13:32:26.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:26 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3703062388' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi057-77576-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:26.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:26 smithi057 ceph-mon[29805]: pgmap v189: 484 pgs: 96 unknown, 388 active+clean; 594 KiB data, 604 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:32:26.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:26 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:26.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:26 smithi057 ceph-mon[29805]: from='client.25927 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi057-77351-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:26.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:26 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3693143549' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi057-77799-15"}]': finished 2024-06-11T13:32:26.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:26 smithi057 ceph-mon[29805]: osdmap e148: 8 total, 8 up, 8 in 2024-06-11T13:32:26.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:26 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3693143549' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi057-77799-15"}]: dispatch 2024-06-11T13:32:26.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:26 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3703062388' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi057-77576-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:26.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:26 smithi107 ceph-mon[33178]: pgmap v189: 484 pgs: 96 unknown, 388 active+clean; 594 KiB data, 604 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:32:26.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:26 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:26.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:26 smithi107 ceph-mon[33178]: from='client.25927 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi057-77351-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:26.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:26 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3693143549' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi057-77799-15"}]': finished 2024-06-11T13:32:26.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:26 smithi107 ceph-mon[33178]: osdmap e148: 8 total, 8 up, 8 in 2024-06-11T13:32:26.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:26 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3693143549' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi057-77799-15"}]: dispatch 2024-06-11T13:32:26.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:26 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3703062388' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi057-77576-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:26.671 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots: Running main() from gmock_main.cc 2024-06-11T13:32:26.671 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots: [==========] Running 13 tests from 4 test suites. 2024-06-11T13:32:26.671 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots: [----------] Global test environment set-up. 2024-06-11T13:32:26.671 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots 2024-06-11T13:32:26.671 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapList 2024-06-11T13:32:26.671 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapList (8564 ms) 2024-06-11T13:32:26.671 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapRemove 2024-06-11T13:32:26.672 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapRemove (7879 ms) 2024-06-11T13:32:26.672 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.Rollback 2024-06-11T13:32:26.672 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots: [ OK ] LibRadosSnapshots.Rollback (6551 ms) 2024-06-11T13:32:26.672 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapGetName 2024-06-11T13:32:26.672 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapGetName (2318 ms) 2024-06-11T13:32:26.672 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots (25312 ms total) 2024-06-11T13:32:26.672 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots: 2024-06-11T13:32:26.672 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged 2024-06-11T13:32:26.672 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Snap 2024-06-11T13:32:26.672 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Snap (7126 ms) 2024-06-11T13:32:26.672 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Rollback 2024-06-11T13:32:26.672 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Rollback (6497 ms) 2024-06-11T13:32:26.673 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.FutureSnapRollback 2024-06-11T13:32:26.673 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.FutureSnapRollback (7385 ms) 2024-06-11T13:32:26.673 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged (21008 ms total) 2024-06-11T13:32:26.673 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots: 2024-06-11T13:32:26.673 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC 2024-06-11T13:32:26.673 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapList 2024-06-11T13:32:26.673 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapList (4341 ms) 2024-06-11T13:32:26.673 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapRemove 2024-06-11T13:32:26.673 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapRemove (2398 ms) 2024-06-11T13:32:26.673 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.Rollback 2024-06-11T13:32:26.673 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.Rollback (2018 ms) 2024-06-11T13:32:26.673 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapGetName 2024-06-11T13:32:26.673 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapGetName (2016 ms) 2024-06-11T13:32:26.674 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC (10773 ms total) 2024-06-11T13:32:26.674 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots: 2024-06-11T13:32:26.674 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC 2024-06-11T13:32:26.674 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Snap 2024-06-11T13:32:26.674 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Snap (4057 ms) 2024-06-11T13:32:26.674 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Rollback 2024-06-11T13:32:26.674 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Rollback (4006 ms) 2024-06-11T13:32:26.674 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC (8063 ms total) 2024-06-11T13:32:26.674 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots: 2024-06-11T13:32:26.674 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots: [----------] Global test environment tear-down 2024-06-11T13:32:26.674 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots: [==========] 13 tests from 4 test suites ran. (102910 ms total) 2024-06-11T13:32:26.674 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots: [ PASSED ] 13 tests. 2024-06-11T13:32:28.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:28 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:28.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:28 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3693143549' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi057-77799-15"}]': finished 2024-06-11T13:32:28.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:28 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3703062388' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi057-77576-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:28.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:28 smithi057 ceph-mon[29805]: osdmap e149: 8 total, 8 up, 8 in 2024-06-11T13:32:28.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:28 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/4163257875' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi057-77362-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:28.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:28 smithi057 ceph-mon[29805]: from='client.35945 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi057-77362-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:28.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:28 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:28.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:28 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3693143549' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi057-77799-15"}]': finished 2024-06-11T13:32:28.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:28 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3703062388' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi057-77576-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:28.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:28 smithi057 ceph-mon[40588]: osdmap e149: 8 total, 8 up, 8 in 2024-06-11T13:32:28.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:28 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/4163257875' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi057-77362-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:28.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:28 smithi057 ceph-mon[40588]: from='client.35945 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi057-77362-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:28.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:28 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:28.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:28 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3693143549' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi057-77799-15"}]': finished 2024-06-11T13:32:28.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:28 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3703062388' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi057-77576-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:28.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:28 smithi107 ceph-mon[33178]: osdmap e149: 8 total, 8 up, 8 in 2024-06-11T13:32:28.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:28 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/4163257875' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi057-77362-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:28.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:28 smithi107 ceph-mon[33178]: from='client.35945 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi057-77362-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:29.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:29 smithi057 ceph-mon[40588]: pgmap v192: 420 pgs: 32 creating+peering, 32 unknown, 356 active+clean; 592 KiB data, 612 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:32:29.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:29 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:29.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:29 smithi057 ceph-mon[40588]: from='client.35945 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi057-77362-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:29.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:29 smithi057 ceph-mon[40588]: osdmap e150: 8 total, 8 up, 8 in 2024-06-11T13:32:29.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:29 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3808443405' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77714-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:29.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:29 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1516650783' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi057-77351-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:29.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:29 smithi057 ceph-mon[40588]: from='client.35960 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi057-77351-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:29.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:29 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:29.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:29 smithi057 ceph-mon[29805]: pgmap v192: 420 pgs: 32 creating+peering, 32 unknown, 356 active+clean; 592 KiB data, 612 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:32:29.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:29 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:29.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:29 smithi057 ceph-mon[29805]: from='client.35945 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi057-77362-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:29.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:29 smithi057 ceph-mon[29805]: osdmap e150: 8 total, 8 up, 8 in 2024-06-11T13:32:29.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:29 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3808443405' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77714-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:29.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:29 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1516650783' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi057-77351-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:29.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:29 smithi057 ceph-mon[29805]: from='client.35960 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi057-77351-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:29.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:29 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:29.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:29 smithi107 ceph-mon[33178]: pgmap v192: 420 pgs: 32 creating+peering, 32 unknown, 356 active+clean; 592 KiB data, 612 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:32:29.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:29 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:29.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:29 smithi107 ceph-mon[33178]: from='client.35945 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi057-77362-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:29.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:29 smithi107 ceph-mon[33178]: osdmap e150: 8 total, 8 up, 8 in 2024-06-11T13:32:29.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:29 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3808443405' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77714-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:29.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:29 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1516650783' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi057-77351-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:29.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:29 smithi107 ceph-mon[33178]: from='client.35960 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi057-77351-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:29.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:29 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:30.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:30 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3808443405' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77714-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:30.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:30 smithi057 ceph-mon[40588]: from='client.35960 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi057-77351-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:30.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:30 smithi057 ceph-mon[40588]: osdmap e151: 8 total, 8 up, 8 in 2024-06-11T13:32:30.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:30 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1122701135' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi057-77576-36"}]: dispatch 2024-06-11T13:32:30.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:30 smithi057 ceph-mon[40588]: from='client.35975 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi057-77576-36"}]: dispatch 2024-06-11T13:32:30.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:30 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1122701135' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi057-77576-36"}]: dispatch 2024-06-11T13:32:30.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:30 smithi057 ceph-mon[40588]: pgmap v195: 452 pgs: 96 unknown, 356 active+clean; 592 KiB data, 612 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:32:30.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:30 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:30.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:30 smithi057 ceph-mon[40588]: from='client.35975 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi057-77576-36"}]: dispatch 2024-06-11T13:32:30.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:30 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1122701135' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi057-77576-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:32:30.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:30 smithi057 ceph-mon[40588]: from='client.35975 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi057-77576-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:32:30.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:30 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3808443405' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77714-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:30.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:30 smithi057 ceph-mon[29805]: from='client.35960 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi057-77351-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:30.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:30 smithi057 ceph-mon[29805]: osdmap e151: 8 total, 8 up, 8 in 2024-06-11T13:32:30.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:30 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1122701135' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi057-77576-36"}]: dispatch 2024-06-11T13:32:30.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:30 smithi057 ceph-mon[29805]: from='client.35975 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi057-77576-36"}]: dispatch 2024-06-11T13:32:30.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:30 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1122701135' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi057-77576-36"}]: dispatch 2024-06-11T13:32:30.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:30 smithi057 ceph-mon[29805]: pgmap v195: 452 pgs: 96 unknown, 356 active+clean; 592 KiB data, 612 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:32:30.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:30 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:30.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:30 smithi057 ceph-mon[29805]: from='client.35975 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi057-77576-36"}]: dispatch 2024-06-11T13:32:30.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:30 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1122701135' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi057-77576-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:32:30.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:30 smithi057 ceph-mon[29805]: from='client.35975 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi057-77576-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:32:30.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:30 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3808443405' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77714-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:30.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:30 smithi107 ceph-mon[33178]: from='client.35960 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi057-77351-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:30.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:30 smithi107 ceph-mon[33178]: osdmap e151: 8 total, 8 up, 8 in 2024-06-11T13:32:30.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:30 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1122701135' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi057-77576-36"}]: dispatch 2024-06-11T13:32:30.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:30 smithi107 ceph-mon[33178]: from='client.35975 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi057-77576-36"}]: dispatch 2024-06-11T13:32:30.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:30 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1122701135' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi057-77576-36"}]: dispatch 2024-06-11T13:32:30.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:30 smithi107 ceph-mon[33178]: pgmap v195: 452 pgs: 96 unknown, 356 active+clean; 592 KiB data, 612 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:32:30.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:30 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:30.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:30 smithi107 ceph-mon[33178]: from='client.35975 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi057-77576-36"}]: dispatch 2024-06-11T13:32:30.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:30 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1122701135' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi057-77576-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:32:30.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:30 smithi107 ceph-mon[33178]: from='client.35975 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi057-77576-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:32:31.449 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:32:31 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:32:31.449 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:31 smithi107 ceph-mon[33178]: from='client.35975 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi057-77576-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:32:31.449 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:31 smithi107 ceph-mon[33178]: osdmap e152: 8 total, 8 up, 8 in 2024-06-11T13:32:31.449 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:31 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1122701135' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi057-77576-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi057-77576-36"}]: dispatch 2024-06-11T13:32:31.449 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:31 smithi107 ceph-mon[33178]: from='client.35975 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi057-77576-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi057-77576-36"}]: dispatch 2024-06-11T13:32:31.449 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:31 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2349695710' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi057-77362-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:31.450 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:31 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:31.450 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:31 smithi107 ceph-mon[33178]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:32:31.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:31 smithi057 ceph-mon[40588]: from='client.35975 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi057-77576-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:32:31.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:31 smithi057 ceph-mon[40588]: osdmap e152: 8 total, 8 up, 8 in 2024-06-11T13:32:31.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:31 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1122701135' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi057-77576-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi057-77576-36"}]: dispatch 2024-06-11T13:32:31.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:31 smithi057 ceph-mon[40588]: from='client.35975 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi057-77576-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi057-77576-36"}]: dispatch 2024-06-11T13:32:31.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:31 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2349695710' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi057-77362-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:31.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:31 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:31.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:31 smithi057 ceph-mon[40588]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:32:31.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:31 smithi057 ceph-mon[29805]: from='client.35975 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi057-77576-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:32:31.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:31 smithi057 ceph-mon[29805]: osdmap e152: 8 total, 8 up, 8 in 2024-06-11T13:32:31.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:31 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1122701135' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi057-77576-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi057-77576-36"}]: dispatch 2024-06-11T13:32:31.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:31 smithi057 ceph-mon[29805]: from='client.35975 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi057-77576-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi057-77576-36"}]: dispatch 2024-06-11T13:32:31.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:31 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2349695710' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi057-77362-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:31.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:31 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:31.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:31 smithi057 ceph-mon[29805]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:32:32.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:32 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2349695710' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi057-77362-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:32.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:32 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:32:32.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:32 smithi057 ceph-mon[40588]: pgmap v198: 484 pgs: 32 unknown, 32 creating+peering, 420 active+clean; 592 KiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:32:32.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:32 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:32.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:32 smithi057 ceph-mon[40588]: osdmap e153: 8 total, 8 up, 8 in 2024-06-11T13:32:32.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:32 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/4254143800' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi057-77351-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:32.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:32 smithi057 ceph-mon[40588]: from='client.35990 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi057-77351-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:32.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:32 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2349695710' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi057-77362-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:32.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:32 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:32:32.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:32 smithi057 ceph-mon[29805]: pgmap v198: 484 pgs: 32 unknown, 32 creating+peering, 420 active+clean; 592 KiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:32:32.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:32 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:32.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:32 smithi057 ceph-mon[29805]: osdmap e153: 8 total, 8 up, 8 in 2024-06-11T13:32:32.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:32 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/4254143800' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi057-77351-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:32.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:32 smithi057 ceph-mon[29805]: from='client.35990 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi057-77351-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:32.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:32 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2349695710' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi057-77362-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:32.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:32 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:32:32.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:32 smithi107 ceph-mon[33178]: pgmap v198: 484 pgs: 32 unknown, 32 creating+peering, 420 active+clean; 592 KiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:32:32.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:32 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:32.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:32 smithi107 ceph-mon[33178]: osdmap e153: 8 total, 8 up, 8 in 2024-06-11T13:32:32.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:32 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/4254143800' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi057-77351-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:32.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:32 smithi107 ceph-mon[33178]: from='client.35990 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi057-77351-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:33.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:33 smithi057 ceph-mon[29805]: from='client.35975 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi057-77576-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi057-77576-36"}]': finished 2024-06-11T13:32:33.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:33 smithi057 ceph-mon[29805]: from='client.35990 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi057-77351-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:33.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:33 smithi057 ceph-mon[29805]: osdmap e154: 8 total, 8 up, 8 in 2024-06-11T13:32:33.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:33 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3198654073' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi057-77714-16"}]: dispatch 2024-06-11T13:32:33.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:33 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:33.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:33 smithi057 ceph-mon[29805]: from='client.25993 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi057-77714-16"}]: dispatch 2024-06-11T13:32:33.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:33 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3198654073' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi057-77714-16"}]: dispatch 2024-06-11T13:32:33.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:33 smithi057 ceph-mon[29805]: from='client.25993 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi057-77714-16"}]: dispatch 2024-06-11T13:32:33.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:33 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3198654073' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi057-77714-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:32:33.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:33 smithi057 ceph-mon[29805]: from='client.25993 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi057-77714-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:32:33.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:33 smithi057 ceph-mon[40588]: from='client.35975 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi057-77576-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi057-77576-36"}]': finished 2024-06-11T13:32:33.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:33 smithi057 ceph-mon[40588]: from='client.35990 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi057-77351-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:33.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:33 smithi057 ceph-mon[40588]: osdmap e154: 8 total, 8 up, 8 in 2024-06-11T13:32:33.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:33 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3198654073' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi057-77714-16"}]: dispatch 2024-06-11T13:32:33.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:33 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:33.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:33 smithi057 ceph-mon[40588]: from='client.25993 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi057-77714-16"}]: dispatch 2024-06-11T13:32:33.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:33 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3198654073' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi057-77714-16"}]: dispatch 2024-06-11T13:32:33.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:33 smithi057 ceph-mon[40588]: from='client.25993 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi057-77714-16"}]: dispatch 2024-06-11T13:32:33.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:33 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3198654073' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi057-77714-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:32:33.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:33 smithi057 ceph-mon[40588]: from='client.25993 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi057-77714-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:32:33.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:33 smithi107 ceph-mon[33178]: from='client.35975 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi057-77576-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi057-77576-36"}]': finished 2024-06-11T13:32:33.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:33 smithi107 ceph-mon[33178]: from='client.35990 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi057-77351-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:33.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:33 smithi107 ceph-mon[33178]: osdmap e154: 8 total, 8 up, 8 in 2024-06-11T13:32:33.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:33 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3198654073' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi057-77714-16"}]: dispatch 2024-06-11T13:32:33.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:33 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:33.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:33 smithi107 ceph-mon[33178]: from='client.25993 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi057-77714-16"}]: dispatch 2024-06-11T13:32:33.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:33 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3198654073' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi057-77714-16"}]: dispatch 2024-06-11T13:32:33.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:33 smithi107 ceph-mon[33178]: from='client.25993 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi057-77714-16"}]: dispatch 2024-06-11T13:32:33.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:33 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3198654073' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi057-77714-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:32:33.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:33 smithi107 ceph-mon[33178]: from='client.25993 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi057-77714-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:32:34.034 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:32:33 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:32:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:32:35.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:34 smithi057 ceph-mon[29805]: from='client.25993 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi057-77714-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:32:35.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:34 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3198654073' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi057-77714-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi057-77714-16"}]: dispatch 2024-06-11T13:32:35.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:34 smithi057 ceph-mon[29805]: osdmap e155: 8 total, 8 up, 8 in 2024-06-11T13:32:35.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:34 smithi057 ceph-mon[29805]: from='client.25993 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi057-77714-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi057-77714-16"}]: dispatch 2024-06-11T13:32:35.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:34 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2208753621' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi057-77362-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:35.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:34 smithi057 ceph-mon[29805]: pgmap v201: 428 pgs: 40 unknown, 388 active+clean; 592 KiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:32:35.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:34 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:35.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:34 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2208753621' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi057-77362-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:35.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:34 smithi057 ceph-mon[40588]: from='client.25993 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi057-77714-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:32:35.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:34 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3198654073' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi057-77714-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi057-77714-16"}]: dispatch 2024-06-11T13:32:35.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:34 smithi057 ceph-mon[40588]: osdmap e155: 8 total, 8 up, 8 in 2024-06-11T13:32:35.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:34 smithi057 ceph-mon[40588]: from='client.25993 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi057-77714-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi057-77714-16"}]: dispatch 2024-06-11T13:32:35.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:34 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2208753621' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi057-77362-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:35.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:34 smithi057 ceph-mon[40588]: pgmap v201: 428 pgs: 40 unknown, 388 active+clean; 592 KiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:32:35.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:34 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:35.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:34 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2208753621' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi057-77362-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:35.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:34 smithi107 ceph-mon[33178]: from='client.25993 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi057-77714-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:32:35.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:34 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3198654073' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi057-77714-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi057-77714-16"}]: dispatch 2024-06-11T13:32:35.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:34 smithi107 ceph-mon[33178]: osdmap e155: 8 total, 8 up, 8 in 2024-06-11T13:32:35.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:34 smithi107 ceph-mon[33178]: from='client.25993 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi057-77714-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi057-77714-16"}]: dispatch 2024-06-11T13:32:35.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:34 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2208753621' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi057-77362-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:35.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:34 smithi107 ceph-mon[33178]: pgmap v201: 428 pgs: 40 unknown, 388 active+clean; 592 KiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:32:35.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:34 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:35.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:34 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2208753621' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi057-77362-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:36.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:35 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:36.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:35 smithi057 ceph-mon[29805]: osdmap e156: 8 total, 8 up, 8 in 2024-06-11T13:32:36.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:35 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/260154181' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi057-77351-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:36.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:35 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1122701135' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi057-77576-36"}]: dispatch 2024-06-11T13:32:36.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:35 smithi057 ceph-mon[29805]: from='client.35975 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi057-77576-36"}]: dispatch 2024-06-11T13:32:36.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:35 smithi057 ceph-mon[29805]: from='client.25999 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi057-77351-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:36.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:35 smithi057 ceph-mon[29805]: from='client.25993 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi057-77714-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi057-77714-16"}]': finished 2024-06-11T13:32:36.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:35 smithi057 ceph-mon[29805]: from='client.35975 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi057-77576-36"}]': finished 2024-06-11T13:32:36.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:35 smithi057 ceph-mon[29805]: from='client.25999 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi057-77351-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:36.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:35 smithi057 ceph-mon[29805]: osdmap e157: 8 total, 8 up, 8 in 2024-06-11T13:32:36.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:35 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1122701135' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi057-77576-36"}]: dispatch 2024-06-11T13:32:36.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:35 smithi057 ceph-mon[29805]: from='client.35975 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi057-77576-36"}]: dispatch 2024-06-11T13:32:36.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:35 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:36.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:35 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:36.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:35 smithi057 ceph-mon[40588]: osdmap e156: 8 total, 8 up, 8 in 2024-06-11T13:32:36.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:35 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/260154181' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi057-77351-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:36.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:35 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1122701135' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi057-77576-36"}]: dispatch 2024-06-11T13:32:36.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:35 smithi057 ceph-mon[40588]: from='client.35975 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi057-77576-36"}]: dispatch 2024-06-11T13:32:36.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:35 smithi057 ceph-mon[40588]: from='client.25999 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi057-77351-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:36.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:35 smithi057 ceph-mon[40588]: from='client.25993 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi057-77714-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi057-77714-16"}]': finished 2024-06-11T13:32:36.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:35 smithi057 ceph-mon[40588]: from='client.35975 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi057-77576-36"}]': finished 2024-06-11T13:32:36.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:35 smithi057 ceph-mon[40588]: from='client.25999 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi057-77351-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:36.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:35 smithi057 ceph-mon[40588]: osdmap e157: 8 total, 8 up, 8 in 2024-06-11T13:32:36.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:35 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1122701135' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi057-77576-36"}]: dispatch 2024-06-11T13:32:36.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:35 smithi057 ceph-mon[40588]: from='client.35975 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi057-77576-36"}]: dispatch 2024-06-11T13:32:36.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:35 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:36.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:35 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:36.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:35 smithi107 ceph-mon[33178]: osdmap e156: 8 total, 8 up, 8 in 2024-06-11T13:32:36.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:35 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/260154181' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi057-77351-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:36.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:35 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1122701135' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi057-77576-36"}]: dispatch 2024-06-11T13:32:36.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:35 smithi107 ceph-mon[33178]: from='client.35975 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi057-77576-36"}]: dispatch 2024-06-11T13:32:36.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:35 smithi107 ceph-mon[33178]: from='client.25999 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi057-77351-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:36.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:35 smithi107 ceph-mon[33178]: from='client.25993 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi057-77714-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi057-77714-16"}]': finished 2024-06-11T13:32:36.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:35 smithi107 ceph-mon[33178]: from='client.35975 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi057-77576-36"}]': finished 2024-06-11T13:32:36.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:35 smithi107 ceph-mon[33178]: from='client.25999 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi057-77351-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:36.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:35 smithi107 ceph-mon[33178]: osdmap e157: 8 total, 8 up, 8 in 2024-06-11T13:32:36.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:35 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1122701135' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi057-77576-36"}]: dispatch 2024-06-11T13:32:36.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:35 smithi107 ceph-mon[33178]: from='client.35975 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi057-77576-36"}]: dispatch 2024-06-11T13:32:36.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:35 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:36.180 INFO:tasks.workunit.client.0.smithi057.stdout:7918 ms total) 2024-06-11T13:32:36.180 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: 2024-06-11T13:32:36.181 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP 2024-06-11T13:32:36.181 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ RUN ] LibRadosTwoPoolsECPP.CopyFrom 2024-06-11T13:32:36.181 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ OK ] LibRadosTwoPoolsECPP.CopyFrom (118 ms) 2024-06-11T13:32:36.181 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP (118 ms total) 2024-06-11T13:32:36.181 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: 2024-06-11T13:32:36.181 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)0, Checksummer::xxhash32, ceph_le > 2024-06-11T13:32:36.182 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Subset 2024-06-11T13:32:36.182 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Subset (33 ms) 2024-06-11T13:32:36.182 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Chunked 2024-06-11T13:32:36.182 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Chunked (2 ms) 2024-06-11T13:32:36.182 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0 (35 ms total) 2024-06-11T13:32:36.182 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: 2024-06-11T13:32:36.182 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)1, Checksummer::xxhash64, ceph_le > 2024-06-11T13:32:36.182 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Subset 2024-06-11T13:32:36.182 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Subset (31 ms) 2024-06-11T13:32:36.182 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Chunked 2024-06-11T13:32:36.182 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Chunked (10 ms) 2024-06-11T13:32:36.183 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1 (41 ms total) 2024-06-11T13:32:36.183 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: 2024-06-11T13:32:36.183 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)2, Checksummer::crc32c, ceph_le > 2024-06-11T13:32:36.183 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Subset 2024-06-11T13:32:36.183 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Subset (62 ms) 2024-06-11T13:32:36.183 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Chunked 2024-06-11T13:32:36.183 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Chunked (4 ms) 2024-06-11T13:32:36.183 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2 (66 ms total) 2024-06-11T13:32:36.183 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: 2024-06-11T13:32:36.183 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP 2024-06-11T13:32:36.183 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ RUN ] LibRadosMiscECPP.CompareExtentRange 2024-06-11T13:32:36.184 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ OK ] LibRadosMiscECPP.CompareExtentRange (1039 ms) 2024-06-11T13:32:36.184 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP (1039 ms total) 2024-06-11T13:32:36.184 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: 2024-06-11T13:32:36.184 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [----------] Global test environment tear-down 2024-06-11T13:32:36.184 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [==========] 31 tests from 7 test suites ran. (112499 ms total) 2024-06-11T13:32:36.184 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc_pp: [ PASSED ] 31 tests. 2024-06-11T13:32:37.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:36 smithi057 ceph-mon[29805]: pgmap v204: 428 pgs: 40 unknown, 388 active+clean; 592 KiB data, 601 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:32:37.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:36 smithi057 ceph-mon[29805]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:32:37.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:36 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:32:37.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:36 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:32:37.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:36 smithi057 ceph-mon[29805]: from='client.35975 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi057-77576-36"}]': finished 2024-06-11T13:32:37.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:36 smithi057 ceph-mon[29805]: osdmap e158: 8 total, 8 up, 8 in 2024-06-11T13:32:37.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:36 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2139979681' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi057-77362-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:37.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:36 smithi057 ceph-mon[29805]: from='client.26005 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi057-77362-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:37.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:36 smithi057 ceph-mon[40588]: pgmap v204: 428 pgs: 40 unknown, 388 active+clean; 592 KiB data, 601 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:32:37.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:36 smithi057 ceph-mon[40588]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:32:37.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:36 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:32:37.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:36 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:32:37.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:36 smithi057 ceph-mon[40588]: from='client.35975 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi057-77576-36"}]': finished 2024-06-11T13:32:37.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:36 smithi057 ceph-mon[40588]: osdmap e158: 8 total, 8 up, 8 in 2024-06-11T13:32:37.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:36 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2139979681' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi057-77362-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:37.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:36 smithi057 ceph-mon[40588]: from='client.26005 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi057-77362-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:37.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:36 smithi107 ceph-mon[33178]: pgmap v204: 428 pgs: 40 unknown, 388 active+clean; 592 KiB data, 601 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:32:37.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:36 smithi107 ceph-mon[33178]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:32:37.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:36 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:32:37.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:36 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:32:37.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:36 smithi107 ceph-mon[33178]: from='client.35975 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi057-77576-36"}]': finished 2024-06-11T13:32:37.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:36 smithi107 ceph-mon[33178]: osdmap e158: 8 total, 8 up, 8 in 2024-06-11T13:32:37.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:36 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2139979681' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi057-77362-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:37.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:36 smithi107 ceph-mon[33178]: from='client.26005 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi057-77362-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:38.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:37 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:38.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:37 smithi057 ceph-mon[29805]: from='client.26005 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi057-77362-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:38.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:37 smithi057 ceph-mon[29805]: osdmap e159: 8 total, 8 up, 8 in 2024-06-11T13:32:38.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:37 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/289337309' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi057-77351-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:38.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:37 smithi057 ceph-mon[29805]: from='client.26008 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi057-77351-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:38.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:37 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:38.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:37 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:38.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:37 smithi057 ceph-mon[40588]: from='client.26005 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi057-77362-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:38.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:37 smithi057 ceph-mon[40588]: osdmap e159: 8 total, 8 up, 8 in 2024-06-11T13:32:38.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:37 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/289337309' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi057-77351-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:38.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:37 smithi057 ceph-mon[40588]: from='client.26008 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi057-77351-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:38.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:37 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:38.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:37 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:38.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:37 smithi107 ceph-mon[33178]: from='client.26005 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi057-77362-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:38.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:37 smithi107 ceph-mon[33178]: osdmap e159: 8 total, 8 up, 8 in 2024-06-11T13:32:38.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:37 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/289337309' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi057-77351-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:38.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:37 smithi107 ceph-mon[33178]: from='client.26008 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi057-77351-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:38.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:37 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:39.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:38 smithi057 ceph-mon[29805]: pgmap v207: 460 pgs: 8 creating+peering, 64 unknown, 388 active+clean; 592 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:32:39.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:38 smithi057 ceph-mon[29805]: from='client.26008 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi057-77351-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:39.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:38 smithi057 ceph-mon[29805]: osdmap e160: 8 total, 8 up, 8 in 2024-06-11T13:32:39.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:38 smithi057 ceph-mon[40588]: pgmap v207: 460 pgs: 8 creating+peering, 64 unknown, 388 active+clean; 592 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:32:39.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:38 smithi057 ceph-mon[40588]: from='client.26008 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi057-77351-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:39.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:38 smithi057 ceph-mon[40588]: osdmap e160: 8 total, 8 up, 8 in 2024-06-11T13:32:39.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:38 smithi107 ceph-mon[33178]: pgmap v207: 460 pgs: 8 creating+peering, 64 unknown, 388 active+clean; 592 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:32:39.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:38 smithi107 ceph-mon[33178]: from='client.26008 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi057-77351-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:39.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:38 smithi107 ceph-mon[33178]: osdmap e160: 8 total, 8 up, 8 in 2024-06-11T13:32:40.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:39 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:40.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:39 smithi057 ceph-mon[29805]: osdmap e161: 8 total, 8 up, 8 in 2024-06-11T13:32:40.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:39 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2262435847' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi057-77362-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:40.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:39 smithi057 ceph-mon[29805]: from='client.36035 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi057-77362-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:40.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:39 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:40.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:39 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:40.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:39 smithi057 ceph-mon[40588]: osdmap e161: 8 total, 8 up, 8 in 2024-06-11T13:32:40.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:39 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2262435847' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi057-77362-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:40.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:39 smithi057 ceph-mon[40588]: from='client.36035 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi057-77362-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:40.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:39 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:40.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:39 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:40.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:39 smithi107 ceph-mon[33178]: osdmap e161: 8 total, 8 up, 8 in 2024-06-11T13:32:40.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:39 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2262435847' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi057-77362-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:40.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:39 smithi107 ceph-mon[33178]: from='client.36035 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi057-77362-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:40.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:39 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:40.210 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: Running main() from gmock_main.cc 2024-06-11T13:32:40.211 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [==========] Running 21 tests from 5 test suites. 2024-06-11T13:32:40.211 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [----------] Global test environment set-up. 2024-06-11T13:32:40.211 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP 2024-06-11T13:32:40.211 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: seed 77714 2024-06-11T13:32:40.211 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapListPP 2024-06-11T13:32:40.211 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapListPP (8564 ms) 2024-06-11T13:32:40.211 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapRemovePP 2024-06-11T13:32:40.211 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapRemovePP (7873 ms) 2024-06-11T13:32:40.211 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.RollbackPP 2024-06-11T13:32:40.211 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.RollbackPP (6557 ms) 2024-06-11T13:32:40.211 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapGetNamePP 2024-06-11T13:32:40.211 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapGetNamePP (3674 ms) 2024-06-11T13:32:40.211 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapCreateRemovePP 2024-06-11T13:32:40.212 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapCreateRemovePP (8106 ms) 2024-06-11T13:32:40.212 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP (34774 ms total) 2024-06-11T13:32:40.212 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: 2024-06-11T13:32:40.212 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP 2024-06-11T13:32:40.212 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapPP 2024-06-11T13:32:40.212 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapPP (6268 ms) 2024-06-11T13:32:40.212 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.RollbackPP 2024-06-11T13:32:40.212 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.RollbackPP (4848 ms) 2024-06-11T13:32:40.212 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP 2024-06-11T13:32:40.212 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP (8156 ms) 2024-06-11T13:32:40.212 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.Bug11677 2024-06-11T13:32:40.212 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.Bug11677 (5702 ms) 2024-06-11T13:32:40.212 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.OrderSnap 2024-06-11T13:32:40.212 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.OrderSnap (2072 ms) 2024-06-11T13:32:40.213 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-06-11T13:32:40.213 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2976-g3f51ea39/rpm/el9/BUILD/ceph-19.0.0-2976-g3f51ea39/src/test/librados/snapshots_cxx.cc:461: Skipped 2024-06-11T13:32:40.213 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: 2024-06-11T13:32:40.213 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback (0 ms) 2024-06-11T13:32:40.213 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap 2024-06-11T13:32:40.213 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: deleting snap 14 in pool LibRadosSnapshotsSelfManagedPP_smithi057-77714-7 2024-06-11T13:32:40.213 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: waiting for snaps to purge 2024-06-11T13:32:40.213 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap (18386 ms) 2024-06-11T13:32:40.213 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP (45432 ms total) 2024-06-11T13:32:40.213 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: 2024-06-11T13:32:40.213 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode 2024-06-11T13:32:40.213 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected 2024-06-11T13:32:40.213 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected (9 ms) 2024-06-11T13:32:40.213 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance 2024-06-11T13:32:40.213 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance (5526 ms) 2024-06-11T13:32:40.213 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode (5535 ms total) 2024-06-11T13:32:40.214 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: 2024-06-11T13:32:40.214 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP 2024-06-11T13:32:40.214 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapListPP 2024-06-11T13:32:40.214 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapListPP (3025 ms) 2024-06-11T13:32:40.214 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapRemovePP 2024-06-11T13:32:40.688 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:40 smithi107 ceph-mon[33178]: pgmap v210: 428 pgs: 8 creating+peering, 32 unknown, 388 active+clean; 592 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:32:40.688 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:40 smithi107 ceph-mon[33178]: from='client.36035 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi057-77362-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:40.688 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:40 smithi107 ceph-mon[33178]: osdmap e162: 8 total, 8 up, 8 in 2024-06-11T13:32:40.688 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:40 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2824114690' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi057-77351-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:40.688 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:40 smithi107 ceph-mon[33178]: from='client.36041 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi057-77351-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:40.688 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:40 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:40.688 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:40 smithi107 ceph-mon[33178]: from='client.36041 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi057-77351-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:41.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:40 smithi057 ceph-mon[29805]: pgmap v210: 428 pgs: 8 creating+peering, 32 unknown, 388 active+clean; 592 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:32:41.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:40 smithi057 ceph-mon[29805]: from='client.36035 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi057-77362-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:41.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:40 smithi057 ceph-mon[29805]: osdmap e162: 8 total, 8 up, 8 in 2024-06-11T13:32:41.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:40 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2824114690' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi057-77351-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:41.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:40 smithi057 ceph-mon[29805]: from='client.36041 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi057-77351-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:41.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:40 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:41.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:40 smithi057 ceph-mon[29805]: from='client.36041 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi057-77351-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:41.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:40 smithi057 ceph-mon[29805]: osdmap e163: 8 total, 8 up, 8 in 2024-06-11T13:32:41.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:40 smithi057 ceph-mon[40588]: pgmap v210: 428 pgs: 8 creating+peering, 32 unknown, 388 active+clean; 592 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:32:41.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:40 smithi057 ceph-mon[40588]: from='client.36035 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi057-77362-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:41.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:40 smithi057 ceph-mon[40588]: osdmap e162: 8 total, 8 up, 8 in 2024-06-11T13:32:41.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:40 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2824114690' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi057-77351-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:41.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:40 smithi057 ceph-mon[40588]: from='client.36041 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi057-77351-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:41.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:40 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:41.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:40 smithi057 ceph-mon[40588]: from='client.36041 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi057-77351-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:41.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:40 smithi057 ceph-mon[40588]: osdmap e163: 8 total, 8 up, 8 in 2024-06-11T13:32:41.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:40 smithi107 ceph-mon[33178]: osdmap e163: 8 total, 8 up, 8 in 2024-06-11T13:32:41.547 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:32:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:32:42.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:41 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:42.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:41 smithi057 ceph-mon[29805]: osdmap e164: 8 total, 8 up, 8 in 2024-06-11T13:32:42.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:41 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3534563968' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi057-77362-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:42.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:41 smithi057 ceph-mon[29805]: from='client.26026 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi057-77362-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:42.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:41 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:42.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:41 smithi057 ceph-mon[40588]: osdmap e164: 8 total, 8 up, 8 in 2024-06-11T13:32:42.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:41 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3534563968' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi057-77362-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:42.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:41 smithi057 ceph-mon[40588]: from='client.26026 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi057-77362-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:42.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:41 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:42.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:41 smithi107 ceph-mon[33178]: osdmap e164: 8 total, 8 up, 8 in 2024-06-11T13:32:42.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:41 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3534563968' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi057-77362-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:42.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:41 smithi107 ceph-mon[33178]: from='client.26026 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi057-77362-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:43.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:42 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:32:43.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:42 smithi057 ceph-mon[29805]: pgmap v213: 428 pgs: 428 active+clean; 593 KiB data, 606 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-06-11T13:32:43.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:42 smithi057 ceph-mon[29805]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:32:43.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:42 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:43.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:42 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi057-77430-1","var":"pg_num","val":"11"}]: dispatch 2024-06-11T13:32:43.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:42 smithi057 ceph-mon[29805]: from='client.24709 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi057-77430-1","var":"pg_num","val":"11"}]: dispatch 2024-06-11T13:32:43.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:42 smithi057 ceph-mon[29805]: from='client.26026 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi057-77362-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:43.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:42 smithi057 ceph-mon[29805]: from='client.24709 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi057-77430-1","var":"pg_num","val":"11"}]': finished 2024-06-11T13:32:43.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:42 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:43.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:42 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:43.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:42 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi057-77430-1","var":"pgp_num","val":"11"}]: dispatch 2024-06-11T13:32:43.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:42 smithi057 ceph-mon[29805]: osdmap e165: 8 total, 8 up, 8 in 2024-06-11T13:32:43.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:42 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2914826904' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi057-77351-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:43.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:42 smithi057 ceph-mon[29805]: from='client.24709 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi057-77430-1","var":"pgp_num","val":"11"}]: dispatch 2024-06-11T13:32:43.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:42 smithi057 ceph-mon[29805]: from='client.26032 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi057-77351-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:43.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:42 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:32:43.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:42 smithi057 ceph-mon[40588]: pgmap v213: 428 pgs: 428 active+clean; 593 KiB data, 606 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-06-11T13:32:43.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:42 smithi057 ceph-mon[40588]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:32:43.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:42 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:43.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:42 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi057-77430-1","var":"pg_num","val":"11"}]: dispatch 2024-06-11T13:32:43.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:42 smithi057 ceph-mon[40588]: from='client.24709 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi057-77430-1","var":"pg_num","val":"11"}]: dispatch 2024-06-11T13:32:43.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:42 smithi057 ceph-mon[40588]: from='client.26026 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi057-77362-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:43.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:42 smithi057 ceph-mon[40588]: from='client.24709 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi057-77430-1","var":"pg_num","val":"11"}]': finished 2024-06-11T13:32:43.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:42 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:43.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:42 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:43.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:42 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi057-77430-1","var":"pgp_num","val":"11"}]: dispatch 2024-06-11T13:32:43.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:42 smithi057 ceph-mon[40588]: osdmap e165: 8 total, 8 up, 8 in 2024-06-11T13:32:43.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:42 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2914826904' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi057-77351-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:43.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:42 smithi057 ceph-mon[40588]: from='client.24709 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi057-77430-1","var":"pgp_num","val":"11"}]: dispatch 2024-06-11T13:32:43.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:42 smithi057 ceph-mon[40588]: from='client.26032 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi057-77351-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:43.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:42 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:32:43.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:42 smithi107 ceph-mon[33178]: pgmap v213: 428 pgs: 428 active+clean; 593 KiB data, 606 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-06-11T13:32:43.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:42 smithi107 ceph-mon[33178]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:32:43.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:42 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:43.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:42 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi057-77430-1","var":"pg_num","val":"11"}]: dispatch 2024-06-11T13:32:43.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:42 smithi107 ceph-mon[33178]: from='client.24709 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi057-77430-1","var":"pg_num","val":"11"}]: dispatch 2024-06-11T13:32:43.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:42 smithi107 ceph-mon[33178]: from='client.26026 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi057-77362-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:43.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:42 smithi107 ceph-mon[33178]: from='client.24709 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi057-77430-1","var":"pg_num","val":"11"}]': finished 2024-06-11T13:32:43.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:42 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:43.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:42 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:43.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:42 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi057-77430-1","var":"pgp_num","val":"11"}]: dispatch 2024-06-11T13:32:43.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:42 smithi107 ceph-mon[33178]: osdmap e165: 8 total, 8 up, 8 in 2024-06-11T13:32:43.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:42 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2914826904' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi057-77351-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:43.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:42 smithi107 ceph-mon[33178]: from='client.24709 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi057-77430-1","var":"pgp_num","val":"11"}]: dispatch 2024-06-11T13:32:43.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:42 smithi107 ceph-mon[33178]: from='client.26032 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi057-77351-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:44.034 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:32:43 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:32:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:32:45.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:44 smithi057 ceph-mon[29805]: pgmap v216: 460 pgs: 64 unknown, 396 active+clean; 592 KiB data, 606 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 2 op/s 2024-06-11T13:32:45.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:44 smithi057 ceph-mon[29805]: from='client.24709 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi057-77430-1","var":"pgp_num","val":"11"}]': finished 2024-06-11T13:32:45.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:44 smithi057 ceph-mon[29805]: from='client.26032 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi057-77351-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:45.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:44 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:45.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:44 smithi057 ceph-mon[29805]: osdmap e166: 8 total, 8 up, 8 in 2024-06-11T13:32:45.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:44 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:45.039 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:44 smithi057 ceph-mon[40588]: pgmap v216: 460 pgs: 64 unknown, 396 active+clean; 592 KiB data, 606 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 2 op/s 2024-06-11T13:32:45.039 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:44 smithi057 ceph-mon[40588]: from='client.24709 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi057-77430-1","var":"pgp_num","val":"11"}]': finished 2024-06-11T13:32:45.039 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:44 smithi057 ceph-mon[40588]: from='client.26032 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi057-77351-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:45.039 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:44 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:45.039 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:44 smithi057 ceph-mon[40588]: osdmap e166: 8 total, 8 up, 8 in 2024-06-11T13:32:45.039 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:44 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:45.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:44 smithi107 ceph-mon[33178]: pgmap v216: 460 pgs: 64 unknown, 396 active+clean; 592 KiB data, 606 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 2 op/s 2024-06-11T13:32:45.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:44 smithi107 ceph-mon[33178]: from='client.24709 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi057-77430-1","var":"pgp_num","val":"11"}]': finished 2024-06-11T13:32:45.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:44 smithi107 ceph-mon[33178]: from='client.26032 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi057-77351-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:45.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:44 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:45.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:44 smithi107 ceph-mon[33178]: osdmap e166: 8 total, 8 up, 8 in 2024-06-11T13:32:45.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:44 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:46.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:45 smithi057 ceph-mon[29805]: osdmap e167: 8 total, 8 up, 8 in 2024-06-11T13:32:46.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:45 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3198654073' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi057-77714-16"}]: dispatch 2024-06-11T13:32:46.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:45 smithi057 ceph-mon[29805]: from='client.25993 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi057-77714-16"}]: dispatch 2024-06-11T13:32:46.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:45 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2253387509' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi057-77362-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:46.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:45 smithi057 ceph-mon[29805]: from='client.36056 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi057-77362-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:46.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:45 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:46.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:45 smithi057 ceph-mon[29805]: from='client.25993 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi057-77714-16"}]': finished 2024-06-11T13:32:46.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:45 smithi057 ceph-mon[29805]: from='client.36056 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi057-77362-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:46.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:45 smithi057 ceph-mon[29805]: osdmap e168: 8 total, 8 up, 8 in 2024-06-11T13:32:46.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:45 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3198654073' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi057-77714-16"}]: dispatch 2024-06-11T13:32:46.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:45 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/4289544595' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi057-77351-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:46.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:45 smithi057 ceph-mon[29805]: from='client.25993 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi057-77714-16"}]: dispatch 2024-06-11T13:32:46.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:45 smithi057 ceph-mon[29805]: from='client.26044 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi057-77351-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:46.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:45 smithi057 ceph-mon[40588]: osdmap e167: 8 total, 8 up, 8 in 2024-06-11T13:32:46.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:45 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3198654073' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi057-77714-16"}]: dispatch 2024-06-11T13:32:46.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:45 smithi057 ceph-mon[40588]: from='client.25993 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi057-77714-16"}]: dispatch 2024-06-11T13:32:46.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:45 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2253387509' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi057-77362-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:46.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:45 smithi057 ceph-mon[40588]: from='client.36056 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi057-77362-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:46.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:45 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:46.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:45 smithi057 ceph-mon[40588]: from='client.25993 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi057-77714-16"}]': finished 2024-06-11T13:32:46.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:45 smithi057 ceph-mon[40588]: from='client.36056 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi057-77362-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:46.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:45 smithi057 ceph-mon[40588]: osdmap e168: 8 total, 8 up, 8 in 2024-06-11T13:32:46.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:45 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3198654073' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi057-77714-16"}]: dispatch 2024-06-11T13:32:46.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:45 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/4289544595' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi057-77351-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:46.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:45 smithi057 ceph-mon[40588]: from='client.25993 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi057-77714-16"}]: dispatch 2024-06-11T13:32:46.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:45 smithi057 ceph-mon[40588]: from='client.26044 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi057-77351-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:46.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:45 smithi107 ceph-mon[33178]: osdmap e167: 8 total, 8 up, 8 in 2024-06-11T13:32:46.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:45 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3198654073' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi057-77714-16"}]: dispatch 2024-06-11T13:32:46.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:45 smithi107 ceph-mon[33178]: from='client.25993 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi057-77714-16"}]: dispatch 2024-06-11T13:32:46.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:45 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2253387509' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi057-77362-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:46.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:45 smithi107 ceph-mon[33178]: from='client.36056 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi057-77362-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:46.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:45 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:46.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:45 smithi107 ceph-mon[33178]: from='client.25993 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi057-77714-16"}]': finished 2024-06-11T13:32:46.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:45 smithi107 ceph-mon[33178]: from='client.36056 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi057-77362-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:46.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:45 smithi107 ceph-mon[33178]: osdmap e168: 8 total, 8 up, 8 in 2024-06-11T13:32:46.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:45 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3198654073' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi057-77714-16"}]: dispatch 2024-06-11T13:32:46.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:45 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/4289544595' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi057-77351-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:46.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:45 smithi107 ceph-mon[33178]: from='client.25993 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi057-77714-16"}]: dispatch 2024-06-11T13:32:46.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:45 smithi107 ceph-mon[33178]: from='client.26044 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi057-77351-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:46.706 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [ OK ] LibRadosSnaps snapshots: Running main() from gmock_main.cc 2024-06-11T13:32:46.706 INFO:tasks.workunit.client.0.smithi057.stdout: snapshots: [==========] Running 11 tests from 2 test suites. 2024-06-11T13:32:46.706 INFO:tasks.workunit.client.0.smithi057.stdout: snapshots: [----------] Global test environment set-up. 2024-06-11T13:32:46.707 INFO:tasks.workunit.client.0.smithi057.stdout: snapshots: [----------] 5 tests from NeoRadosSnapshots 2024-06-11T13:32:46.707 INFO:tasks.workunit.client.0.smithi057.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapList 2024-06-11T13:32:46.707 INFO:tasks.workunit.client.0.smithi057.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapList (27954 ms) 2024-06-11T13:32:46.707 INFO:tasks.workunit.client.0.smithi057.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapRemove 2024-06-11T13:32:46.707 INFO:tasks.workunit.client.0.smithi057.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapRemove (12607 ms) 2024-06-11T13:32:46.707 INFO:tasks.workunit.client.0.smithi057.stdout: snapshots: [ RUN ] NeoRadosSnapshots.Rollback 2024-06-11T13:32:46.707 INFO:tasks.workunit.client.0.smithi057.stdout: snapshots: [ OK ] NeoRadosSnapshots.Rollback (9450 ms) 2024-06-11T13:32:46.707 INFO:tasks.workunit.client.0.smithi057.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapGetName 2024-06-11T13:32:46.707 INFO:tasks.workunit.client.0.smithi057.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapGetName (7540 ms) 2024-06-11T13:32:46.707 INFO:tasks.workunit.client.0.smithi057.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapCreateRemove 2024-06-11T13:32:46.707 INFO:tasks.workunit.client.0.smithi057.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapCreateRemove (11520 ms) 2024-06-11T13:32:46.707 INFO:tasks.workunit.client.0.smithi057.stdout: snapshots: [----------] 5 tests from NeoRadosSnapshots (69071 ms total) 2024-06-11T13:32:46.708 INFO:tasks.workunit.client.0.smithi057.stdout: snapshots: 2024-06-11T13:32:46.708 INFO:tasks.workunit.client.0.smithi057.stdout: snapshots: [----------] 6 tests from NeoRadosSelfManagedSnaps 2024-06-11T13:32:46.708 INFO:tasks.workunit.client.0.smithi057.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Snap 2024-06-11T13:32:46.708 INFO:tasks.workunit.client.0.smithi057.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Snap (6720 ms) 2024-06-11T13:32:46.708 INFO:tasks.workunit.client.0.smithi057.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Rollback 2024-06-11T13:32:46.708 INFO:tasks.workunit.client.0.smithi057.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Rollback (8740 ms) 2024-06-11T13:32:46.708 INFO:tasks.workunit.client.0.smithi057.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.SnapOverlap 2024-06-11T13:32:46.708 INFO:tasks.workunit.client.0.smithi057.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.SnapOverlap (7997 ms) 2024-06-11T13:32:46.708 INFO:tasks.workunit.client.0.smithi057.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Bug11677 2024-06-11T13:32:46.708 INFO:tasks.workunit.client.0.smithi057.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Bug11677 (6026 ms) 2024-06-11T13:32:46.708 INFO:tasks.workunit.client.0.smithi057.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.OrderSnap 2024-06-11T13:32:46.708 INFO:tasks.workunit.client.0.smithi057.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.OrderSnap (4019 ms) 2024-06-11T13:32:46.709 INFO:tasks.workunit.client.0.smithi057.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.ReusePurgedSnap 2024-06-11T13:32:46.709 INFO:tasks.workunit.client.0.smithi057.stdout: snapshots: Deleting snap 3 in pool ReusePurgedSnapsmithi057-78727-11. 2024-06-11T13:32:46.709 INFO:tasks.workunit.client.0.smithi057.stdout: snapshots: Waiting for snaps to purge. 2024-06-11T13:32:46.709 INFO:tasks.workunit.client.0.smithi057.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.ReusePurgedSnap (20028 ms) 2024-06-11T13:32:46.709 INFO:tasks.workunit.client.0.smithi057.stdout: snapshots: [----------] 6 tests from NeoRadosSelfManagedSnaps (53530 ms total) 2024-06-11T13:32:46.709 INFO:tasks.workunit.client.0.smithi057.stdout: snapshots: 2024-06-11T13:32:46.709 INFO:tasks.workunit.client.0.smithi057.stdout: snapshots: [----------] Global test environment tear-down 2024-06-11T13:32:46.709 INFO:tasks.workunit.client.0.smithi057.stdout: snapshots: [==========] 11 tests from 2 test suites ran. (122601 ms total) 2024-06-11T13:32:46.709 INFO:tasks.workunit.client.0.smithi057.stdout: snapshots: [ PASSED ] 11 tests. 2024-06-11T13:32:47.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:46 smithi057 ceph-mon[29805]: pgmap v219: 420 pgs: 32 unknown, 388 active+clean; 592 KiB data, 606 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:32:47.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:46 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:47.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:46 smithi057 ceph-mon[29805]: from='client.25993 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi057-77714-16"}]': finished 2024-06-11T13:32:47.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:46 smithi057 ceph-mon[29805]: from='client.26044 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi057-77351-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:47.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:46 smithi057 ceph-mon[29805]: osdmap e169: 8 total, 8 up, 8 in 2024-06-11T13:32:47.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:46 smithi057 ceph-mon[40588]: pgmap v219: 420 pgs: 32 unknown, 388 active+clean; 592 KiB data, 606 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:32:47.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:46 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:47.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:46 smithi057 ceph-mon[40588]: from='client.25993 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi057-77714-16"}]': finished 2024-06-11T13:32:47.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:46 smithi057 ceph-mon[40588]: from='client.26044 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi057-77351-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:47.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:46 smithi057 ceph-mon[40588]: osdmap e169: 8 total, 8 up, 8 in 2024-06-11T13:32:47.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:46 smithi107 ceph-mon[33178]: pgmap v219: 420 pgs: 32 unknown, 388 active+clean; 592 KiB data, 606 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:32:47.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:46 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:47.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:46 smithi107 ceph-mon[33178]: from='client.25993 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi057-77714-16"}]': finished 2024-06-11T13:32:47.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:46 smithi107 ceph-mon[33178]: from='client.26044 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi057-77351-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:47.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:46 smithi107 ceph-mon[33178]: osdmap e169: 8 total, 8 up, 8 in 2024-06-11T13:32:47.704 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: Running main() from gmock_main.cc 2024-06-11T13:32:47.705 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [==========] Running 42 tests from 2 test suites. 2024-06-11T13:32:47.705 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [----------] Global test environment set-up. 2024-06-11T13:32:47.705 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [----------] 26 tests from LibRadosAio 2024-06-11T13:32:47.705 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ RUN ] LibRadosAio.TooBig 2024-06-11T13:32:47.705 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ OK ] LibRadosAio.TooBig (18154 ms) 2024-06-11T13:32:47.705 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ RUN ] LibRadosAio.SimpleWrite 2024-06-11T13:32:47.705 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ OK ] LibRadosAio.SimpleWrite (7584 ms) 2024-06-11T13:32:47.705 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ RUN ] LibRadosAio.WaitForSafe 2024-06-11T13:32:47.705 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ OK ] LibRadosAio.WaitForSafe (9272 ms) 2024-06-11T13:32:47.705 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip 2024-06-11T13:32:47.705 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip (6103 ms) 2024-06-11T13:32:47.706 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip2 2024-06-11T13:32:47.706 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip2 (6171 ms) 2024-06-11T13:32:47.706 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip3 2024-06-11T13:32:47.706 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip3 (7059 ms) 2024-06-11T13:32:47.706 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripAppend 2024-06-11T13:32:47.706 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ OK ] LibRadosAio.RoundTripAppend (5139 ms) 2024-06-11T13:32:47.706 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ RUN ] LibRadosAio.RemoveTest 2024-06-11T13:32:47.706 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ OK ] LibRadosAio.RemoveTest (4831 ms) 2024-06-11T13:32:47.706 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ RUN ] LibRadosAio.XattrsRoundTrip 2024-06-11T13:32:47.706 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ OK ] LibRadosAio.XattrsRoundTrip (3890 ms) 2024-06-11T13:32:47.706 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ RUN ] LibRadosAio.RmXattr 2024-06-11T13:32:47.706 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ OK ] LibRadosAio.RmXattr (4367 ms) 2024-06-11T13:32:47.706 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ RUN ] LibRadosAio.XattrIter 2024-06-11T13:32:47.707 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ OK ] LibRadosAio.XattrIter (6074 ms) 2024-06-11T13:32:47.707 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ RUN ] LibRadosAio.IsComplete 2024-06-11T13:32:47.707 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ OK ] LibRadosAio.IsComplete (3414 ms) 2024-06-11T13:32:47.707 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ RUN ] LibRadosAio.IsSafe 2024-06-11T13:32:47.707 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ OK ] LibRadosAio.IsSafe (3029 ms) 2024-06-11T13:32:47.707 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ RUN ] LibRadosAio.ReturnValue 2024-06-11T13:32:47.707 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ OK ] LibRadosAio.ReturnValue (2974 ms) 2024-06-11T13:32:47.707 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ RUN ] LibRadosAio.Flush 2024-06-11T13:32:47.707 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ OK ] LibRadosAio.Flush (3492 ms) 2024-06-11T13:32:47.707 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ RUN ] LibRadosAio.FlushAsync 2024-06-11T13:32:47.707 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ OK ] LibRadosAio.FlushAsync (2532 ms) 2024-06-11T13:32:47.707 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteFull 2024-06-11T13:32:47.707 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteFull (3026 ms) 2024-06-11T13:32:47.707 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteSame 2024-06-11T13:32:47.708 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteSame (2998 ms) 2024-06-11T13:32:47.708 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStat 2024-06-11T13:32:47.708 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ OK ] LibRadosAio.SimpleStat (3002 ms) 2024-06-11T13:32:47.708 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ RUN ] LibRadosAio.OperateMtime 2024-06-11T13:32:47.708 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ OK ] LibRadosAio.OperateMtime (3446 ms) 2024-06-11T13:32:47.708 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ RUN ] LibRadosAio.Operate2Mtime 2024-06-11T13:32:47.708 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ OK ] LibRadosAio.Operate2Mtime (3052 ms) 2024-06-11T13:32:47.708 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStatNS 2024-06-11T13:32:47.708 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ OK ] LibRadosAio.SimpleStatNS (3008 ms) 2024-06-11T13:32:47.708 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ RUN ] LibRadosAio.StatRemove 2024-06-11T13:32:47.708 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ OK ] LibRadosAio.StatRemove (3019 ms) 2024-06-11T13:32:47.708 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ RUN ] LibRadosAio.ExecuteClass 2024-06-11T13:32:47.708 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ OK ] LibRadosAio.ExecuteClass (2478 ms) 2024-06-11T13:32:47.708 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ RUN ] LibRadosAio.MultiWrite 2024-06-11T13:32:47.708 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ OK ] LibRadosAio.MultiWrite (3007 ms) 2024-06-11T13:32:47.709 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ RUN ] LibRadosAio.AioUnlock 2024-06-11T13:32:47.709 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ OK ] LibRadosAio.AioUnlock (3028 ms) 2024-06-11T13:32:47.709 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [----------] 26 tests from LibRadosAio (124150 ms total) 2024-06-11T13:32:47.709 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: 2024-06-11T13:32:47.709 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [----------] 16 tests from LibRadosAioEC 2024-06-11T13:32:48.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:47 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3514137530' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21"}]: dispatch 2024-06-11T13:32:48.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:47 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:48.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:47 smithi057 ceph-mon[40588]: from='client.36071 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21"}]: dispatch 2024-06-11T13:32:48.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:47 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3514137530' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21"}]: dispatch 2024-06-11T13:32:48.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:47 smithi057 ceph-mon[40588]: from='client.36071 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21"}]: dispatch 2024-06-11T13:32:48.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:47 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3514137530' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:32:48.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:47 smithi057 ceph-mon[40588]: from='client.36071 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:32:48.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:47 smithi057 ceph-mon[40588]: from='client.36071 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:32:48.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:47 smithi057 ceph-mon[40588]: osdmap e170: 8 total, 8 up, 8 in 2024-06-11T13:32:48.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:47 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2978761174' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi057-77362-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:48.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:47 smithi057 ceph-mon[40588]: from='client.26053 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi057-77362-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:48.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:47 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3514137530' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21"}]: dispatch 2024-06-11T13:32:48.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:47 smithi057 ceph-mon[40588]: from='client.36071 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21"}]: dispatch 2024-06-11T13:32:48.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:47 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3514137530' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21"}]: dispatch 2024-06-11T13:32:48.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:47 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:48.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:47 smithi057 ceph-mon[29805]: from='client.36071 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21"}]: dispatch 2024-06-11T13:32:48.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:47 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3514137530' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21"}]: dispatch 2024-06-11T13:32:48.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:47 smithi057 ceph-mon[29805]: from='client.36071 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21"}]: dispatch 2024-06-11T13:32:48.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:47 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3514137530' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:32:48.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:47 smithi057 ceph-mon[29805]: from='client.36071 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:32:48.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:47 smithi057 ceph-mon[29805]: from='client.36071 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:32:48.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:47 smithi057 ceph-mon[29805]: osdmap e170: 8 total, 8 up, 8 in 2024-06-11T13:32:48.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:47 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2978761174' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi057-77362-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:48.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:47 smithi057 ceph-mon[29805]: from='client.26053 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi057-77362-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:48.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:47 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3514137530' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21"}]: dispatch 2024-06-11T13:32:48.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:47 smithi057 ceph-mon[29805]: from='client.36071 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21"}]: dispatch 2024-06-11T13:32:48.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:47 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3514137530' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21"}]: dispatch 2024-06-11T13:32:48.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:47 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:48.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:47 smithi107 ceph-mon[33178]: from='client.36071 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21"}]: dispatch 2024-06-11T13:32:48.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:47 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3514137530' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21"}]: dispatch 2024-06-11T13:32:48.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:47 smithi107 ceph-mon[33178]: from='client.36071 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21"}]: dispatch 2024-06-11T13:32:48.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:47 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3514137530' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:32:48.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:47 smithi107 ceph-mon[33178]: from='client.36071 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:32:48.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:47 smithi107 ceph-mon[33178]: from='client.36071 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:32:48.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:47 smithi107 ceph-mon[33178]: osdmap e170: 8 total, 8 up, 8 in 2024-06-11T13:32:48.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:47 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2978761174' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi057-77362-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:48.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:47 smithi107 ceph-mon[33178]: from='client.26053 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi057-77362-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:48.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:47 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3514137530' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21"}]: dispatch 2024-06-11T13:32:48.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:47 smithi107 ceph-mon[33178]: from='client.36071 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21"}]: dispatch 2024-06-11T13:32:49.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:49 smithi057 ceph-mon[29805]: pgmap v222: 388 pgs: 32 creating+peering, 356 active+clean; 592 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 2.7 KiB/s wr, 7 op/s 2024-06-11T13:32:49.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:49 smithi057 ceph-mon[29805]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:32:49.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:49 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2770965589' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi057-77351-27"}]: dispatch 2024-06-11T13:32:49.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:49 smithi057 ceph-mon[29805]: from='client.26062 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi057-77351-27"}]: dispatch 2024-06-11T13:32:49.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:49 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2770965589' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi057-77351-27"}]: dispatch 2024-06-11T13:32:49.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:49 smithi057 ceph-mon[29805]: from='client.26062 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi057-77351-27"}]: dispatch 2024-06-11T13:32:49.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:49 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2770965589' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi057-77351-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:32:49.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:49 smithi057 ceph-mon[29805]: from='client.26062 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi057-77351-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:32:49.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:49 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:49.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:49 smithi057 ceph-mon[29805]: from='client.26053 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi057-77362-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:49.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:49 smithi057 ceph-mon[40588]: pgmap v222: 388 pgs: 32 creating+peering, 356 active+clean; 592 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 2.7 KiB/s wr, 7 op/s 2024-06-11T13:32:49.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:49 smithi057 ceph-mon[40588]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:32:49.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:49 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2770965589' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi057-77351-27"}]: dispatch 2024-06-11T13:32:49.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:49 smithi057 ceph-mon[40588]: from='client.26062 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi057-77351-27"}]: dispatch 2024-06-11T13:32:49.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:49 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2770965589' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi057-77351-27"}]: dispatch 2024-06-11T13:32:49.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:49 smithi057 ceph-mon[40588]: from='client.26062 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi057-77351-27"}]: dispatch 2024-06-11T13:32:49.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:49 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2770965589' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi057-77351-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:32:49.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:49 smithi057 ceph-mon[40588]: from='client.26062 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi057-77351-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:32:49.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:49 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:49.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:49 smithi057 ceph-mon[40588]: from='client.26053 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi057-77362-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:49.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:49 smithi107 ceph-mon[33178]: pgmap v222: 388 pgs: 32 creating+peering, 356 active+clean; 592 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 2.7 KiB/s wr, 7 op/s 2024-06-11T13:32:49.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:49 smithi107 ceph-mon[33178]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:32:49.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:49 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2770965589' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi057-77351-27"}]: dispatch 2024-06-11T13:32:49.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:49 smithi107 ceph-mon[33178]: from='client.26062 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi057-77351-27"}]: dispatch 2024-06-11T13:32:49.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:49 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2770965589' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi057-77351-27"}]: dispatch 2024-06-11T13:32:49.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:49 smithi107 ceph-mon[33178]: from='client.26062 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi057-77351-27"}]: dispatch 2024-06-11T13:32:49.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:49 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2770965589' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi057-77351-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:32:49.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:49 smithi107 ceph-mon[33178]: from='client.26062 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi057-77351-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:32:49.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:49 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:49.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:49 smithi107 ceph-mon[33178]: from='client.26053 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi057-77362-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:51.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:50 smithi057 ceph-mon[29805]: from='client.26062 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi057-77351-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:32:51.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:50 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2770965589' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi057-77351-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi057-77351-27"}]: dispatch 2024-06-11T13:32:51.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:50 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:51.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:50 smithi057 ceph-mon[29805]: osdmap e171: 8 total, 8 up, 8 in 2024-06-11T13:32:51.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:50 smithi057 ceph-mon[29805]: from='client.26062 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi057-77351-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi057-77351-27"}]: dispatch 2024-06-11T13:32:51.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:50 smithi057 ceph-mon[29805]: pgmap v225: 388 pgs: 32 unknown, 356 active+clean; 592 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-06-11T13:32:51.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:50 smithi057 ceph-mon[29805]: from='client.36071 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21"}]': finished 2024-06-11T13:32:51.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:50 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:51.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:50 smithi057 ceph-mon[29805]: osdmap e172: 8 total, 8 up, 8 in 2024-06-11T13:32:51.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:50 smithi057 ceph-mon[40588]: from='client.26062 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi057-77351-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:32:51.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:50 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2770965589' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi057-77351-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi057-77351-27"}]: dispatch 2024-06-11T13:32:51.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:50 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:51.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:50 smithi057 ceph-mon[40588]: osdmap e171: 8 total, 8 up, 8 in 2024-06-11T13:32:51.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:50 smithi057 ceph-mon[40588]: from='client.26062 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi057-77351-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi057-77351-27"}]: dispatch 2024-06-11T13:32:51.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:50 smithi057 ceph-mon[40588]: pgmap v225: 388 pgs: 32 unknown, 356 active+clean; 592 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-06-11T13:32:51.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:50 smithi057 ceph-mon[40588]: from='client.36071 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21"}]': finished 2024-06-11T13:32:51.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:50 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:51.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:50 smithi057 ceph-mon[40588]: osdmap e172: 8 total, 8 up, 8 in 2024-06-11T13:32:51.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:50 smithi107 ceph-mon[33178]: from='client.26062 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi057-77351-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:32:51.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:50 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2770965589' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi057-77351-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi057-77351-27"}]: dispatch 2024-06-11T13:32:51.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:50 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:51.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:50 smithi107 ceph-mon[33178]: osdmap e171: 8 total, 8 up, 8 in 2024-06-11T13:32:51.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:50 smithi107 ceph-mon[33178]: from='client.26062 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi057-77351-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi057-77351-27"}]: dispatch 2024-06-11T13:32:51.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:50 smithi107 ceph-mon[33178]: pgmap v225: 388 pgs: 32 unknown, 356 active+clean; 592 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-06-11T13:32:51.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:50 smithi107 ceph-mon[33178]: from='client.36071 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21"}]': finished 2024-06-11T13:32:51.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:50 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:51.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:50 smithi107 ceph-mon[33178]: osdmap e172: 8 total, 8 up, 8 in 2024-06-11T13:32:51.547 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:32:51 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:32:52.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:51 smithi057 ceph-mon[40588]: from='client.26062 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi057-77351-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi057-77351-27"}]': finished 2024-06-11T13:32:52.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:51 smithi057 ceph-mon[40588]: osdmap e173: 8 total, 8 up, 8 in 2024-06-11T13:32:52.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:51 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1108522702' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi057-77362-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:52.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:51 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:52.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:51 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:32:52.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:51 smithi057 ceph-mon[29805]: from='client.26062 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi057-77351-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi057-77351-27"}]': finished 2024-06-11T13:32:52.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:51 smithi057 ceph-mon[29805]: osdmap e173: 8 total, 8 up, 8 in 2024-06-11T13:32:52.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:51 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1108522702' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi057-77362-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:52.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:51 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:52.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:51 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:32:52.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:51 smithi107 ceph-mon[33178]: from='client.26062 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi057-77351-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi057-77351-27"}]': finished 2024-06-11T13:32:52.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:51 smithi107 ceph-mon[33178]: osdmap e173: 8 total, 8 up, 8 in 2024-06-11T13:32:52.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:51 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1108522702' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi057-77362-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:52.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:51 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:52.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:51 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:32:53.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:52 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:32:53.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:52 smithi057 ceph-mon[40588]: pgmap v228: 404 pgs: 40 creating+peering, 364 active+clean; 597 KiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-06-11T13:32:53.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:52 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1108522702' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi057-77362-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:53.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:52 smithi057 ceph-mon[40588]: osdmap e174: 8 total, 8 up, 8 in 2024-06-11T13:32:53.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:52 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:53.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:52 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:32:53.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:52 smithi057 ceph-mon[29805]: pgmap v228: 404 pgs: 40 creating+peering, 364 active+clean; 597 KiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-06-11T13:32:53.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:52 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1108522702' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi057-77362-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:53.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:52 smithi057 ceph-mon[29805]: osdmap e174: 8 total, 8 up, 8 in 2024-06-11T13:32:53.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:52 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:53.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:52 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:32:53.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:52 smithi107 ceph-mon[33178]: pgmap v228: 404 pgs: 40 creating+peering, 364 active+clean; 597 KiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-06-11T13:32:53.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:52 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1108522702' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi057-77362-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:53.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:52 smithi107 ceph-mon[33178]: osdmap e174: 8 total, 8 up, 8 in 2024-06-11T13:32:53.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:52 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:54.034 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:32:53 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:32:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:32:54.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:53 smithi057 ceph-mon[29805]: osdmap e175: 8 total, 8 up, 8 in 2024-06-11T13:32:54.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:53 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2770965589' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi057-77351-27"}]: dispatch 2024-06-11T13:32:54.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:53 smithi057 ceph-mon[29805]: from='client.26062 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi057-77351-27"}]: dispatch 2024-06-11T13:32:54.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:53 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:54.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:53 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:32:54.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:53 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:32:54.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:53 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi057-77430-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-06-11T13:32:54.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:53 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi057-77430-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-06-11T13:32:54.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:53 smithi057 ceph-mon[40588]: osdmap e175: 8 total, 8 up, 8 in 2024-06-11T13:32:54.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:53 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2770965589' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi057-77351-27"}]: dispatch 2024-06-11T13:32:54.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:53 smithi057 ceph-mon[40588]: from='client.26062 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi057-77351-27"}]: dispatch 2024-06-11T13:32:54.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:53 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:54.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:53 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:32:54.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:53 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:32:54.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:53 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi057-77430-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-06-11T13:32:54.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:53 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi057-77430-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-06-11T13:32:54.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:53 smithi107 ceph-mon[33178]: osdmap e175: 8 total, 8 up, 8 in 2024-06-11T13:32:54.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:53 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2770965589' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi057-77351-27"}]: dispatch 2024-06-11T13:32:54.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:53 smithi107 ceph-mon[33178]: from='client.26062 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi057-77351-27"}]: dispatch 2024-06-11T13:32:54.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:53 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:54.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:53 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:32:54.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:53 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:32:54.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:53 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi057-77430-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-06-11T13:32:54.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:53 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi057-77430-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-06-11T13:32:55.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:54 smithi057 ceph-mon[29805]: pgmap v231: 364 pgs: 364 active+clean; 597 KiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-06-11T13:32:55.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:54 smithi057 ceph-mon[29805]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:32:55.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:54 smithi057 ceph-mon[29805]: from='client.26062 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi057-77351-27"}]': finished 2024-06-11T13:32:55.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:54 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:32:55.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:54 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi057-77430-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-06-11T13:32:55.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:54 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2770965589' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi057-77351-27"}]: dispatch 2024-06-11T13:32:55.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:54 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-13"}]: dispatch 2024-06-11T13:32:55.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:54 smithi057 ceph-mon[29805]: osdmap e176: 8 total, 8 up, 8 in 2024-06-11T13:32:55.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:54 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2495688684' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi057-77362-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:55.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:54 smithi057 ceph-mon[29805]: from='client.36083 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi057-77362-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:55.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:54 smithi057 ceph-mon[29805]: from='client.26062 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi057-77351-27"}]: dispatch 2024-06-11T13:32:55.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:54 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-13"}]: dispatch 2024-06-11T13:32:55.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:54 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:55.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:54 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi057-77430-1","var":"pg_num","val":"11"}]: dispatch 2024-06-11T13:32:55.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:54 smithi057 ceph-mon[29805]: from='client.24709 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi057-77430-1","var":"pg_num","val":"11"}]: dispatch 2024-06-11T13:32:55.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:54 smithi057 ceph-mon[40588]: pgmap v231: 364 pgs: 364 active+clean; 597 KiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-06-11T13:32:55.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:54 smithi057 ceph-mon[40588]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:32:55.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:54 smithi057 ceph-mon[40588]: from='client.26062 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi057-77351-27"}]': finished 2024-06-11T13:32:55.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:54 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:32:55.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:54 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi057-77430-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-06-11T13:32:55.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:54 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2770965589' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi057-77351-27"}]: dispatch 2024-06-11T13:32:55.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:54 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-13"}]: dispatch 2024-06-11T13:32:55.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:54 smithi057 ceph-mon[40588]: osdmap e176: 8 total, 8 up, 8 in 2024-06-11T13:32:55.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:54 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2495688684' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi057-77362-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:55.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:54 smithi057 ceph-mon[40588]: from='client.36083 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi057-77362-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:55.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:54 smithi057 ceph-mon[40588]: from='client.26062 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi057-77351-27"}]: dispatch 2024-06-11T13:32:55.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:54 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-13"}]: dispatch 2024-06-11T13:32:55.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:54 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:55.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:54 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi057-77430-1","var":"pg_num","val":"11"}]: dispatch 2024-06-11T13:32:55.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:54 smithi057 ceph-mon[40588]: from='client.24709 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi057-77430-1","var":"pg_num","val":"11"}]: dispatch 2024-06-11T13:32:55.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:54 smithi107 ceph-mon[33178]: pgmap v231: 364 pgs: 364 active+clean; 597 KiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-06-11T13:32:55.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:54 smithi107 ceph-mon[33178]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:32:55.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:54 smithi107 ceph-mon[33178]: from='client.26062 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi057-77351-27"}]': finished 2024-06-11T13:32:55.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:54 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:32:55.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:54 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi057-77430-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-06-11T13:32:55.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:54 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2770965589' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi057-77351-27"}]: dispatch 2024-06-11T13:32:55.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:54 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-13"}]: dispatch 2024-06-11T13:32:55.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:54 smithi107 ceph-mon[33178]: osdmap e176: 8 total, 8 up, 8 in 2024-06-11T13:32:55.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:54 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2495688684' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi057-77362-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:55.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:54 smithi107 ceph-mon[33178]: from='client.36083 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi057-77362-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:55.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:54 smithi107 ceph-mon[33178]: from='client.26062 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi057-77351-27"}]: dispatch 2024-06-11T13:32:55.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:54 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-13"}]: dispatch 2024-06-11T13:32:55.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:54 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:55.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:54 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi057-77430-1","var":"pg_num","val":"11"}]: dispatch 2024-06-11T13:32:55.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:54 smithi107 ceph-mon[33178]: from='client.24709 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi057-77430-1","var":"pg_num","val":"11"}]: dispatch 2024-06-11T13:32:57.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:56 smithi057 ceph-mon[40588]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:32:57.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:56 smithi057 ceph-mon[40588]: from='client.36083 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi057-77362-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:57.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:56 smithi057 ceph-mon[40588]: from='client.26062 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi057-77351-27"}]': finished 2024-06-11T13:32:57.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:56 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-13"}]': finished 2024-06-11T13:32:57.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:56 smithi057 ceph-mon[40588]: from='client.24709 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi057-77430-1","var":"pg_num","val":"11"}]': finished 2024-06-11T13:32:57.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:56 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:57.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:56 smithi057 ceph-mon[40588]: osdmap e177: 8 total, 8 up, 8 in 2024-06-11T13:32:57.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:56 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:57.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:56 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi057-77430-1","var":"pgp_num","val":"11"}]: dispatch 2024-06-11T13:32:57.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:56 smithi057 ceph-mon[40588]: from='client.24709 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi057-77430-1","var":"pgp_num","val":"11"}]: dispatch 2024-06-11T13:32:57.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:56 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1729312227' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi057-77351-28"}]: dispatch 2024-06-11T13:32:57.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:56 smithi057 ceph-mon[40588]: from='client.26071 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi057-77351-28"}]: dispatch 2024-06-11T13:32:57.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:56 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1729312227' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi057-77351-28"}]: dispatch 2024-06-11T13:32:57.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:56 smithi057 ceph-mon[40588]: from='client.26071 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi057-77351-28"}]: dispatch 2024-06-11T13:32:57.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:56 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1729312227' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi057-77351-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:32:57.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:56 smithi057 ceph-mon[40588]: from='client.26071 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi057-77351-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:32:57.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:56 smithi057 ceph-mon[40588]: from='client.24709 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi057-77430-1","var":"pgp_num","val":"11"}]': finished 2024-06-11T13:32:57.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:56 smithi057 ceph-mon[40588]: from='client.26071 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi057-77351-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:32:57.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:56 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:57.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:56 smithi057 ceph-mon[40588]: osdmap e178: 8 total, 8 up, 8 in 2024-06-11T13:32:57.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:56 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1729312227' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi057-77351-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi057-77351-28"}]: dispatch 2024-06-11T13:32:57.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:56 smithi057 ceph-mon[40588]: from='client.26071 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi057-77351-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi057-77351-28"}]: dispatch 2024-06-11T13:32:57.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:56 smithi057 ceph-mon[29805]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:32:57.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:56 smithi057 ceph-mon[29805]: from='client.36083 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi057-77362-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:57.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:56 smithi057 ceph-mon[29805]: from='client.26062 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi057-77351-27"}]': finished 2024-06-11T13:32:57.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:56 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-13"}]': finished 2024-06-11T13:32:57.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:56 smithi057 ceph-mon[29805]: from='client.24709 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi057-77430-1","var":"pg_num","val":"11"}]': finished 2024-06-11T13:32:57.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:56 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:57.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:56 smithi057 ceph-mon[29805]: osdmap e177: 8 total, 8 up, 8 in 2024-06-11T13:32:57.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:56 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:57.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:56 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi057-77430-1","var":"pgp_num","val":"11"}]: dispatch 2024-06-11T13:32:57.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:56 smithi057 ceph-mon[29805]: from='client.24709 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi057-77430-1","var":"pgp_num","val":"11"}]: dispatch 2024-06-11T13:32:57.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:56 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1729312227' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi057-77351-28"}]: dispatch 2024-06-11T13:32:57.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:56 smithi057 ceph-mon[29805]: from='client.26071 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi057-77351-28"}]: dispatch 2024-06-11T13:32:57.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:56 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1729312227' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi057-77351-28"}]: dispatch 2024-06-11T13:32:57.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:56 smithi057 ceph-mon[29805]: from='client.26071 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi057-77351-28"}]: dispatch 2024-06-11T13:32:57.038 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:56 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1729312227' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi057-77351-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:32:57.038 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:56 smithi057 ceph-mon[29805]: from='client.26071 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi057-77351-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:32:57.038 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:56 smithi057 ceph-mon[29805]: from='client.24709 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi057-77430-1","var":"pgp_num","val":"11"}]': finished 2024-06-11T13:32:57.038 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:56 smithi057 ceph-mon[29805]: from='client.26071 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi057-77351-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:32:57.038 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:56 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:57.038 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:56 smithi057 ceph-mon[29805]: osdmap e178: 8 total, 8 up, 8 in 2024-06-11T13:32:57.038 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:56 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1729312227' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi057-77351-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi057-77351-28"}]: dispatch 2024-06-11T13:32:57.038 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:56 smithi057 ceph-mon[29805]: from='client.26071 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi057-77351-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi057-77351-28"}]: dispatch 2024-06-11T13:32:57.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:56 smithi107 ceph-mon[33178]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:32:57.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:56 smithi107 ceph-mon[33178]: from='client.36083 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi057-77362-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:57.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:56 smithi107 ceph-mon[33178]: from='client.26062 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi057-77351-27"}]': finished 2024-06-11T13:32:57.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:56 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-13"}]': finished 2024-06-11T13:32:57.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:56 smithi107 ceph-mon[33178]: from='client.24709 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi057-77430-1","var":"pg_num","val":"11"}]': finished 2024-06-11T13:32:57.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:56 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:57.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:56 smithi107 ceph-mon[33178]: osdmap e177: 8 total, 8 up, 8 in 2024-06-11T13:32:57.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:56 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:57.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:56 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi057-77430-1","var":"pgp_num","val":"11"}]: dispatch 2024-06-11T13:32:57.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:56 smithi107 ceph-mon[33178]: from='client.24709 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi057-77430-1","var":"pgp_num","val":"11"}]: dispatch 2024-06-11T13:32:57.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:56 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1729312227' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi057-77351-28"}]: dispatch 2024-06-11T13:32:57.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:56 smithi107 ceph-mon[33178]: from='client.26071 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi057-77351-28"}]: dispatch 2024-06-11T13:32:57.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:56 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1729312227' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi057-77351-28"}]: dispatch 2024-06-11T13:32:57.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:56 smithi107 ceph-mon[33178]: from='client.26071 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi057-77351-28"}]: dispatch 2024-06-11T13:32:57.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:56 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1729312227' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi057-77351-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:32:57.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:56 smithi107 ceph-mon[33178]: from='client.26071 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi057-77351-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:32:57.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:56 smithi107 ceph-mon[33178]: from='client.24709 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi057-77430-1","var":"pgp_num","val":"11"}]': finished 2024-06-11T13:32:57.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:56 smithi107 ceph-mon[33178]: from='client.26071 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi057-77351-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:32:57.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:56 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640488007' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-06-11T13:32:57.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:56 smithi107 ceph-mon[33178]: osdmap e178: 8 total, 8 up, 8 in 2024-06-11T13:32:57.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:56 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1729312227' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi057-77351-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi057-77351-28"}]: dispatch 2024-06-11T13:32:57.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:56 smithi107 ceph-mon[33178]: from='client.26071 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi057-77351-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi057-77351-28"}]: dispatch 2024-06-11T13:32:58.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:57 smithi057 ceph-mon[29805]: pgmap v234: 396 pgs: 32 unknown, 364 active+clean; 597 KiB data, 602 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:32:58.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:57 smithi057 ceph-mon[29805]: osdmap e179: 8 total, 8 up, 8 in 2024-06-11T13:32:58.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:57 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/832398256' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi057-77362-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:58.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:57 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:58.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:57 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:58.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:57 smithi057 ceph-mon[29805]: from='client.36092 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi057-77362-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:58.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:57 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1857408807' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi057-77430-2"}]: dispatch 2024-06-11T13:32:58.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:57 smithi057 ceph-mon[29805]: from='client.36098 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi057-77430-2"}]: dispatch 2024-06-11T13:32:58.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:57 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1857408807' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi057-77430-2"}]: dispatch 2024-06-11T13:32:58.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:57 smithi057 ceph-mon[29805]: from='client.36098 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi057-77430-2"}]: dispatch 2024-06-11T13:32:58.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:57 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1857408807' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi057-77430-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:32:58.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:57 smithi057 ceph-mon[29805]: from='client.36098 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi057-77430-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:32:58.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:57 smithi057 ceph-mon[40588]: pgmap v234: 396 pgs: 32 unknown, 364 active+clean; 597 KiB data, 602 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:32:58.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:57 smithi057 ceph-mon[40588]: osdmap e179: 8 total, 8 up, 8 in 2024-06-11T13:32:58.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:57 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/832398256' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi057-77362-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:58.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:57 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:58.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:57 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:58.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:57 smithi057 ceph-mon[40588]: from='client.36092 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi057-77362-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:58.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:57 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1857408807' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi057-77430-2"}]: dispatch 2024-06-11T13:32:58.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:57 smithi057 ceph-mon[40588]: from='client.36098 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi057-77430-2"}]: dispatch 2024-06-11T13:32:58.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:57 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1857408807' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi057-77430-2"}]: dispatch 2024-06-11T13:32:58.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:57 smithi057 ceph-mon[40588]: from='client.36098 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi057-77430-2"}]: dispatch 2024-06-11T13:32:58.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:57 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1857408807' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi057-77430-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:32:58.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:57 smithi057 ceph-mon[40588]: from='client.36098 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi057-77430-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:32:58.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:57 smithi107 ceph-mon[33178]: pgmap v234: 396 pgs: 32 unknown, 364 active+clean; 597 KiB data, 602 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:32:58.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:57 smithi107 ceph-mon[33178]: osdmap e179: 8 total, 8 up, 8 in 2024-06-11T13:32:58.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:57 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/832398256' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi057-77362-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:58.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:57 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:58.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:57 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:58.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:57 smithi107 ceph-mon[33178]: from='client.36092 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi057-77362-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:32:58.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:57 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1857408807' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi057-77430-2"}]: dispatch 2024-06-11T13:32:58.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:57 smithi107 ceph-mon[33178]: from='client.36098 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi057-77430-2"}]: dispatch 2024-06-11T13:32:58.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:57 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1857408807' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi057-77430-2"}]: dispatch 2024-06-11T13:32:58.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:57 smithi107 ceph-mon[33178]: from='client.36098 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi057-77430-2"}]: dispatch 2024-06-11T13:32:58.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:57 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1857408807' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi057-77430-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:32:58.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:57 smithi107 ceph-mon[33178]: from='client.36098 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi057-77430-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:32:59.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:59 smithi057 ceph-mon[29805]: pgmap v237: 364 pgs: 64 unknown, 300 active+clean; 608 KiB data, 577 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:32:59.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:59 smithi057 ceph-mon[29805]: from='client.26071 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi057-77351-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi057-77351-28"}]': finished 2024-06-11T13:32:59.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:59 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:59.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:59 smithi057 ceph-mon[29805]: from='client.36092 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi057-77362-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:59.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:59 smithi057 ceph-mon[29805]: from='client.36098 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi057-77430-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:32:59.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:59 smithi057 ceph-mon[29805]: osdmap e180: 8 total, 8 up, 8 in 2024-06-11T13:32:59.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:59 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1857408807' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi057-77430-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi057-77430-2"}]: dispatch 2024-06-11T13:32:59.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:32:59 smithi057 ceph-mon[29805]: from='client.36098 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi057-77430-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi057-77430-2"}]: dispatch 2024-06-11T13:32:59.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:59 smithi057 ceph-mon[40588]: pgmap v237: 364 pgs: 64 unknown, 300 active+clean; 608 KiB data, 577 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:32:59.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:59 smithi057 ceph-mon[40588]: from='client.26071 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi057-77351-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi057-77351-28"}]': finished 2024-06-11T13:32:59.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:59 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:59.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:59 smithi057 ceph-mon[40588]: from='client.36092 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi057-77362-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:59.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:59 smithi057 ceph-mon[40588]: from='client.36098 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi057-77430-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:32:59.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:59 smithi057 ceph-mon[40588]: osdmap e180: 8 total, 8 up, 8 in 2024-06-11T13:32:59.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:59 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1857408807' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi057-77430-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi057-77430-2"}]: dispatch 2024-06-11T13:32:59.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:32:59 smithi057 ceph-mon[40588]: from='client.36098 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi057-77430-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi057-77430-2"}]: dispatch 2024-06-11T13:32:59.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:59 smithi107 ceph-mon[33178]: pgmap v237: 364 pgs: 64 unknown, 300 active+clean; 608 KiB data, 577 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:32:59.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:59 smithi107 ceph-mon[33178]: from='client.26071 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi057-77351-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi057-77351-28"}]': finished 2024-06-11T13:32:59.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:59 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:59.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:59 smithi107 ceph-mon[33178]: from='client.36092 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi057-77362-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:32:59.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:59 smithi107 ceph-mon[33178]: from='client.36098 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi057-77430-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:32:59.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:59 smithi107 ceph-mon[33178]: osdmap e180: 8 total, 8 up, 8 in 2024-06-11T13:32:59.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:59 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1857408807' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi057-77430-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi057-77430-2"}]: dispatch 2024-06-11T13:32:59.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:32:59 smithi107 ceph-mon[33178]: from='client.36098 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi057-77430-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi057-77430-2"}]: dispatch 2024-06-11T13:33:00.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:00 smithi057 ceph-mon[29805]: osdmap e181: 8 total, 8 up, 8 in 2024-06-11T13:33:00.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:00 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:33:00.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:00 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:33:00.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:00 smithi057 ceph-mon[29805]: pgmap v240: 340 pgs: 40 unknown, 300 active+clean; 608 KiB data, 577 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:33:00.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:00 smithi057 ceph-mon[29805]: from='client.36098 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi057-77430-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi057-77430-2"}]': finished 2024-06-11T13:33:00.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:00 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-15", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:33:00.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:00 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-15"}]: dispatch 2024-06-11T13:33:00.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:00 smithi057 ceph-mon[29805]: osdmap e182: 8 total, 8 up, 8 in 2024-06-11T13:33:00.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:00 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1729312227' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi057-77351-28"}]: dispatch 2024-06-11T13:33:00.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:00 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3532803000' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi057-77362-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:00.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:00 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-15"}]: dispatch 2024-06-11T13:33:00.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:00 smithi057 ceph-mon[29805]: from='client.26071 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi057-77351-28"}]: dispatch 2024-06-11T13:33:00.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:00 smithi057 ceph-mon[29805]: from='client.36104 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi057-77362-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:00.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:00 smithi057 ceph-mon[40588]: osdmap e181: 8 total, 8 up, 8 in 2024-06-11T13:33:00.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:00 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:33:00.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:00 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:33:00.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:00 smithi057 ceph-mon[40588]: pgmap v240: 340 pgs: 40 unknown, 300 active+clean; 608 KiB data, 577 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:33:00.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:00 smithi057 ceph-mon[40588]: from='client.36098 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi057-77430-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi057-77430-2"}]': finished 2024-06-11T13:33:00.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:00 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-15", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:33:00.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:00 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-15"}]: dispatch 2024-06-11T13:33:00.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:00 smithi057 ceph-mon[40588]: osdmap e182: 8 total, 8 up, 8 in 2024-06-11T13:33:00.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:00 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1729312227' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi057-77351-28"}]: dispatch 2024-06-11T13:33:00.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:00 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3532803000' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi057-77362-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:00.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:00 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-15"}]: dispatch 2024-06-11T13:33:00.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:00 smithi057 ceph-mon[40588]: from='client.26071 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi057-77351-28"}]: dispatch 2024-06-11T13:33:00.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:00 smithi057 ceph-mon[40588]: from='client.36104 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi057-77362-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:00.551 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:00 smithi107 ceph-mon[33178]: osdmap e181: 8 total, 8 up, 8 in 2024-06-11T13:33:00.551 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:00 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:33:00.551 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:00 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:33:00.551 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:00 smithi107 ceph-mon[33178]: pgmap v240: 340 pgs: 40 unknown, 300 active+clean; 608 KiB data, 577 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:33:00.551 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:00 smithi107 ceph-mon[33178]: from='client.36098 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi057-77430-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi057-77430-2"}]': finished 2024-06-11T13:33:00.551 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:00 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-15", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:33:00.551 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:00 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-15"}]: dispatch 2024-06-11T13:33:00.551 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:00 smithi107 ceph-mon[33178]: osdmap e182: 8 total, 8 up, 8 in 2024-06-11T13:33:00.552 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:00 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1729312227' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi057-77351-28"}]: dispatch 2024-06-11T13:33:00.552 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:00 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3532803000' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi057-77362-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:00.552 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:00 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-15"}]: dispatch 2024-06-11T13:33:00.552 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:00 smithi107 ceph-mon[33178]: from='client.26071 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi057-77351-28"}]: dispatch 2024-06-11T13:33:00.552 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:00 smithi107 ceph-mon[33178]: from='client.36104 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi057-77362-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:00.770 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleWritery=11 expected=11 2024-06-11T13:33:00.770 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : seek to 17:863748b0:::15:head 2024-06-11T13:33:00.771 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : cursor()=17:863748b0:::15:head expected=17:863748b0:::15:head 2024-06-11T13:33:00.771 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > 17:863748b0:::15:head -> 15 2024-06-11T13:33:00.771 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : entry=15 expected=15 2024-06-11T13:33:00.771 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : seek to 17:6cac518f:::0:head 2024-06-11T13:33:00.771 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : cursor()=17:6cac518f:::0:head expected=17:6cac518f:::0:head 2024-06-11T13:33:00.771 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > 17:6cac518f:::0:head -> 0 2024-06-11T13:33:00.771 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : entry=0 expected=0 2024-06-11T13:33:00.771 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : seek to 17:62a1935d:::14:head 2024-06-11T13:33:00.771 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : cursor()=17:62a1935d:::14:head expected=17:62a1935d:::14:head 2024-06-11T13:33:00.771 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > 17:62a1935d:::14:head -> 14 2024-06-11T13:33:00.771 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : entry=14 expected=14 2024-06-11T13:33:00.771 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : seek to 17:5c6b0b28:::7:head 2024-06-11T13:33:00.771 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : cursor()=17:5c6b0b28:::7:head expected=17:5c6b0b28:::7:head 2024-06-11T13:33:00.771 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > 17:5c6b0b28:::7:head -> 7 2024-06-11T13:33:00.772 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : entry=7 expected=7 2024-06-11T13:33:00.772 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : seek to 17:f905c69b:::2:head 2024-06-11T13:33:00.772 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : cursor()=17:f905c69b:::2:head expected=17:f905c69b:::2:head 2024-06-11T13:33:00.772 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > 17:f905c69b:::2:head -> 2 2024-06-11T13:33:00.772 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : entry=2 expected=2 2024-06-11T13:33:00.772 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : seek to 17:e960b815:::9:head 2024-06-11T13:33:00.772 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : cursor()=17:e960b815:::9:head expected=17:e960b815:::9:head 2024-06-11T13:33:00.772 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > 17:e960b815:::9:head -> 9 2024-06-11T13:33:00.772 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : entry=9 expected=9 2024-06-11T13:33:00.772 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : seek to 17:566253c9:::13:head 2024-06-11T13:33:00.772 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : cursor()=17:566253c9:::13:head expected=17:566253c9:::13:head 2024-06-11T13:33:00.772 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > 17:566253c9:::13:head -> 13 2024-06-11T13:33:00.772 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : entry=13 expected=13 2024-06-11T13:33:00.772 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : seek to 17:02547ec2:::1:head 2024-06-11T13:33:00.772 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : cursor()=17:02547ec2:::1:head expected=17:02547ec2:::1:head 2024-06-11T13:33:00.773 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: > 17:02547ec2:::1:head -> 1 2024-06-11T13:33:00.773 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: : entry=1 expected=1 2024-06-11T13:33:00.773 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: [ OK ] LibRadosList.ListObjectsCursor (216 ms) 2024-06-11T13:33:00.773 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjects 2024-06-11T13:33:00.773 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: [ OK ] LibRadosList.EnumerateObjects (103479 ms) 2024-06-11T13:33:00.773 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjectsSplit 2024-06-11T13:33:00.773 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: split 0/5 -> MIN 17:33333333::::head 2024-06-11T13:33:00.773 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: split 1/5 -> 17:33333333::::head 17:66666666::::head 2024-06-11T13:33:00.773 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: split 2/5 -> 17:66666666::::head 17:99999999::::head 2024-06-11T13:33:00.773 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: split 3/5 -> 17:99999999::::head 17:cccccccc::::head 2024-06-11T13:33:00.773 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: split 4/5 -> 17:cccccccc::::head MAX 2024-06-11T13:33:00.773 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: [ OK ] LibRadosList.EnumerateObjectsSplit (12413 ms) 2024-06-11T13:33:00.773 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: [----------] 7 tests from LibRadosList (116459 ms total) 2024-06-11T13:33:00.773 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: 2024-06-11T13:33:00.774 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: [----------] 3 tests from LibRadosListEC 2024-06-11T13:33:00.774 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: [ RUN ] LibRadosListEC.ListObjects 2024-06-11T13:33:00.774 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: [ OK ] LibRadosListEC.ListObjects (1038 ms) 2024-06-11T13:33:00.774 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsNS 2024-06-11T13:33:00.774 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: myset foo1,foo2,foo3 2024-06-11T13:33:00.774 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: foo1 2024-06-11T13:33:00.774 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: foo2 2024-06-11T13:33:00.774 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: foo3 2024-06-11T13:33:00.774 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: myset foo1,foo4,foo5 2024-06-11T13:33:00.774 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: foo4 2024-06-11T13:33:00.774 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: foo5 2024-06-11T13:33:00.774 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: foo1 2024-06-11T13:33:00.774 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: myset foo6,foo7 2024-06-11T13:33:00.774 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: foo7 2024-06-11T13:33:00.775 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: foo6 2024-06-11T13:33:00.775 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-06-11T13:33:00.775 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: ns1:foo4 2024-06-11T13:33:00.775 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: ns1:foo5 2024-06-11T13:33:00.775 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: ns2:foo7 2024-06-11T13:33:00.775 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: ns2:foo6 2024-06-11T13:33:00.775 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: ns1:foo1 2024-06-11T13:33:00.775 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: :foo1 2024-06-11T13:33:00.775 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: :foo2 2024-06-11T13:33:00.775 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: :foo3 2024-06-11T13:33:00.775 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsNS (53 ms) 2024-06-11T13:33:01.449 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:33:01 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:33:01.449 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:01 smithi107 ceph-mon[33178]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:33:01.449 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:01 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-15"}]': finished 2024-06-11T13:33:01.449 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:01 smithi107 ceph-mon[33178]: from='client.26071 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi057-77351-28"}]': finished 2024-06-11T13:33:01.449 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:01 smithi107 ceph-mon[33178]: from='client.36104 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi057-77362-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:01.449 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:01 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-15", "mode": "writeback"}]: dispatch 2024-06-11T13:33:01.449 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:01 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1729312227' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi057-77351-28"}]: dispatch 2024-06-11T13:33:01.449 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:01 smithi107 ceph-mon[33178]: osdmap e183: 8 total, 8 up, 8 in 2024-06-11T13:33:01.450 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:01 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-15", "mode": "writeback"}]: dispatch 2024-06-11T13:33:01.450 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:01 smithi107 ceph-mon[33178]: from='client.26071 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi057-77351-28"}]: dispatch 2024-06-11T13:33:01.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:01 smithi057 ceph-mon[29805]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:33:01.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:01 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-15"}]': finished 2024-06-11T13:33:01.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:01 smithi057 ceph-mon[29805]: from='client.26071 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi057-77351-28"}]': finished 2024-06-11T13:33:01.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:01 smithi057 ceph-mon[29805]: from='client.36104 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi057-77362-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:01.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:01 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-15", "mode": "writeback"}]: dispatch 2024-06-11T13:33:01.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:01 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1729312227' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi057-77351-28"}]: dispatch 2024-06-11T13:33:01.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:01 smithi057 ceph-mon[29805]: osdmap e183: 8 total, 8 up, 8 in 2024-06-11T13:33:01.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:01 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-15", "mode": "writeback"}]: dispatch 2024-06-11T13:33:01.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:01 smithi057 ceph-mon[29805]: from='client.26071 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi057-77351-28"}]: dispatch 2024-06-11T13:33:01.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:01 smithi057 ceph-mon[40588]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:33:01.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:01 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-15"}]': finished 2024-06-11T13:33:01.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:01 smithi057 ceph-mon[40588]: from='client.26071 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi057-77351-28"}]': finished 2024-06-11T13:33:01.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:01 smithi057 ceph-mon[40588]: from='client.36104 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi057-77362-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:01.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:01 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-15", "mode": "writeback"}]: dispatch 2024-06-11T13:33:01.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:01 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1729312227' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi057-77351-28"}]: dispatch 2024-06-11T13:33:01.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:01 smithi057 ceph-mon[40588]: osdmap e183: 8 total, 8 up, 8 in 2024-06-11T13:33:01.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:01 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-15", "mode": "writeback"}]: dispatch 2024-06-11T13:33:01.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:01 smithi057 ceph-mon[40588]: from='client.26071 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi057-77351-28"}]: dispatch 2024-06-11T13:33:01.670 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: [ RUN ] LibRadosListEC.L api_aio_pp: Running main() from gmock_main.cc 2024-06-11T13:33:01.670 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [==========] Running 57 tests from 4 test suites. 2024-06-11T13:33:01.670 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [----------] Global test environment set-up. 2024-06-11T13:33:01.670 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio 2024-06-11T13:33:01.670 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAio.TooBigPP 2024-06-11T13:33:01.670 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAio.TooBigPP (19085 ms) 2024-06-11T13:33:01.670 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolQuotaPP 2024-06-11T13:33:01.670 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolQuotaPP (27183 ms) 2024-06-11T13:33:01.670 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleWritePP 2024-06-11T13:33:01.670 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleWritePP (10812 ms) 2024-06-11T13:33:01.670 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAio.WaitForSafePP 2024-06-11T13:33:01.670 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAio.WaitForSafePP (4797 ms) 2024-06-11T13:33:01.671 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP 2024-06-11T13:33:01.671 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP (3829 ms) 2024-06-11T13:33:01.671 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP2 2024-06-11T13:33:01.671 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP2 (4911 ms) 2024-06-11T13:33:01.671 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP3 2024-06-11T13:33:01.671 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP3 (5720 ms) 2024-06-11T13:33:01.671 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripSparseReadPP 2024-06-11T13:33:01.671 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripSparseReadPP (4324 ms) 2024-06-11T13:33:01.671 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsCompletePP 2024-06-11T13:33:01.671 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAio.IsCompletePP (3411 ms) 2024-06-11T13:33:01.671 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsSafePP 2024-06-11T13:33:01.671 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAio.IsSafePP (3439 ms) 2024-06-11T13:33:01.671 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAio.ReturnValuePP 2024-06-11T13:33:01.671 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAio.ReturnValuePP (2558 ms) 2024-06-11T13:33:01.672 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushPP 2024-06-11T13:33:01.672 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushPP (2996 ms) 2024-06-11T13:33:01.672 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushAsyncPP 2024-06-11T13:33:01.672 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushAsyncPP (3021 ms) 2024-06-11T13:33:01.672 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP 2024-06-11T13:33:01.672 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP (3008 ms) 2024-06-11T13:33:01.672 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP2 2024-06-11T13:33:01.672 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP2 (3007 ms) 2024-06-11T13:33:01.672 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP 2024-06-11T13:33:01.672 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP (3430 ms) 2024-06-11T13:33:01.672 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP2 2024-06-11T13:33:01.672 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP2 (3062 ms) 2024-06-11T13:33:01.672 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPPNS 2024-06-11T13:33:01.672 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPPNS (3004 ms) 2024-06-11T13:33:01.672 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPP 2024-06-11T13:33:01.673 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPP (3029 ms) 2024-06-11T13:33:01.673 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime 2024-06-11T13:33:01.673 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime (2470 ms) 2024-06-11T13:33:01.673 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime2 2024-06-11T13:33:01.673 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime2 (3018 ms) 2024-06-11T13:33:01.673 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAio.StatRemovePP 2024-06-11T13:33:01.673 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAio.StatRemovePP (3038 ms) 2024-06-11T13:33:01.673 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAio.ExecuteClassPP 2024-06-11T13:33:01.673 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAio.ExecuteClassPP (3446 ms) 2024-06-11T13:33:01.673 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAio.OmapPP 2024-06-11T13:33:01.673 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAio.OmapPP (2558 ms) 2024-06-11T13:33:01.673 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiWritePP 2024-06-11T13:33:01.673 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiWritePP (2946 ms) 2024-06-11T13:33:01.673 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAio.AioUnlockPP 2024-06-11T13:33:01.674 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAio.AioUnlockPP (3461 ms) 2024-06-11T13:33:01.674 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripAppendPP 2024-06-11T13:33:02.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:02 smithi057 ceph-mon[29805]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:33:02.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:02 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-15", "mode": "writeback"}]': finished 2024-06-11T13:33:02.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:02 smithi057 ceph-mon[29805]: from='client.26071 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi057-77351-28"}]': finished 2024-06-11T13:33:02.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:02 smithi057 ceph-mon[29805]: osdmap e184: 8 total, 8 up, 8 in 2024-06-11T13:33:02.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:02 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1857408807' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi057-77430-2"}]: dispatch 2024-06-11T13:33:02.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:02 smithi057 ceph-mon[29805]: from='client.36098 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi057-77430-2"}]: dispatch 2024-06-11T13:33:02.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:02 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/141109580' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi057-77351-29"}]: dispatch 2024-06-11T13:33:02.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:02 smithi057 ceph-mon[29805]: from='client.26098 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi057-77351-29"}]: dispatch 2024-06-11T13:33:02.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:02 smithi057 ceph-mon[40588]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:33:02.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:02 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-15", "mode": "writeback"}]': finished 2024-06-11T13:33:02.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:02 smithi057 ceph-mon[40588]: from='client.26071 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi057-77351-28"}]': finished 2024-06-11T13:33:02.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:02 smithi057 ceph-mon[40588]: osdmap e184: 8 total, 8 up, 8 in 2024-06-11T13:33:02.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:02 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1857408807' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi057-77430-2"}]: dispatch 2024-06-11T13:33:02.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:02 smithi057 ceph-mon[40588]: from='client.36098 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi057-77430-2"}]: dispatch 2024-06-11T13:33:02.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:02 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/141109580' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi057-77351-29"}]: dispatch 2024-06-11T13:33:02.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:02 smithi057 ceph-mon[40588]: from='client.26098 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi057-77351-29"}]: dispatch 2024-06-11T13:33:02.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:02 smithi107 ceph-mon[33178]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:33:02.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:02 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-15", "mode": "writeback"}]': finished 2024-06-11T13:33:02.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:02 smithi107 ceph-mon[33178]: from='client.26071 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi057-77351-28"}]': finished 2024-06-11T13:33:02.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:02 smithi107 ceph-mon[33178]: osdmap e184: 8 total, 8 up, 8 in 2024-06-11T13:33:02.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:02 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1857408807' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi057-77430-2"}]: dispatch 2024-06-11T13:33:02.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:02 smithi107 ceph-mon[33178]: from='client.36098 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi057-77430-2"}]: dispatch 2024-06-11T13:33:02.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:02 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/141109580' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi057-77351-29"}]: dispatch 2024-06-11T13:33:02.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:02 smithi107 ceph-mon[33178]: from='client.26098 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi057-77351-29"}]: dispatch 2024-06-11T13:33:03.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:03 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:33:03.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:03 smithi057 ceph-mon[29805]: pgmap v243: 372 pgs: 372 active+clean; 608 KiB data, 549 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 9.2 KiB/s wr, 15 op/s 2024-06-11T13:33:03.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:03 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/141109580' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi057-77351-29"}]: dispatch 2024-06-11T13:33:03.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:03 smithi057 ceph-mon[29805]: from='client.26098 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi057-77351-29"}]: dispatch 2024-06-11T13:33:03.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:03 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/141109580' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi057-77351-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:03.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:03 smithi057 ceph-mon[29805]: from='client.26098 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi057-77351-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:03.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:03 smithi057 ceph-mon[29805]: from='client.36098 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi057-77430-2"}]': finished 2024-06-11T13:33:03.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:03 smithi057 ceph-mon[29805]: from='client.26098 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi057-77351-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:33:03.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:03 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/141109580' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi057-77351-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi057-77351-29"}]: dispatch 2024-06-11T13:33:03.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:03 smithi057 ceph-mon[29805]: osdmap e185: 8 total, 8 up, 8 in 2024-06-11T13:33:03.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:03 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1857408807' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi057-77430-2"}]: dispatch 2024-06-11T13:33:03.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:03 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3514137530' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21"}]: dispatch 2024-06-11T13:33:03.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:03 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2261405029' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi057-77362-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:03.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:03 smithi057 ceph-mon[29805]: from='client.36098 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi057-77430-2"}]: dispatch 2024-06-11T13:33:03.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:03 smithi057 ceph-mon[29805]: from='client.26098 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi057-77351-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi057-77351-29"}]: dispatch 2024-06-11T13:33:03.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:03 smithi057 ceph-mon[29805]: from='client.36071 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21"}]: dispatch 2024-06-11T13:33:03.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:03 smithi057 ceph-mon[29805]: from='client.26101 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi057-77362-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:03.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:03 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:33:03.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:03 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:33:03.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:03 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:33:03.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:03 smithi057 ceph-mon[40588]: pgmap v243: 372 pgs: 372 active+clean; 608 KiB data, 549 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 9.2 KiB/s wr, 15 op/s 2024-06-11T13:33:03.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:03 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/141109580' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi057-77351-29"}]: dispatch 2024-06-11T13:33:03.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:03 smithi057 ceph-mon[40588]: from='client.26098 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi057-77351-29"}]: dispatch 2024-06-11T13:33:03.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:03 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/141109580' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi057-77351-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:03.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:03 smithi057 ceph-mon[40588]: from='client.26098 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi057-77351-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:03.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:03 smithi057 ceph-mon[40588]: from='client.36098 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi057-77430-2"}]': finished 2024-06-11T13:33:03.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:03 smithi057 ceph-mon[40588]: from='client.26098 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi057-77351-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:33:03.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:03 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/141109580' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi057-77351-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi057-77351-29"}]: dispatch 2024-06-11T13:33:03.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:03 smithi057 ceph-mon[40588]: osdmap e185: 8 total, 8 up, 8 in 2024-06-11T13:33:03.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:03 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1857408807' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi057-77430-2"}]: dispatch 2024-06-11T13:33:03.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:03 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3514137530' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21"}]: dispatch 2024-06-11T13:33:03.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:03 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2261405029' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi057-77362-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:03.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:03 smithi057 ceph-mon[40588]: from='client.36098 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi057-77430-2"}]: dispatch 2024-06-11T13:33:03.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:03 smithi057 ceph-mon[40588]: from='client.26098 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi057-77351-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi057-77351-29"}]: dispatch 2024-06-11T13:33:03.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:03 smithi057 ceph-mon[40588]: from='client.36071 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21"}]: dispatch 2024-06-11T13:33:03.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:03 smithi057 ceph-mon[40588]: from='client.26101 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi057-77362-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:03.538 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:03 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:33:03.538 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:03 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:33:03.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:03 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:33:03.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:03 smithi107 ceph-mon[33178]: pgmap v243: 372 pgs: 372 active+clean; 608 KiB data, 549 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 9.2 KiB/s wr, 15 op/s 2024-06-11T13:33:03.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:03 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/141109580' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi057-77351-29"}]: dispatch 2024-06-11T13:33:03.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:03 smithi107 ceph-mon[33178]: from='client.26098 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi057-77351-29"}]: dispatch 2024-06-11T13:33:03.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:03 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/141109580' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi057-77351-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:03.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:03 smithi107 ceph-mon[33178]: from='client.26098 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi057-77351-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:03.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:03 smithi107 ceph-mon[33178]: from='client.36098 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi057-77430-2"}]': finished 2024-06-11T13:33:03.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:03 smithi107 ceph-mon[33178]: from='client.26098 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi057-77351-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:33:03.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:03 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/141109580' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi057-77351-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi057-77351-29"}]: dispatch 2024-06-11T13:33:03.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:03 smithi107 ceph-mon[33178]: osdmap e185: 8 total, 8 up, 8 in 2024-06-11T13:33:03.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:03 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1857408807' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi057-77430-2"}]: dispatch 2024-06-11T13:33:03.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:03 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3514137530' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21"}]: dispatch 2024-06-11T13:33:03.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:03 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2261405029' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi057-77362-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:03.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:03 smithi107 ceph-mon[33178]: from='client.36098 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi057-77430-2"}]: dispatch 2024-06-11T13:33:03.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:03 smithi107 ceph-mon[33178]: from='client.26098 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi057-77351-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi057-77351-29"}]: dispatch 2024-06-11T13:33:03.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:03 smithi107 ceph-mon[33178]: from='client.36071 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21"}]: dispatch 2024-06-11T13:33:03.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:03 smithi107 ceph-mon[33178]: from='client.26101 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi057-77362-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:03.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:03 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:33:03.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:03 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:33:04.034 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:33:03 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:33:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:33:04.727 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAhotsECPP.SnapRemovePP (2026 ms) 2024-06-11T13:33:04.728 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.RollbackPP 2024-06-11T13:33:04.728 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.RollbackPP (1450 ms) 2024-06-11T13:33:04.728 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapGetNamePP 2024-06-11T13:33:04.728 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapGetNamePP (2011 ms) 2024-06-11T13:33:04.728 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP (8512 ms total) 2024-06-11T13:33:04.728 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: 2024-06-11T13:33:04.728 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP 2024-06-11T13:33:04.728 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.SnapPP 2024-06-11T13:33:04.729 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.SnapPP (4047 ms) 2024-06-11T13:33:04.729 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.RollbackPP 2024-06-11T13:33:04.729 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.RollbackPP (3915 ms) 2024-06-11T13:33:04.729 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.Bug11677 2024-06-11T13:33:04.729 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.Bug11677 (4002 ms) 2024-06-11T13:33:04.729 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP (11964 ms total) 2024-06-11T13:33:04.729 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: 2024-06-11T13:33:04.729 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [----------] Global test environment tear-down 2024-06-11T13:33:04.730 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [==========] 21 tests from 5 test suites ran. (141011 ms total) 2024-06-11T13:33:04.730 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [ PASSED ] 20 tests. 2024-06-11T13:33:04.730 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [ SKIPPED ] 1 test, listed below: 2024-06-11T13:33:04.730 INFO:tasks.workunit.client.0.smithi057.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-06-11T13:33:05.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:04 smithi057 ceph-mon[40588]: pgmap v246: 356 pgs: 32 unknown, 324 active+clean; 584 KiB data, 549 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:33:05.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:04 smithi057 ceph-mon[40588]: from='client.36098 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi057-77430-2"}]': finished 2024-06-11T13:33:05.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:04 smithi057 ceph-mon[40588]: from='client.36071 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21"}]': finished 2024-06-11T13:33:05.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:04 smithi057 ceph-mon[40588]: from='client.26101 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi057-77362-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:05.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:04 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:33:05.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:04 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-15"}]: dispatch 2024-06-11T13:33:05.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:04 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3514137530' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21"}]: dispatch 2024-06-11T13:33:05.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:04 smithi057 ceph-mon[40588]: osdmap e186: 8 total, 8 up, 8 in 2024-06-11T13:33:05.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:04 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-15"}]: dispatch 2024-06-11T13:33:05.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:04 smithi057 ceph-mon[40588]: from='client.36071 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21"}]: dispatch 2024-06-11T13:33:05.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:04 smithi057 ceph-mon[29805]: pgmap v246: 356 pgs: 32 unknown, 324 active+clean; 584 KiB data, 549 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:33:05.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:04 smithi057 ceph-mon[29805]: from='client.36098 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi057-77430-2"}]': finished 2024-06-11T13:33:05.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:04 smithi057 ceph-mon[29805]: from='client.36071 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21"}]': finished 2024-06-11T13:33:05.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:04 smithi057 ceph-mon[29805]: from='client.26101 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi057-77362-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:05.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:04 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:33:05.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:04 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-15"}]: dispatch 2024-06-11T13:33:05.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:04 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3514137530' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21"}]: dispatch 2024-06-11T13:33:05.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:04 smithi057 ceph-mon[29805]: osdmap e186: 8 total, 8 up, 8 in 2024-06-11T13:33:05.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:04 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-15"}]: dispatch 2024-06-11T13:33:05.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:04 smithi057 ceph-mon[29805]: from='client.36071 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21"}]: dispatch 2024-06-11T13:33:05.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:04 smithi107 ceph-mon[33178]: pgmap v246: 356 pgs: 32 unknown, 324 active+clean; 584 KiB data, 549 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:33:05.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:04 smithi107 ceph-mon[33178]: from='client.36098 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi057-77430-2"}]': finished 2024-06-11T13:33:05.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:04 smithi107 ceph-mon[33178]: from='client.36071 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21"}]': finished 2024-06-11T13:33:05.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:04 smithi107 ceph-mon[33178]: from='client.26101 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi057-77362-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:05.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:04 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:33:05.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:04 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-15"}]: dispatch 2024-06-11T13:33:05.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:04 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3514137530' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21"}]: dispatch 2024-06-11T13:33:05.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:04 smithi107 ceph-mon[33178]: osdmap e186: 8 total, 8 up, 8 in 2024-06-11T13:33:05.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:04 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-15"}]: dispatch 2024-06-11T13:33:05.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:04 smithi107 ceph-mon[33178]: from='client.36071 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21"}]: dispatch 2024-06-11T13:33:06.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:05 smithi057 ceph-mon[29805]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:33:06.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:05 smithi057 ceph-mon[29805]: from='client.26098 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi057-77351-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi057-77351-29"}]': finished 2024-06-11T13:33:06.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:05 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-15"}]': finished 2024-06-11T13:33:06.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:05 smithi057 ceph-mon[29805]: from='client.36071 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21"}]': finished 2024-06-11T13:33:06.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:05 smithi057 ceph-mon[29805]: osdmap e187: 8 total, 8 up, 8 in 2024-06-11T13:33:06.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:05 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/736786630' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77430-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:06.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:05 smithi057 ceph-mon[29805]: from='client.36122 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77430-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:06.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:05 smithi057 ceph-mon[29805]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:33:06.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:05 smithi057 ceph-mon[40588]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:33:06.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:05 smithi057 ceph-mon[40588]: from='client.26098 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi057-77351-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi057-77351-29"}]': finished 2024-06-11T13:33:06.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:05 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-15"}]': finished 2024-06-11T13:33:06.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:05 smithi057 ceph-mon[40588]: from='client.36071 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21"}]': finished 2024-06-11T13:33:06.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:05 smithi057 ceph-mon[40588]: osdmap e187: 8 total, 8 up, 8 in 2024-06-11T13:33:06.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:05 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/736786630' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77430-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:06.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:05 smithi057 ceph-mon[40588]: from='client.36122 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77430-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:06.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:05 smithi057 ceph-mon[40588]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:33:06.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:05 smithi107 ceph-mon[33178]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:33:06.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:05 smithi107 ceph-mon[33178]: from='client.26098 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi057-77351-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi057-77351-29"}]': finished 2024-06-11T13:33:06.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:05 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-15"}]': finished 2024-06-11T13:33:06.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:05 smithi107 ceph-mon[33178]: from='client.36071 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi057-77714-21"}]': finished 2024-06-11T13:33:06.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:05 smithi107 ceph-mon[33178]: osdmap e187: 8 total, 8 up, 8 in 2024-06-11T13:33:06.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:05 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/736786630' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77430-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:06.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:05 smithi107 ceph-mon[33178]: from='client.36122 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77430-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:06.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:05 smithi107 ceph-mon[33178]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:33:06.734 INFO:tasks.workunit.client.0.smithi057.stdout:istObjectsStart 2024-06-11T13:33:06.735 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: 1 0 2024-06-11T13:33:06.735 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: 10 0 2024-06-11T13:33:06.735 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: 13 0 2024-06-11T13:33:06.735 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: 7 0 2024-06-11T13:33:06.735 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: 14 0 2024-06-11T13:33:06.735 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: 0 0 2024-06-11T13:33:06.735 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: 15 0 2024-06-11T13:33:06.735 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: 11 0 2024-06-11T13:33:06.735 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: 5 0 2024-06-11T13:33:06.735 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: 8 0 2024-06-11T13:33:06.735 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: 6 0 2024-06-11T13:33:06.735 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: 3 0 2024-06-11T13:33:06.736 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: 4 0 2024-06-11T13:33:06.736 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: 12 0 2024-06-11T13:33:06.736 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: 9 0 2024-06-11T13:33:06.736 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: 2 0 2024-06-11T13:33:06.736 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-06-11T13:33:06.736 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsStart (45 ms) 2024-06-11T13:33:06.736 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: [----------] 3 tests from LibRadosListEC (1136 ms total) 2024-06-11T13:33:06.736 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: 2024-06-11T13:33:06.736 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: [----------] 1 test from LibRadosListNP 2024-06-11T13:33:06.736 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: [ RUN ] LibRadosListNP.ListObjectsError 2024-06-11T13:33:06.736 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: [ OK ] LibRadosListNP.ListObjectsError (3048 ms) 2024-06-11T13:33:06.736 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: [----------] 1 test from LibRadosListNP (3048 ms total) 2024-06-11T13:33:06.736 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: 2024-06-11T13:33:06.737 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: [----------] Global test environment tear-down 2024-06-11T13:33:06.737 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: [==========] 11 tests from 3 test suites ran. (143101 ms total) 2024-06-11T13:33:06.737 INFO:tasks.workunit.client.0.smithi057.stdout: api_list: [ PASSED ] 11 tests. 2024-06-11T13:33:07.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:06 smithi057 ceph-mon[29805]: pgmap v249: 364 pgs: 40 unknown, 324 active+clean; 584 KiB data, 549 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:33:07.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:06 smithi057 ceph-mon[29805]: from='client.36122 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77430-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:07.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:06 smithi057 ceph-mon[29805]: osdmap e188: 8 total, 8 up, 8 in 2024-06-11T13:33:07.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:06 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2640657441' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi057-77362-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:07.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:06 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/736786630' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi057-77430-3","pool2":"test-rados-api-smithi057-77430-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-06-11T13:33:07.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:06 smithi057 ceph-mon[29805]: from='client.36128 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi057-77362-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:07.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:06 smithi057 ceph-mon[29805]: from='client.36122 ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi057-77430-3","pool2":"test-rados-api-smithi057-77430-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-06-11T13:33:07.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:06 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:33:07.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:06 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:33:07.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:06 smithi057 ceph-mon[40588]: pgmap v249: 364 pgs: 40 unknown, 324 active+clean; 584 KiB data, 549 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:33:07.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:06 smithi057 ceph-mon[40588]: from='client.36122 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77430-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:07.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:06 smithi057 ceph-mon[40588]: osdmap e188: 8 total, 8 up, 8 in 2024-06-11T13:33:07.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:06 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2640657441' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi057-77362-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:07.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:06 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/736786630' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi057-77430-3","pool2":"test-rados-api-smithi057-77430-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-06-11T13:33:07.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:06 smithi057 ceph-mon[40588]: from='client.36128 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi057-77362-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:07.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:06 smithi057 ceph-mon[40588]: from='client.36122 ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi057-77430-3","pool2":"test-rados-api-smithi057-77430-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-06-11T13:33:07.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:06 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:33:07.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:06 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:33:07.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:06 smithi107 ceph-mon[33178]: pgmap v249: 364 pgs: 40 unknown, 324 active+clean; 584 KiB data, 549 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:33:07.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:06 smithi107 ceph-mon[33178]: from='client.36122 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77430-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:07.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:06 smithi107 ceph-mon[33178]: osdmap e188: 8 total, 8 up, 8 in 2024-06-11T13:33:07.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:06 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2640657441' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi057-77362-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:07.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:06 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/736786630' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi057-77430-3","pool2":"test-rados-api-smithi057-77430-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-06-11T13:33:07.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:06 smithi107 ceph-mon[33178]: from='client.36128 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi057-77362-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:07.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:06 smithi107 ceph-mon[33178]: from='client.36122 ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi057-77430-3","pool2":"test-rados-api-smithi057-77430-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-06-11T13:33:07.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:06 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:33:07.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:06 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:33:08.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:07 smithi057 ceph-mon[40588]: from='client.36128 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi057-77362-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:08.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:07 smithi057 ceph-mon[40588]: from='client.36122 ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi057-77430-3","pool2":"test-rados-api-smithi057-77430-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-06-11T13:33:08.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:07 smithi057 ceph-mon[40588]: osdmap e189: 8 total, 8 up, 8 in 2024-06-11T13:33:08.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:07 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:08.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:07 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/141109580' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi057-77351-29"}]: dispatch 2024-06-11T13:33:08.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:07 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:08.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:07 smithi057 ceph-mon[40588]: from='client.26098 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi057-77351-29"}]: dispatch 2024-06-11T13:33:08.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:07 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:08.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:07 smithi057 ceph-mon[40588]: from='client.26098 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi057-77351-29"}]': finished 2024-06-11T13:33:08.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:07 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/141109580' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi057-77351-29"}]: dispatch 2024-06-11T13:33:08.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:07 smithi057 ceph-mon[40588]: osdmap e190: 8 total, 8 up, 8 in 2024-06-11T13:33:08.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:07 smithi057 ceph-mon[40588]: from='client.26098 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi057-77351-29"}]: dispatch 2024-06-11T13:33:08.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:07 smithi057 ceph-mon[29805]: from='client.36128 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi057-77362-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:08.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:07 smithi057 ceph-mon[29805]: from='client.36122 ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi057-77430-3","pool2":"test-rados-api-smithi057-77430-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-06-11T13:33:08.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:07 smithi057 ceph-mon[29805]: osdmap e189: 8 total, 8 up, 8 in 2024-06-11T13:33:08.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:07 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:08.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:07 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/141109580' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi057-77351-29"}]: dispatch 2024-06-11T13:33:08.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:07 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:08.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:07 smithi057 ceph-mon[29805]: from='client.26098 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi057-77351-29"}]: dispatch 2024-06-11T13:33:08.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:07 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:08.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:07 smithi057 ceph-mon[29805]: from='client.26098 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi057-77351-29"}]': finished 2024-06-11T13:33:08.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:07 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/141109580' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi057-77351-29"}]: dispatch 2024-06-11T13:33:08.038 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:07 smithi057 ceph-mon[29805]: osdmap e190: 8 total, 8 up, 8 in 2024-06-11T13:33:08.038 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:07 smithi057 ceph-mon[29805]: from='client.26098 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi057-77351-29"}]: dispatch 2024-06-11T13:33:08.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:07 smithi107 ceph-mon[33178]: from='client.36128 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi057-77362-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:08.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:07 smithi107 ceph-mon[33178]: from='client.36122 ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi057-77430-3","pool2":"test-rados-api-smithi057-77430-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-06-11T13:33:08.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:07 smithi107 ceph-mon[33178]: osdmap e189: 8 total, 8 up, 8 in 2024-06-11T13:33:08.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:07 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:08.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:07 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/141109580' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi057-77351-29"}]: dispatch 2024-06-11T13:33:08.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:07 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:08.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:07 smithi107 ceph-mon[33178]: from='client.26098 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi057-77351-29"}]: dispatch 2024-06-11T13:33:08.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:07 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:08.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:07 smithi107 ceph-mon[33178]: from='client.26098 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi057-77351-29"}]': finished 2024-06-11T13:33:08.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:07 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/141109580' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi057-77351-29"}]: dispatch 2024-06-11T13:33:08.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:07 smithi107 ceph-mon[33178]: osdmap e190: 8 total, 8 up, 8 in 2024-06-11T13:33:08.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:07 smithi107 ceph-mon[33178]: from='client.26098 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi057-77351-29"}]: dispatch 2024-06-11T13:33:09.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:08 smithi057 ceph-mon[29805]: pgmap v252: 356 pgs: 32 creating+peering, 32 unknown, 292 active+clean; 584 KiB data, 554 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:33:09.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:08 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:33:09.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:08 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:33:09.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:08 smithi057 ceph-mon[29805]: from='client.26098 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi057-77351-29"}]': finished 2024-06-11T13:33:09.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:08 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-17", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:33:09.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:08 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-17"}]: dispatch 2024-06-11T13:33:09.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:08 smithi057 ceph-mon[29805]: osdmap e191: 8 total, 8 up, 8 in 2024-06-11T13:33:09.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:08 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/4203691425' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi057-77362-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:09.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:08 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-17"}]: dispatch 2024-06-11T13:33:09.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:08 smithi057 ceph-mon[29805]: from='client.26119 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi057-77362-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:09.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:08 smithi057 ceph-mon[40588]: pgmap v252: 356 pgs: 32 creating+peering, 32 unknown, 292 active+clean; 584 KiB data, 554 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:33:09.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:08 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:33:09.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:08 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:33:09.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:08 smithi057 ceph-mon[40588]: from='client.26098 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi057-77351-29"}]': finished 2024-06-11T13:33:09.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:08 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-17", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:33:09.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:08 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-17"}]: dispatch 2024-06-11T13:33:09.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:08 smithi057 ceph-mon[40588]: osdmap e191: 8 total, 8 up, 8 in 2024-06-11T13:33:09.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:08 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/4203691425' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi057-77362-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:09.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:08 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-17"}]: dispatch 2024-06-11T13:33:09.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:08 smithi057 ceph-mon[40588]: from='client.26119 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi057-77362-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:09.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:08 smithi107 ceph-mon[33178]: pgmap v252: 356 pgs: 32 creating+peering, 32 unknown, 292 active+clean; 584 KiB data, 554 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:33:09.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:08 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:33:09.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:08 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:33:09.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:08 smithi107 ceph-mon[33178]: from='client.26098 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi057-77351-29"}]': finished 2024-06-11T13:33:09.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:08 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-17", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:33:09.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:08 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-17"}]: dispatch 2024-06-11T13:33:09.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:08 smithi107 ceph-mon[33178]: osdmap e191: 8 total, 8 up, 8 in 2024-06-11T13:33:09.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:08 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/4203691425' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi057-77362-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:09.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:08 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-17"}]: dispatch 2024-06-11T13:33:09.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:08 smithi107 ceph-mon[33178]: from='client.26119 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi057-77362-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:10.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:09 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3971268278' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi057-77351-30"}]: dispatch 2024-06-11T13:33:10.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:09 smithi057 ceph-mon[29805]: from='client.36140 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi057-77351-30"}]: dispatch 2024-06-11T13:33:10.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:09 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3971268278' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi057-77351-30"}]: dispatch 2024-06-11T13:33:10.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:09 smithi057 ceph-mon[29805]: from='client.36140 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi057-77351-30"}]: dispatch 2024-06-11T13:33:10.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:09 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3971268278' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi057-77351-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:10.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:09 smithi057 ceph-mon[29805]: from='client.36140 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi057-77351-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:10.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:09 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-17"}]': finished 2024-06-11T13:33:10.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:09 smithi057 ceph-mon[29805]: from='client.26119 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi057-77362-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:10.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:09 smithi057 ceph-mon[29805]: from='client.36140 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi057-77351-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:33:10.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:09 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-17", "mode": "writeback"}]: dispatch 2024-06-11T13:33:10.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:09 smithi057 ceph-mon[29805]: osdmap e192: 8 total, 8 up, 8 in 2024-06-11T13:33:10.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:09 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-17", "mode": "writeback"}]: dispatch 2024-06-11T13:33:10.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:09 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3971268278' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi057-77351-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi057-77351-30"}]: dispatch 2024-06-11T13:33:10.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:09 smithi057 ceph-mon[29805]: from='client.36140 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi057-77351-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi057-77351-30"}]: dispatch 2024-06-11T13:33:10.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:09 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3971268278' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi057-77351-30"}]: dispatch 2024-06-11T13:33:10.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:09 smithi057 ceph-mon[40588]: from='client.36140 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi057-77351-30"}]: dispatch 2024-06-11T13:33:10.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:09 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3971268278' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi057-77351-30"}]: dispatch 2024-06-11T13:33:10.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:09 smithi057 ceph-mon[40588]: from='client.36140 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi057-77351-30"}]: dispatch 2024-06-11T13:33:10.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:09 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3971268278' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi057-77351-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:10.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:09 smithi057 ceph-mon[40588]: from='client.36140 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi057-77351-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:10.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:09 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-17"}]': finished 2024-06-11T13:33:10.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:09 smithi057 ceph-mon[40588]: from='client.26119 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi057-77362-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:10.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:09 smithi057 ceph-mon[40588]: from='client.36140 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi057-77351-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:33:10.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:09 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-17", "mode": "writeback"}]: dispatch 2024-06-11T13:33:10.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:09 smithi057 ceph-mon[40588]: osdmap e192: 8 total, 8 up, 8 in 2024-06-11T13:33:10.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:09 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-17", "mode": "writeback"}]: dispatch 2024-06-11T13:33:10.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:09 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3971268278' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi057-77351-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi057-77351-30"}]: dispatch 2024-06-11T13:33:10.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:09 smithi057 ceph-mon[40588]: from='client.36140 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi057-77351-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi057-77351-30"}]: dispatch 2024-06-11T13:33:10.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:09 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3971268278' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi057-77351-30"}]: dispatch 2024-06-11T13:33:10.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:09 smithi107 ceph-mon[33178]: from='client.36140 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi057-77351-30"}]: dispatch 2024-06-11T13:33:10.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:09 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3971268278' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi057-77351-30"}]: dispatch 2024-06-11T13:33:10.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:09 smithi107 ceph-mon[33178]: from='client.36140 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi057-77351-30"}]: dispatch 2024-06-11T13:33:10.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:09 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3971268278' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi057-77351-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:10.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:09 smithi107 ceph-mon[33178]: from='client.36140 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi057-77351-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:10.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:09 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-17"}]': finished 2024-06-11T13:33:10.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:09 smithi107 ceph-mon[33178]: from='client.26119 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi057-77362-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:10.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:09 smithi107 ceph-mon[33178]: from='client.36140 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi057-77351-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:33:10.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:09 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-17", "mode": "writeback"}]: dispatch 2024-06-11T13:33:10.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:09 smithi107 ceph-mon[33178]: osdmap e192: 8 total, 8 up, 8 in 2024-06-11T13:33:10.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:09 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-17", "mode": "writeback"}]: dispatch 2024-06-11T13:33:10.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:09 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3971268278' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi057-77351-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi057-77351-30"}]: dispatch 2024-06-11T13:33:10.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:09 smithi107 ceph-mon[33178]: from='client.36140 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi057-77351-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi057-77351-30"}]: dispatch 2024-06-11T13:33:11.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:11 smithi057 ceph-mon[29805]: pgmap v255: 356 pgs: 64 unknown, 292 active+clean; 584 KiB data, 554 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:33:11.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:11 smithi057 ceph-mon[29805]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:33:11.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:11 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-17", "mode": "writeback"}]': finished 2024-06-11T13:33:11.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:11 smithi057 ceph-mon[40588]: pgmap v255: 356 pgs: 64 unknown, 292 active+clean; 584 KiB data, 554 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:33:11.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:11 smithi057 ceph-mon[40588]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:33:11.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:11 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-17", "mode": "writeback"}]': finished 2024-06-11T13:33:11.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:11 smithi107 ceph-mon[33178]: pgmap v255: 356 pgs: 64 unknown, 292 active+clean; 584 KiB data, 554 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:33:11.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:11 smithi107 ceph-mon[33178]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:33:11.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:11 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-17", "mode": "writeback"}]': finished 2024-06-11T13:33:11.547 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:33:11 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:33:12.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:12 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:33:12.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:12 smithi057 ceph-mon[29805]: osdmap e193: 8 total, 8 up, 8 in 2024-06-11T13:33:12.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:12 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:33:12.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:12 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:33:12.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:12 smithi057 ceph-mon[29805]: pgmap v258: 324 pgs: 324 active+clean; 584 KiB data, 555 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-06-11T13:33:12.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:12 smithi057 ceph-mon[29805]: from='client.36140 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi057-77351-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi057-77351-30"}]': finished 2024-06-11T13:33:12.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:12 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:33:12.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:12 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-17"}]: dispatch 2024-06-11T13:33:12.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:12 smithi057 ceph-mon[29805]: osdmap e194: 8 total, 8 up, 8 in 2024-06-11T13:33:12.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:12 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-17"}]: dispatch 2024-06-11T13:33:12.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:12 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3975925264' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi057-77362-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:12.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:12 smithi057 ceph-mon[29805]: from='client.36146 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi057-77362-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:12.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:12 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:33:12.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:12 smithi057 ceph-mon[40588]: osdmap e193: 8 total, 8 up, 8 in 2024-06-11T13:33:12.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:12 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:33:12.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:12 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:33:12.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:12 smithi057 ceph-mon[40588]: pgmap v258: 324 pgs: 324 active+clean; 584 KiB data, 555 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-06-11T13:33:12.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:12 smithi057 ceph-mon[40588]: from='client.36140 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi057-77351-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi057-77351-30"}]': finished 2024-06-11T13:33:12.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:12 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:33:12.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:12 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-17"}]: dispatch 2024-06-11T13:33:12.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:12 smithi057 ceph-mon[40588]: osdmap e194: 8 total, 8 up, 8 in 2024-06-11T13:33:12.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:12 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-17"}]: dispatch 2024-06-11T13:33:12.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:12 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3975925264' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi057-77362-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:12.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:12 smithi057 ceph-mon[40588]: from='client.36146 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi057-77362-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:12.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:12 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:33:12.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:12 smithi107 ceph-mon[33178]: osdmap e193: 8 total, 8 up, 8 in 2024-06-11T13:33:12.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:12 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:33:12.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:12 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:33:12.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:12 smithi107 ceph-mon[33178]: pgmap v258: 324 pgs: 324 active+clean; 584 KiB data, 555 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-06-11T13:33:12.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:12 smithi107 ceph-mon[33178]: from='client.36140 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi057-77351-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi057-77351-30"}]': finished 2024-06-11T13:33:12.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:12 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:33:12.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:12 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-17"}]: dispatch 2024-06-11T13:33:12.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:12 smithi107 ceph-mon[33178]: osdmap e194: 8 total, 8 up, 8 in 2024-06-11T13:33:12.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:12 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-17"}]: dispatch 2024-06-11T13:33:12.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:12 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3975925264' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi057-77362-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:12.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:12 smithi107 ceph-mon[33178]: from='client.36146 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi057-77362-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:13.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:13 smithi057 ceph-mon[29805]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:33:13.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:13 smithi057 ceph-mon[29805]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:33:13.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:13 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-17"}]': finished 2024-06-11T13:33:13.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:13 smithi057 ceph-mon[29805]: from='client.36146 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi057-77362-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:13.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:13 smithi057 ceph-mon[29805]: osdmap e195: 8 total, 8 up, 8 in 2024-06-11T13:33:13.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:13 smithi057 ceph-mon[40588]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:33:13.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:13 smithi057 ceph-mon[40588]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:33:13.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:13 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-17"}]': finished 2024-06-11T13:33:13.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:13 smithi057 ceph-mon[40588]: from='client.36146 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi057-77362-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:13.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:13 smithi057 ceph-mon[40588]: osdmap e195: 8 total, 8 up, 8 in 2024-06-11T13:33:13.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:13 smithi107 ceph-mon[33178]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:33:13.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:13 smithi107 ceph-mon[33178]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:33:13.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:13 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-17"}]': finished 2024-06-11T13:33:13.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:13 smithi107 ceph-mon[33178]: from='client.36146 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi057-77362-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:13.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:13 smithi107 ceph-mon[33178]: osdmap e195: 8 total, 8 up, 8 in 2024-06-11T13:33:14.034 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:33:13 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:33:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:33:14.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:14 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3975925264' entity='client.admin' cmd=[{ 2024-06-11T13:33:14.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:14 smithi057 ceph-mon[40588]: "prefix": "osd pool set", 2024-06-11T13:33:14.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:14 smithi057 ceph-mon[40588]: "pool": "PoolEIOFlag_smithi057-77362-33", 2024-06-11T13:33:14.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:14 smithi057 ceph-mon[40588]: "var": "eio", 2024-06-11T13:33:14.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:14 smithi057 ceph-mon[40588]: "val": "true" 2024-06-11T13:33:14.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:14 smithi057 ceph-mon[40588]: }]: dispatch 2024-06-11T13:33:14.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:14 smithi057 ceph-mon[40588]: from='client.36146 ' entity='client.admin' cmd=[{ 2024-06-11T13:33:14.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:14 smithi057 ceph-mon[40588]: "prefix": "osd pool set", 2024-06-11T13:33:14.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:14 smithi057 ceph-mon[40588]: "pool": "PoolEIOFlag_smithi057-77362-33", 2024-06-11T13:33:14.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:14 smithi057 ceph-mon[40588]: "var": "eio", 2024-06-11T13:33:14.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:14 smithi057 ceph-mon[40588]: "val": "true" 2024-06-11T13:33:14.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:14 smithi057 ceph-mon[40588]: }]: dispatch 2024-06-11T13:33:14.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:14 smithi057 ceph-mon[40588]: from='client.36146 ' entity='client.admin' cmd='[{ 2024-06-11T13:33:14.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:14 smithi057 ceph-mon[40588]: "prefix": "osd pool set", 2024-06-11T13:33:14.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:14 smithi057 ceph-mon[40588]: "pool": "PoolEIOFlag_smithi057-77362-33", 2024-06-11T13:33:14.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:14 smithi057 ceph-mon[40588]: "var": "eio", 2024-06-11T13:33:14.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:14 smithi057 ceph-mon[40588]: "val": "true" 2024-06-11T13:33:14.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:14 smithi057 ceph-mon[40588]: }]': finished 2024-06-11T13:33:14.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:14 smithi057 ceph-mon[40588]: osdmap e196: 8 total, 8 up, 8 in 2024-06-11T13:33:14.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:14 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3971268278' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi057-77351-30"}]: dispatch 2024-06-11T13:33:14.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:14 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3975925264' entity='client.admin' cmd=[{ 2024-06-11T13:33:14.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:14 smithi057 ceph-mon[29805]: "prefix": "osd pool set", 2024-06-11T13:33:14.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:14 smithi057 ceph-mon[29805]: "pool": "PoolEIOFlag_smithi057-77362-33", 2024-06-11T13:33:14.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:14 smithi057 ceph-mon[29805]: "var": "eio", 2024-06-11T13:33:14.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:14 smithi057 ceph-mon[29805]: "val": "true" 2024-06-11T13:33:14.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:14 smithi057 ceph-mon[29805]: }]: dispatch 2024-06-11T13:33:14.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:14 smithi057 ceph-mon[29805]: from='client.36146 ' entity='client.admin' cmd=[{ 2024-06-11T13:33:14.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:14 smithi057 ceph-mon[29805]: "prefix": "osd pool set", 2024-06-11T13:33:14.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:14 smithi057 ceph-mon[29805]: "pool": "PoolEIOFlag_smithi057-77362-33", 2024-06-11T13:33:14.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:14 smithi057 ceph-mon[29805]: "var": "eio", 2024-06-11T13:33:14.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:14 smithi057 ceph-mon[29805]: "val": "true" 2024-06-11T13:33:14.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:14 smithi057 ceph-mon[29805]: }]: dispatch 2024-06-11T13:33:14.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:14 smithi057 ceph-mon[29805]: from='client.36146 ' entity='client.admin' cmd='[{ 2024-06-11T13:33:14.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:14 smithi057 ceph-mon[29805]: "prefix": "osd pool set", 2024-06-11T13:33:14.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:14 smithi057 ceph-mon[29805]: "pool": "PoolEIOFlag_smithi057-77362-33", 2024-06-11T13:33:14.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:14 smithi057 ceph-mon[29805]: "var": "eio", 2024-06-11T13:33:14.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:14 smithi057 ceph-mon[29805]: "val": "true" 2024-06-11T13:33:14.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:14 smithi057 ceph-mon[29805]: }]': finished 2024-06-11T13:33:14.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:14 smithi057 ceph-mon[29805]: osdmap e196: 8 total, 8 up, 8 in 2024-06-11T13:33:14.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:14 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3971268278' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi057-77351-30"}]: dispatch 2024-06-11T13:33:14.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:14 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3975925264' entity='client.admin' cmd=[{ 2024-06-11T13:33:14.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:14 smithi107 ceph-mon[33178]: "prefix": "osd pool set", 2024-06-11T13:33:14.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:14 smithi107 ceph-mon[33178]: "pool": "PoolEIOFlag_smithi057-77362-33", 2024-06-11T13:33:14.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:14 smithi107 ceph-mon[33178]: "var": "eio", 2024-06-11T13:33:14.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:14 smithi107 ceph-mon[33178]: "val": "true" 2024-06-11T13:33:14.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:14 smithi107 ceph-mon[33178]: }]: dispatch 2024-06-11T13:33:14.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:14 smithi107 ceph-mon[33178]: from='client.36146 ' entity='client.admin' cmd=[{ 2024-06-11T13:33:14.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:14 smithi107 ceph-mon[33178]: "prefix": "osd pool set", 2024-06-11T13:33:14.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:14 smithi107 ceph-mon[33178]: "pool": "PoolEIOFlag_smithi057-77362-33", 2024-06-11T13:33:14.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:14 smithi107 ceph-mon[33178]: "var": "eio", 2024-06-11T13:33:14.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:14 smithi107 ceph-mon[33178]: "val": "true" 2024-06-11T13:33:14.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:14 smithi107 ceph-mon[33178]: }]: dispatch 2024-06-11T13:33:14.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:14 smithi107 ceph-mon[33178]: from='client.36146 ' entity='client.admin' cmd='[{ 2024-06-11T13:33:14.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:14 smithi107 ceph-mon[33178]: "prefix": "osd pool set", 2024-06-11T13:33:14.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:14 smithi107 ceph-mon[33178]: "pool": "PoolEIOFlag_smithi057-77362-33", 2024-06-11T13:33:14.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:14 smithi107 ceph-mon[33178]: "var": "eio", 2024-06-11T13:33:14.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:14 smithi107 ceph-mon[33178]: "val": "true" 2024-06-11T13:33:14.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:14 smithi107 ceph-mon[33178]: }]': finished 2024-06-11T13:33:14.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:14 smithi107 ceph-mon[33178]: osdmap e196: 8 total, 8 up, 8 in 2024-06-11T13:33:14.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:14 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3971268278' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi057-77351-30"}]: dispatch 2024-06-11T13:33:15.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:15 smithi057 ceph-mon[40588]: pgmap v261: 364 pgs: 40 unknown, 324 active+clean; 584 KiB data, 555 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-06-11T13:33:15.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:15 smithi057 ceph-mon[40588]: from='client.36140 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi057-77351-30"}]: dispatch 2024-06-11T13:33:15.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:15 smithi057 ceph-mon[40588]: from='client.36140 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi057-77351-30"}]': finished 2024-06-11T13:33:15.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:15 smithi057 ceph-mon[40588]: osdmap e197: 8 total, 8 up, 8 in 2024-06-11T13:33:15.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:15 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:15.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:15 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:15.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:15 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3971268278' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi057-77351-30"}]: dispatch 2024-06-11T13:33:15.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:15 smithi057 ceph-mon[40588]: from='client.36140 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi057-77351-30"}]: dispatch 2024-06-11T13:33:15.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:15 smithi057 ceph-mon[29805]: pgmap v261: 364 pgs: 40 unknown, 324 active+clean; 584 KiB data, 555 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-06-11T13:33:15.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:15 smithi057 ceph-mon[29805]: from='client.36140 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi057-77351-30"}]: dispatch 2024-06-11T13:33:15.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:15 smithi057 ceph-mon[29805]: from='client.36140 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi057-77351-30"}]': finished 2024-06-11T13:33:15.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:15 smithi057 ceph-mon[29805]: osdmap e197: 8 total, 8 up, 8 in 2024-06-11T13:33:15.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:15 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:15.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:15 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:15.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:15 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3971268278' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi057-77351-30"}]: dispatch 2024-06-11T13:33:15.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:15 smithi057 ceph-mon[29805]: from='client.36140 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi057-77351-30"}]: dispatch 2024-06-11T13:33:15.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:15 smithi107 ceph-mon[33178]: pgmap v261: 364 pgs: 40 unknown, 324 active+clean; 584 KiB data, 555 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-06-11T13:33:15.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:15 smithi107 ceph-mon[33178]: from='client.36140 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi057-77351-30"}]: dispatch 2024-06-11T13:33:15.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:15 smithi107 ceph-mon[33178]: from='client.36140 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi057-77351-30"}]': finished 2024-06-11T13:33:15.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:15 smithi107 ceph-mon[33178]: osdmap e197: 8 total, 8 up, 8 in 2024-06-11T13:33:15.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:15 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:15.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:15 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:15.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:15 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3971268278' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi057-77351-30"}]: dispatch 2024-06-11T13:33:15.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:15 smithi107 ceph-mon[33178]: from='client.36140 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi057-77351-30"}]: dispatch 2024-06-11T13:33:15.741 INFO:tasks.workunit.client.0.smithi057.stdout: 0x564078a6ebd0 2024-06-11T13:33:15.741 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6f080 2024-06-11T13:33:15.741 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6ece0 2024-06-11T13:33:15.741 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6edf0 2024-06-11T13:33:15.741 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6ef40 2024-06-11T13:33:15.741 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6f660 2024-06-11T13:33:15.741 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6f1b0 2024-06-11T13:33:15.741 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6d790 2024-06-11T13:33:15.741 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6d8e0 2024-06-11T13:33:15.741 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6f770 2024-06-11T13:33:15.741 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6fc20 2024-06-11T13:33:15.742 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6fd30 2024-06-11T13:33:15.742 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a70310 2024-06-11T13:33:15.742 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6f880 2024-06-11T13:33:15.742 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6f9d0 2024-06-11T13:33:15.742 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6fb00 2024-06-11T13:33:15.742 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a70440 2024-06-11T13:33:15.742 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6fe60 2024-06-11T13:33:15.742 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6ff90 2024-06-11T13:33:15.742 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a700e0 2024-06-11T13:33:15.742 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a701f0 2024-06-11T13:33:15.742 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a70a40 2024-06-11T13:33:15.742 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a70590 2024-06-11T13:33:15.742 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a706e0 2024-06-11T13:33:15.743 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a707f0 2024-06-11T13:33:15.743 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a70b50 2024-06-11T13:33:15.743 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a71020 2024-06-11T13:33:15.743 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a71150 2024-06-11T13:33:15.743 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a70920 2024-06-11T13:33:15.743 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a71260 2024-06-11T13:33:15.743 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6f2c0 2024-06-11T13:33:15.743 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6f410 2024-06-11T13:33:15.743 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a6f520 2024-06-11T13:33:15.743 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a71860 2024-06-11T13:33:15.743 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a713b0 2024-06-11T13:33:15.743 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a70c80 2024-06-11T13:33:15.743 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a70dd0 2024-06-11T13:33:15.743 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a70ee0 2024-06-11T13:33:15.744 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a71e40 2024-06-11T13:33:15.744 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a71990 2024-06-11T13:33:15.744 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a71ac0 2024-06-11T13:33:15.744 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a71c10 2024-06-11T13:33:15.744 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a71f50 2024-06-11T13:33:15.744 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a72440 2024-06-11T13:33:15.744 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a72550 2024-06-11T13:33:15.744 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a714c0 2024-06-11T13:33:15.744 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a715f0 2024-06-11T13:33:15.744 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a71720 2024-06-11T13:33:15.744 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a72680 2024-06-11T13:33:15.745 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a72b50 2024-06-11T13:33:15.745 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a72cc0 2024-06-11T13:33:15.745 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a732c0 2024-06-11T13:33:15.745 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a727b0 2024-06-11T13:33:15.745 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a72900 2024-06-11T13:33:15.745 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a72a30 2024-06-11T13:33:15.745 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a73410 2024-06-11T13:33:15.745 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a72e10 2024-06-11T13:33:15.745 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a72f60 2024-06-11T13:33:15.746 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a73070 2024-06-11T13:33:15.746 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a731a0 2024-06-11T13:33:15.746 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a739f0 2024-06-11T13:33:15.746 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a73540 2024-06-11T13:33:15.746 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a73690 2024-06-11T13:33:15.746 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a71d20 2024-06-11T13:33:15.746 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a737e0 2024-06-11T13:33:15.746 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a73fd0 2024-06-11T13:33:15.746 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a73b20 2024-06-11T13:33:15.746 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a73c70 2024-06-11T13:33:15.746 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a73dc0 2024-06-11T13:33:15.747 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a74140 2024-06-11T13:33:15.747 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a74660 2024-06-11T13:33:15.747 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a74790 2024-06-11T13:33:15.747 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a74270 2024-06-11T13:33:15.747 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a74380 2024-06-11T13:33:15.747 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a74490 2024-06-11T13:33:15.747 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a748e0 2024-06-11T13:33:15.747 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a74ae0 2024-06-11T13:33:15.747 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a750c0 2024-06-11T13:33:15.747 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a74c10 2024-06-11T13:33:15.747 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a74d60 2024-06-11T13:33:15.747 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a74e70 2024-06-11T13:33:15.747 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a751d0 2024-06-11T13:33:17.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:16 smithi057 ceph-mon[40588]: pgmap v264: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 555 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:33:17.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:16 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:17.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:16 smithi057 ceph-mon[40588]: from='client.36140 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi057-77351-30"}]': finished 2024-06-11T13:33:17.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:16 smithi057 ceph-mon[40588]: osdmap e198: 8 total, 8 up, 8 in 2024-06-11T13:33:17.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:16 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2254557722' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi057-77362-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:17.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:16 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:33:17.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:16 smithi057 ceph-mon[40588]: from='client.26137 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi057-77362-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:17.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:16 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:33:17.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:16 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1471229600' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi057-77351-31"}]: dispatch 2024-06-11T13:33:17.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:16 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1471229600' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi057-77351-31"}]: dispatch 2024-06-11T13:33:17.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:16 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1471229600' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi057-77351-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:17.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:16 smithi057 ceph-mon[29805]: pgmap v264: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 555 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:33:17.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:16 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:17.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:16 smithi057 ceph-mon[29805]: from='client.36140 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi057-77351-30"}]': finished 2024-06-11T13:33:17.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:16 smithi057 ceph-mon[29805]: osdmap e198: 8 total, 8 up, 8 in 2024-06-11T13:33:17.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:16 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2254557722' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi057-77362-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:17.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:16 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:33:17.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:16 smithi057 ceph-mon[29805]: from='client.26137 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi057-77362-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:17.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:16 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:33:17.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:16 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1471229600' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi057-77351-31"}]: dispatch 2024-06-11T13:33:17.038 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:16 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1471229600' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi057-77351-31"}]: dispatch 2024-06-11T13:33:17.038 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:16 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1471229600' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi057-77351-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:17.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:16 smithi107 ceph-mon[33178]: pgmap v264: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 555 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:33:17.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:16 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:17.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:16 smithi107 ceph-mon[33178]: from='client.36140 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi057-77351-30"}]': finished 2024-06-11T13:33:17.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:16 smithi107 ceph-mon[33178]: osdmap e198: 8 total, 8 up, 8 in 2024-06-11T13:33:17.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:16 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2254557722' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi057-77362-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:17.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:16 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:33:17.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:16 smithi107 ceph-mon[33178]: from='client.26137 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi057-77362-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:17.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:16 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:33:17.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:16 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1471229600' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi057-77351-31"}]: dispatch 2024-06-11T13:33:17.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:16 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1471229600' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi057-77351-31"}]: dispatch 2024-06-11T13:33:17.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:16 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1471229600' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi057-77351-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:18.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:18 smithi057 ceph-mon[40588]: from='client.26137 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi057-77362-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:18.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:18 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-19", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:33:18.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:18 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1471229600' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi057-77351-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:33:18.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:18 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-19"}]: dispatch 2024-06-11T13:33:18.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:18 smithi057 ceph-mon[40588]: osdmap e199: 8 total, 8 up, 8 in 2024-06-11T13:33:18.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:18 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1471229600' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi057-77351-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi057-77351-31"}]: dispatch 2024-06-11T13:33:18.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:18 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-19"}]: dispatch 2024-06-11T13:33:18.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:18 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-19"}]': finished 2024-06-11T13:33:18.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:18 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-19", "mode": "writeback"}]: dispatch 2024-06-11T13:33:18.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:18 smithi057 ceph-mon[40588]: osdmap e200: 8 total, 8 up, 8 in 2024-06-11T13:33:18.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:18 smithi057 ceph-mon[29805]: from='client.26137 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi057-77362-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:18.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:18 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-19", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:33:18.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:18 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1471229600' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi057-77351-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:33:18.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:18 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-19"}]: dispatch 2024-06-11T13:33:18.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:18 smithi057 ceph-mon[29805]: osdmap e199: 8 total, 8 up, 8 in 2024-06-11T13:33:18.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:18 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1471229600' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi057-77351-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi057-77351-31"}]: dispatch 2024-06-11T13:33:18.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:18 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-19"}]: dispatch 2024-06-11T13:33:18.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:18 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-19"}]': finished 2024-06-11T13:33:18.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:18 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-19", "mode": "writeback"}]: dispatch 2024-06-11T13:33:18.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:18 smithi057 ceph-mon[29805]: osdmap e200: 8 total, 8 up, 8 in 2024-06-11T13:33:18.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:18 smithi107 ceph-mon[33178]: from='client.26137 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi057-77362-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:18.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:18 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-19", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:33:18.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:18 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1471229600' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi057-77351-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:33:18.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:18 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-19"}]: dispatch 2024-06-11T13:33:18.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:18 smithi107 ceph-mon[33178]: osdmap e199: 8 total, 8 up, 8 in 2024-06-11T13:33:18.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:18 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1471229600' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi057-77351-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi057-77351-31"}]: dispatch 2024-06-11T13:33:18.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:18 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-19"}]: dispatch 2024-06-11T13:33:18.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:18 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-19"}]': finished 2024-06-11T13:33:18.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:18 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-19", "mode": "writeback"}]: dispatch 2024-06-11T13:33:18.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:18 smithi107 ceph-mon[33178]: osdmap e200: 8 total, 8 up, 8 in 2024-06-11T13:33:19.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:19 smithi057 ceph-mon[29805]: pgmap v267: 356 pgs: 32 creating+peering, 324 active+clean; 584 KiB data, 563 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T13:33:19.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:19 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-19", "mode": "writeback"}]: dispatch 2024-06-11T13:33:19.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:19 smithi057 ceph-mon[29805]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:33:19.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:19 smithi057 ceph-mon[29805]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:33:19.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:19 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1471229600' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi057-77351-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi057-77351-31"}]': finished 2024-06-11T13:33:19.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:19 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-19", "mode": "writeback"}]': finished 2024-06-11T13:33:19.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:19 smithi057 ceph-mon[29805]: osdmap e201: 8 total, 8 up, 8 in 2024-06-11T13:33:19.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:19 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3376312451' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi057-77362-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:19.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:19 smithi057 ceph-mon[29805]: from='client.36164 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi057-77362-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:19.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:19 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:33:19.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:19 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:33:19.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:19 smithi057 ceph-mon[40588]: pgmap v267: 356 pgs: 32 creating+peering, 324 active+clean; 584 KiB data, 563 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T13:33:19.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:19 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-19", "mode": "writeback"}]: dispatch 2024-06-11T13:33:19.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:19 smithi057 ceph-mon[40588]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:33:19.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:19 smithi057 ceph-mon[40588]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:33:19.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:19 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1471229600' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi057-77351-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi057-77351-31"}]': finished 2024-06-11T13:33:19.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:19 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-19", "mode": "writeback"}]': finished 2024-06-11T13:33:19.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:19 smithi057 ceph-mon[40588]: osdmap e201: 8 total, 8 up, 8 in 2024-06-11T13:33:19.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:19 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3376312451' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi057-77362-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:19.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:19 smithi057 ceph-mon[40588]: from='client.36164 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi057-77362-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:19.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:19 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:33:19.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:19 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:33:19.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:19 smithi107 ceph-mon[33178]: pgmap v267: 356 pgs: 32 creating+peering, 324 active+clean; 584 KiB data, 563 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T13:33:19.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:19 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-19", "mode": "writeback"}]: dispatch 2024-06-11T13:33:19.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:19 smithi107 ceph-mon[33178]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:33:19.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:19 smithi107 ceph-mon[33178]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:33:19.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:19 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1471229600' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi057-77351-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi057-77351-31"}]': finished 2024-06-11T13:33:19.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:19 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-19", "mode": "writeback"}]': finished 2024-06-11T13:33:19.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:19 smithi107 ceph-mon[33178]: osdmap e201: 8 total, 8 up, 8 in 2024-06-11T13:33:19.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:19 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3376312451' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi057-77362-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:19.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:19 smithi107 ceph-mon[33178]: from='client.36164 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi057-77362-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:19.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:19 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:33:19.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:19 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:33:21.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:20 smithi057 ceph-mon[40588]: pgmap v270: 364 pgs: 40 unknown, 324 active+clean; 584 KiB data, 563 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:33:21.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:20 smithi057 ceph-mon[40588]: from='client.36164 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi057-77362-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:21.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:20 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:33:21.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:20 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-19"}]: dispatch 2024-06-11T13:33:21.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:20 smithi057 ceph-mon[40588]: osdmap e202: 8 total, 8 up, 8 in 2024-06-11T13:33:21.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:20 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-19"}]: dispatch 2024-06-11T13:33:21.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:20 smithi057 ceph-mon[29805]: pgmap v270: 364 pgs: 40 unknown, 324 active+clean; 584 KiB data, 563 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:33:21.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:20 smithi057 ceph-mon[29805]: from='client.36164 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi057-77362-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:21.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:20 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:33:21.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:20 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-19"}]: dispatch 2024-06-11T13:33:21.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:20 smithi057 ceph-mon[29805]: osdmap e202: 8 total, 8 up, 8 in 2024-06-11T13:33:21.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:20 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-19"}]: dispatch 2024-06-11T13:33:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:20 smithi107 ceph-mon[33178]: pgmap v270: 364 pgs: 40 unknown, 324 active+clean; 584 KiB data, 563 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:33:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:20 smithi107 ceph-mon[33178]: from='client.36164 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi057-77362-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:20 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:33:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:20 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-19"}]: dispatch 2024-06-11T13:33:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:20 smithi107 ceph-mon[33178]: osdmap e202: 8 total, 8 up, 8 in 2024-06-11T13:33:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:20 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-19"}]: dispatch 2024-06-11T13:33:21.547 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:33:21 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:33:22.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:22 smithi057 ceph-mon[40588]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:33:22.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:22 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-19"}]': finished 2024-06-11T13:33:22.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:22 smithi057 ceph-mon[40588]: osdmap e203: 8 total, 8 up, 8 in 2024-06-11T13:33:22.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:22 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1471229600' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi057-77351-31"}]: dispatch 2024-06-11T13:33:22.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:22 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:33:22.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:22 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1471229600' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi057-77351-31"}]': finished 2024-06-11T13:33:22.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:22 smithi057 ceph-mon[29805]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:33:22.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:22 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-19"}]': finished 2024-06-11T13:33:22.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:22 smithi057 ceph-mon[29805]: osdmap e203: 8 total, 8 up, 8 in 2024-06-11T13:33:22.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:22 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1471229600' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi057-77351-31"}]: dispatch 2024-06-11T13:33:22.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:22 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:33:22.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:22 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1471229600' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi057-77351-31"}]': finished 2024-06-11T13:33:22.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:22 smithi107 ceph-mon[33178]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:33:22.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:22 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-19"}]': finished 2024-06-11T13:33:22.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:22 smithi107 ceph-mon[33178]: osdmap e203: 8 total, 8 up, 8 in 2024-06-11T13:33:22.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:22 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1471229600' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi057-77351-31"}]: dispatch 2024-06-11T13:33:22.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:22 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:33:22.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:22 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1471229600' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi057-77351-31"}]': finished 2024-06-11T13:33:23.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:23 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:33:23.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:23 smithi057 ceph-mon[40588]: pgmap v273: 324 pgs: 324 active+clean; 584 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-06-11T13:33:23.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:23 smithi057 ceph-mon[40588]: osdmap e204: 8 total, 8 up, 8 in 2024-06-11T13:33:23.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:23 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1471229600' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi057-77351-31"}]: dispatch 2024-06-11T13:33:23.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:23 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1399749962' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi057-77362-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:23.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:23 smithi057 ceph-mon[40588]: from='client.36170 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi057-77362-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:23.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:23 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1471229600' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi057-77351-31"}]': finished 2024-06-11T13:33:23.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:23 smithi057 ceph-mon[40588]: from='client.36170 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi057-77362-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:23.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:23 smithi057 ceph-mon[40588]: osdmap e205: 8 total, 8 up, 8 in 2024-06-11T13:33:23.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:23 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:23.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:23 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:23.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:23 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3137310988' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi057-77351-32"}]: dispatch 2024-06-11T13:33:23.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:23 smithi057 ceph-mon[40588]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi057-77351-32"}]: dispatch 2024-06-11T13:33:23.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:23 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3137310988' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi057-77351-32"}]: dispatch 2024-06-11T13:33:23.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:23 smithi057 ceph-mon[40588]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi057-77351-32"}]: dispatch 2024-06-11T13:33:23.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:23 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3137310988' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi057-77351-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:23.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:23 smithi057 ceph-mon[40588]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi057-77351-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:23.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:23 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:33:23.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:23 smithi057 ceph-mon[29805]: pgmap v273: 324 pgs: 324 active+clean; 584 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-06-11T13:33:23.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:23 smithi057 ceph-mon[29805]: osdmap e204: 8 total, 8 up, 8 in 2024-06-11T13:33:23.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:23 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1471229600' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi057-77351-31"}]: dispatch 2024-06-11T13:33:23.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:23 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1399749962' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi057-77362-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:23.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:23 smithi057 ceph-mon[29805]: from='client.36170 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi057-77362-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:23.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:23 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1471229600' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi057-77351-31"}]': finished 2024-06-11T13:33:23.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:23 smithi057 ceph-mon[29805]: from='client.36170 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi057-77362-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:23.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:23 smithi057 ceph-mon[29805]: osdmap e205: 8 total, 8 up, 8 in 2024-06-11T13:33:23.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:23 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:23.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:23 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:23.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:23 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3137310988' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi057-77351-32"}]: dispatch 2024-06-11T13:33:23.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:23 smithi057 ceph-mon[29805]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi057-77351-32"}]: dispatch 2024-06-11T13:33:23.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:23 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3137310988' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi057-77351-32"}]: dispatch 2024-06-11T13:33:23.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:23 smithi057 ceph-mon[29805]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi057-77351-32"}]: dispatch 2024-06-11T13:33:23.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:23 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3137310988' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi057-77351-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:23.538 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:23 smithi057 ceph-mon[29805]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi057-77351-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:23.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:23 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:33:23.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:23 smithi107 ceph-mon[33178]: pgmap v273: 324 pgs: 324 active+clean; 584 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-06-11T13:33:23.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:23 smithi107 ceph-mon[33178]: osdmap e204: 8 total, 8 up, 8 in 2024-06-11T13:33:23.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:23 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1471229600' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi057-77351-31"}]: dispatch 2024-06-11T13:33:23.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:23 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1399749962' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi057-77362-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:23.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:23 smithi107 ceph-mon[33178]: from='client.36170 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi057-77362-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:23.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:23 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1471229600' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi057-77351-31"}]': finished 2024-06-11T13:33:23.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:23 smithi107 ceph-mon[33178]: from='client.36170 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi057-77362-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:23.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:23 smithi107 ceph-mon[33178]: osdmap e205: 8 total, 8 up, 8 in 2024-06-11T13:33:23.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:23 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:23.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:23 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:23.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:23 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3137310988' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi057-77351-32"}]: dispatch 2024-06-11T13:33:23.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:23 smithi107 ceph-mon[33178]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi057-77351-32"}]: dispatch 2024-06-11T13:33:23.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:23 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3137310988' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi057-77351-32"}]: dispatch 2024-06-11T13:33:23.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:23 smithi107 ceph-mon[33178]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi057-77351-32"}]: dispatch 2024-06-11T13:33:23.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:23 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3137310988' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi057-77351-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:23.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:23 smithi107 ceph-mon[33178]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi057-77351-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:24.034 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:33:23 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:33:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:33:25.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:25 smithi057 ceph-mon[29805]: pgmap v276: 356 pgs: 64 unknown, 292 active+clean; 584 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:33:25.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:25 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:25.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:25 smithi057 ceph-mon[29805]: from='client.36176 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi057-77351-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:33:25.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:25 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:33:25.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:25 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3137310988' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi057-77351-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi057-77351-32"}]: dispatch 2024-06-11T13:33:25.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:25 smithi057 ceph-mon[29805]: osdmap e206: 8 total, 8 up, 8 in 2024-06-11T13:33:25.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:25 smithi057 ceph-mon[29805]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi057-77351-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi057-77351-32"}]: dispatch 2024-06-11T13:33:25.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:25 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3974826981' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi057-77362-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:25.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:25 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:33:25.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:25 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:33:25.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:25 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:33:25.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:25 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:33:25.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:25 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:33:25.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:25 smithi057 ceph-mon[29805]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:33:25.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:25 smithi057 ceph-mon[40588]: pgmap v276: 356 pgs: 64 unknown, 292 active+clean; 584 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:33:25.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:25 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:25.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:25 smithi057 ceph-mon[40588]: from='client.36176 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi057-77351-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:33:25.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:25 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:33:25.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:25 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3137310988' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi057-77351-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi057-77351-32"}]: dispatch 2024-06-11T13:33:25.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:25 smithi057 ceph-mon[40588]: osdmap e206: 8 total, 8 up, 8 in 2024-06-11T13:33:25.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:25 smithi057 ceph-mon[40588]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi057-77351-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi057-77351-32"}]: dispatch 2024-06-11T13:33:25.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:25 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3974826981' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi057-77362-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:25.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:25 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:33:25.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:25 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:33:25.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:25 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:33:25.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:25 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:33:25.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:25 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:33:25.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:25 smithi057 ceph-mon[40588]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:33:25.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:25 smithi107 ceph-mon[33178]: pgmap v276: 356 pgs: 64 unknown, 292 active+clean; 584 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:33:25.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:25 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:25.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:25 smithi107 ceph-mon[33178]: from='client.36176 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi057-77351-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:33:25.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:25 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:33:25.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:25 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3137310988' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi057-77351-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi057-77351-32"}]: dispatch 2024-06-11T13:33:25.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:25 smithi107 ceph-mon[33178]: osdmap e206: 8 total, 8 up, 8 in 2024-06-11T13:33:25.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:25 smithi107 ceph-mon[33178]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi057-77351-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi057-77351-32"}]: dispatch 2024-06-11T13:33:25.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:25 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3974826981' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi057-77362-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:25.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:25 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:33:25.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:25 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:33:25.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:25 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:33:25.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:25 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:33:25.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:25 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:33:25.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:25 smithi107 ceph-mon[33178]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:33:26.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:26 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3974826981' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi057-77362-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:26.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:26 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-21", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:33:26.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:26 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-21"}]: dispatch 2024-06-11T13:33:26.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:26 smithi057 ceph-mon[40588]: osdmap e207: 8 total, 8 up, 8 in 2024-06-11T13:33:26.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:26 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-21"}]: dispatch 2024-06-11T13:33:26.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:26 smithi057 ceph-mon[40588]: pgmap v279: 388 pgs: 96 unknown, 292 active+clean; 584 KiB data, 564 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:33:26.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:26 smithi057 ceph-mon[40588]: from='client.36176 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi057-77351-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi057-77351-32"}]': finished 2024-06-11T13:33:26.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:26 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-21"}]': finished 2024-06-11T13:33:26.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:26 smithi057 ceph-mon[40588]: osdmap e208: 8 total, 8 up, 8 in 2024-06-11T13:33:26.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:26 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-21", "mode": "writeback"}]: dispatch 2024-06-11T13:33:26.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:26 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-21", "mode": "writeback"}]: dispatch 2024-06-11T13:33:26.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:26 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2566853966' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi057-77362-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:26.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:26 smithi057 ceph-mon[40588]: from='client.36188 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi057-77362-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:26.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:26 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3974826981' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi057-77362-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:26.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:26 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-21", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:33:26.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:26 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-21"}]: dispatch 2024-06-11T13:33:26.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:26 smithi057 ceph-mon[29805]: osdmap e207: 8 total, 8 up, 8 in 2024-06-11T13:33:26.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:26 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-21"}]: dispatch 2024-06-11T13:33:26.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:26 smithi057 ceph-mon[29805]: pgmap v279: 388 pgs: 96 unknown, 292 active+clean; 584 KiB data, 564 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:33:26.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:26 smithi057 ceph-mon[29805]: from='client.36176 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi057-77351-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi057-77351-32"}]': finished 2024-06-11T13:33:26.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:26 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-21"}]': finished 2024-06-11T13:33:26.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:26 smithi057 ceph-mon[29805]: osdmap e208: 8 total, 8 up, 8 in 2024-06-11T13:33:26.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:26 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-21", "mode": "writeback"}]: dispatch 2024-06-11T13:33:26.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:26 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-21", "mode": "writeback"}]: dispatch 2024-06-11T13:33:26.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:26 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2566853966' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi057-77362-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:26.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:26 smithi057 ceph-mon[29805]: from='client.36188 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi057-77362-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:26.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:26 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3974826981' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi057-77362-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:26.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:26 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-21", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:33:26.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:26 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-21"}]: dispatch 2024-06-11T13:33:26.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:26 smithi107 ceph-mon[33178]: osdmap e207: 8 total, 8 up, 8 in 2024-06-11T13:33:26.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:26 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-21"}]: dispatch 2024-06-11T13:33:26.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:26 smithi107 ceph-mon[33178]: pgmap v279: 388 pgs: 96 unknown, 292 active+clean; 584 KiB data, 564 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:33:26.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:26 smithi107 ceph-mon[33178]: from='client.36176 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi057-77351-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi057-77351-32"}]': finished 2024-06-11T13:33:26.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:26 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-21"}]': finished 2024-06-11T13:33:26.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:26 smithi107 ceph-mon[33178]: osdmap e208: 8 total, 8 up, 8 in 2024-06-11T13:33:26.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:26 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-21", "mode": "writeback"}]: dispatch 2024-06-11T13:33:26.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:26 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-21", "mode": "writeback"}]: dispatch 2024-06-11T13:33:26.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:26 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2566853966' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi057-77362-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:26.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:26 smithi107 ceph-mon[33178]: from='client.36188 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi057-77362-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:27.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:27 smithi057 ceph-mon[29805]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:33:27.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:27 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-21", "mode": "writeback"}]': finished 2024-06-11T13:33:27.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:27 smithi057 ceph-mon[29805]: from='client.36188 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi057-77362-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:27.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:27 smithi057 ceph-mon[29805]: osdmap e209: 8 total, 8 up, 8 in 2024-06-11T13:33:27.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:27 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:33:27.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:27 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:33:27.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:27 smithi057 ceph-mon[40588]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:33:27.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:27 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-21", "mode": "writeback"}]': finished 2024-06-11T13:33:27.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:27 smithi057 ceph-mon[40588]: from='client.36188 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi057-77362-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:27.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:27 smithi057 ceph-mon[40588]: osdmap e209: 8 total, 8 up, 8 in 2024-06-11T13:33:27.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:27 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:33:27.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:27 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:33:27.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:27 smithi107 ceph-mon[33178]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:33:27.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:27 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-21", "mode": "writeback"}]': finished 2024-06-11T13:33:27.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:27 smithi107 ceph-mon[33178]: from='client.36188 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi057-77362-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:27.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:27 smithi107 ceph-mon[33178]: osdmap e209: 8 total, 8 up, 8 in 2024-06-11T13:33:27.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:27 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:33:27.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:27 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:33:29.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:28 smithi057 ceph-mon[29805]: pgmap v282: 428 pgs: 40 creating+peering, 388 active+clean; 584 KiB data, 573 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-11T13:33:29.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:28 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:33:29.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:28 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-21"}]: dispatch 2024-06-11T13:33:29.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:28 smithi057 ceph-mon[29805]: osdmap e210: 8 total, 8 up, 8 in 2024-06-11T13:33:29.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:28 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-21"}]: dispatch 2024-06-11T13:33:29.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:28 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3137310988' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi057-77351-32"}]: dispatch 2024-06-11T13:33:29.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:28 smithi057 ceph-mon[29805]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi057-77351-32"}]: dispatch 2024-06-11T13:33:29.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:28 smithi057 ceph-mon[40588]: pgmap v282: 428 pgs: 40 creating+peering, 388 active+clean; 584 KiB data, 573 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-11T13:33:29.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:28 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:33:29.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:28 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-21"}]: dispatch 2024-06-11T13:33:29.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:28 smithi057 ceph-mon[40588]: osdmap e210: 8 total, 8 up, 8 in 2024-06-11T13:33:29.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:28 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-21"}]: dispatch 2024-06-11T13:33:29.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:28 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3137310988' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi057-77351-32"}]: dispatch 2024-06-11T13:33:29.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:28 smithi057 ceph-mon[40588]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi057-77351-32"}]: dispatch 2024-06-11T13:33:29.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:28 smithi107 ceph-mon[33178]: pgmap v282: 428 pgs: 40 creating+peering, 388 active+clean; 584 KiB data, 573 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-11T13:33:29.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:28 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:33:29.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:28 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-21"}]: dispatch 2024-06-11T13:33:29.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:28 smithi107 ceph-mon[33178]: osdmap e210: 8 total, 8 up, 8 in 2024-06-11T13:33:29.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:28 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-21"}]: dispatch 2024-06-11T13:33:29.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:28 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3137310988' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi057-77351-32"}]: dispatch 2024-06-11T13:33:29.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:28 smithi107 ceph-mon[33178]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi057-77351-32"}]: dispatch 2024-06-11T13:33:30.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:29 smithi107 ceph-mon[33178]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:33:30.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:29 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-21"}]': finished 2024-06-11T13:33:30.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:29 smithi107 ceph-mon[33178]: from='client.36176 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi057-77351-32"}]': finished 2024-06-11T13:33:30.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:29 smithi107 ceph-mon[33178]: osdmap e211: 8 total, 8 up, 8 in 2024-06-11T13:33:30.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:29 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3137310988' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi057-77351-32"}]: dispatch 2024-06-11T13:33:30.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:29 smithi107 ceph-mon[33178]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi057-77351-32"}]: dispatch 2024-06-11T13:33:30.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:29 smithi057 ceph-mon[29805]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:33:30.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:29 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-21"}]': finished 2024-06-11T13:33:30.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:29 smithi057 ceph-mon[29805]: from='client.36176 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi057-77351-32"}]': finished 2024-06-11T13:33:30.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:29 smithi057 ceph-mon[29805]: osdmap e211: 8 total, 8 up, 8 in 2024-06-11T13:33:30.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:29 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3137310988' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi057-77351-32"}]: dispatch 2024-06-11T13:33:30.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:29 smithi057 ceph-mon[29805]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi057-77351-32"}]: dispatch 2024-06-11T13:33:30.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:29 smithi057 ceph-mon[40588]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:33:30.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:29 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-21"}]': finished 2024-06-11T13:33:30.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:29 smithi057 ceph-mon[40588]: from='client.36176 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi057-77351-32"}]': finished 2024-06-11T13:33:30.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:29 smithi057 ceph-mon[40588]: osdmap e211: 8 total, 8 up, 8 in 2024-06-11T13:33:30.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:29 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3137310988' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi057-77351-32"}]: dispatch 2024-06-11T13:33:30.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:29 smithi057 ceph-mon[40588]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi057-77351-32"}]: dispatch 2024-06-11T13:33:31.237 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:30 smithi107 ceph-mon[33178]: pgmap v285: 356 pgs: 356 active+clean; 584 KiB data, 573 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-11T13:33:31.237 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:30 smithi107 ceph-mon[33178]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:33:31.237 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:30 smithi107 ceph-mon[33178]: from='client.36176 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi057-77351-32"}]': finished 2024-06-11T13:33:31.237 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:30 smithi107 ceph-mon[33178]: osdmap e212: 8 total, 8 up, 8 in 2024-06-11T13:33:31.237 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:30 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3763978023' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi057-77351-33"}]: dispatch 2024-06-11T13:33:31.237 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:30 smithi107 ceph-mon[33178]: from='client.26182 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi057-77351-33"}]: dispatch 2024-06-11T13:33:31.237 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:30 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3763978023' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi057-77351-33"}]: dispatch 2024-06-11T13:33:31.238 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:30 smithi107 ceph-mon[33178]: from='client.26182 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi057-77351-33"}]: dispatch 2024-06-11T13:33:31.238 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:30 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3763978023' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi057-77351-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:31.238 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:30 smithi107 ceph-mon[33178]: from='client.26182 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi057-77351-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:31.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:30 smithi057 ceph-mon[29805]: pgmap v285: 356 pgs: 356 active+clean; 584 KiB data, 573 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-11T13:33:31.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:30 smithi057 ceph-mon[29805]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:33:31.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:30 smithi057 ceph-mon[29805]: from='client.36176 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi057-77351-32"}]': finished 2024-06-11T13:33:31.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:30 smithi057 ceph-mon[29805]: osdmap e212: 8 total, 8 up, 8 in 2024-06-11T13:33:31.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:30 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3763978023' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi057-77351-33"}]: dispatch 2024-06-11T13:33:31.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:30 smithi057 ceph-mon[29805]: from='client.26182 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi057-77351-33"}]: dispatch 2024-06-11T13:33:31.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:30 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3763978023' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi057-77351-33"}]: dispatch 2024-06-11T13:33:31.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:30 smithi057 ceph-mon[29805]: from='client.26182 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi057-77351-33"}]: dispatch 2024-06-11T13:33:31.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:30 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3763978023' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi057-77351-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:31.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:30 smithi057 ceph-mon[29805]: from='client.26182 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi057-77351-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:31.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:30 smithi057 ceph-mon[40588]: pgmap v285: 356 pgs: 356 active+clean; 584 KiB data, 573 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-11T13:33:31.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:30 smithi057 ceph-mon[40588]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:33:31.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:30 smithi057 ceph-mon[40588]: from='client.36176 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi057-77351-32"}]': finished 2024-06-11T13:33:31.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:30 smithi057 ceph-mon[40588]: osdmap e212: 8 total, 8 up, 8 in 2024-06-11T13:33:31.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:30 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3763978023' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi057-77351-33"}]: dispatch 2024-06-11T13:33:31.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:30 smithi057 ceph-mon[40588]: from='client.26182 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi057-77351-33"}]: dispatch 2024-06-11T13:33:31.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:30 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3763978023' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi057-77351-33"}]: dispatch 2024-06-11T13:33:31.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:30 smithi057 ceph-mon[40588]: from='client.26182 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi057-77351-33"}]: dispatch 2024-06-11T13:33:31.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:30 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3763978023' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi057-77351-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:31.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:30 smithi057 ceph-mon[40588]: from='client.26182 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi057-77351-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:31.547 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:33:31 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:33:32.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:32 smithi107 ceph-mon[33178]: from='client.26182 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi057-77351-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:33:32.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:32 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3763978023' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi057-77351-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi057-77351-33"}]: dispatch 2024-06-11T13:33:32.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:32 smithi107 ceph-mon[33178]: osdmap e213: 8 total, 8 up, 8 in 2024-06-11T13:33:32.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:32 smithi107 ceph-mon[33178]: from='client.26182 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi057-77351-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi057-77351-33"}]: dispatch 2024-06-11T13:33:32.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:32 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:32.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:32 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3939166731' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi057-77362-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:32.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:32 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:32.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:32 smithi107 ceph-mon[33178]: from='client.26185 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi057-77362-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:32.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:32 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:33:32.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:32 smithi107 ceph-mon[33178]: pgmap v288: 356 pgs: 64 creating+peering, 292 active+clean; 584 KiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:33:32.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:32 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:32.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:32 smithi057 ceph-mon[29805]: from='client.26182 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi057-77351-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:33:32.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:32 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3763978023' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi057-77351-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi057-77351-33"}]: dispatch 2024-06-11T13:33:32.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:32 smithi057 ceph-mon[29805]: osdmap e213: 8 total, 8 up, 8 in 2024-06-11T13:33:32.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:32 smithi057 ceph-mon[29805]: from='client.26182 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi057-77351-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi057-77351-33"}]: dispatch 2024-06-11T13:33:32.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:32 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:32.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:32 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3939166731' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi057-77362-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:32.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:32 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:32.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:32 smithi057 ceph-mon[29805]: from='client.26185 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi057-77362-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:32.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:32 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:33:32.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:32 smithi057 ceph-mon[29805]: pgmap v288: 356 pgs: 64 creating+peering, 292 active+clean; 584 KiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:33:32.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:32 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:32.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:32 smithi057 ceph-mon[40588]: from='client.26182 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi057-77351-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:33:32.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:32 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3763978023' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi057-77351-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi057-77351-33"}]: dispatch 2024-06-11T13:33:32.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:32 smithi057 ceph-mon[40588]: osdmap e213: 8 total, 8 up, 8 in 2024-06-11T13:33:32.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:32 smithi057 ceph-mon[40588]: from='client.26182 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi057-77351-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi057-77351-33"}]: dispatch 2024-06-11T13:33:32.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:32 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:32.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:32 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3939166731' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi057-77362-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:32.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:32 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:32.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:32 smithi057 ceph-mon[40588]: from='client.26185 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi057-77362-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:32.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:32 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:33:32.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:32 smithi057 ceph-mon[40588]: pgmap v288: 356 pgs: 64 creating+peering, 292 active+clean; 584 KiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:33:32.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:32 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:33.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:33 smithi107 ceph-mon[33178]: from='client.26185 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi057-77362-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:33.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:33 smithi107 ceph-mon[33178]: osdmap e214: 8 total, 8 up, 8 in 2024-06-11T13:33:33.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:33 smithi107 ceph-mon[33178]: from='client.26182 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi057-77351-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi057-77351-33"}]': finished 2024-06-11T13:33:33.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:33 smithi107 ceph-mon[33178]: osdmap e215: 8 total, 8 up, 8 in 2024-06-11T13:33:33.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:33 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/996922510' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi057-77362-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:33.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:33 smithi107 ceph-mon[33178]: from='client.36203 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi057-77362-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:33.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:33 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:33:33.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:33 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:33:33.732 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:33 smithi057 ceph-mon[40588]: from='client.26185 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi057-77362-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:33.732 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:33 smithi057 ceph-mon[40588]: osdmap e214: 8 total, 8 up, 8 in 2024-06-11T13:33:33.732 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:33 smithi057 ceph-mon[40588]: from='client.26182 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi057-77351-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi057-77351-33"}]': finished 2024-06-11T13:33:33.732 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:33 smithi057 ceph-mon[40588]: osdmap e215: 8 total, 8 up, 8 in 2024-06-11T13:33:33.732 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:33 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/996922510' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi057-77362-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:33.732 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:33 smithi057 ceph-mon[40588]: from='client.36203 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi057-77362-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:33.732 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:33 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:33:33.732 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:33 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:33:33.733 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:33 smithi057 ceph-mon[29805]: from='client.26185 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi057-77362-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:33.733 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:33 smithi057 ceph-mon[29805]: osdmap e214: 8 total, 8 up, 8 in 2024-06-11T13:33:33.733 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:33 smithi057 ceph-mon[29805]: from='client.26182 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi057-77351-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi057-77351-33"}]': finished 2024-06-11T13:33:33.733 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:33 smithi057 ceph-mon[29805]: osdmap e215: 8 total, 8 up, 8 in 2024-06-11T13:33:33.733 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:33 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/996922510' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi057-77362-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:33.733 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:33 smithi057 ceph-mon[29805]: from='client.36203 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi057-77362-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:33.733 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:33 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:33:33.733 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:33 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:33:34.034 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:33:33 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:33:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:33:34.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:34 smithi107 ceph-mon[33178]: pgmap v291: 396 pgs: 40 unknown, 64 creating+peering, 292 active+clean; 584 KiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:33:34.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:34 smithi107 ceph-mon[33178]: from='client.36203 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi057-77362-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:34.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:34 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-23", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:33:34.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:34 smithi107 ceph-mon[33178]: osdmap e216: 8 total, 8 up, 8 in 2024-06-11T13:33:34.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:34 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-23"}]: dispatch 2024-06-11T13:33:34.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:34 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-23"}]: dispatch 2024-06-11T13:33:34.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:34 smithi057 ceph-mon[29805]: pgmap v291: 396 pgs: 40 unknown, 64 creating+peering, 292 active+clean; 584 KiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:33:34.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:34 smithi057 ceph-mon[29805]: from='client.36203 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi057-77362-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:34.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:34 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-23", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:33:34.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:34 smithi057 ceph-mon[29805]: osdmap e216: 8 total, 8 up, 8 in 2024-06-11T13:33:34.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:34 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-23"}]: dispatch 2024-06-11T13:33:34.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:34 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-23"}]: dispatch 2024-06-11T13:33:34.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:34 smithi057 ceph-mon[40588]: pgmap v291: 396 pgs: 40 unknown, 64 creating+peering, 292 active+clean; 584 KiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:33:34.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:34 smithi057 ceph-mon[40588]: from='client.36203 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi057-77362-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:34.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:34 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-23", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:33:34.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:34 smithi057 ceph-mon[40588]: osdmap e216: 8 total, 8 up, 8 in 2024-06-11T13:33:34.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:34 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-23"}]: dispatch 2024-06-11T13:33:34.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:34 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-23"}]: dispatch 2024-06-11T13:33:36.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:35 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-23"}]': finished 2024-06-11T13:33:36.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:35 smithi057 ceph-mon[29805]: osdmap e217: 8 total, 8 up, 8 in 2024-06-11T13:33:36.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:35 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-23", "mode": "writeback"}]: dispatch 2024-06-11T13:33:36.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:35 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-23", "mode": "writeback"}]: dispatch 2024-06-11T13:33:36.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:35 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3763978023' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi057-77351-33"}]: dispatch 2024-06-11T13:33:36.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:35 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3891109704' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi057-77362-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:36.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:35 smithi057 ceph-mon[29805]: from='client.26182 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi057-77351-33"}]: dispatch 2024-06-11T13:33:36.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:35 smithi057 ceph-mon[29805]: from='client.36209 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi057-77362-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:36.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:35 smithi057 ceph-mon[29805]: pgmap v294: 420 pgs: 64 unknown, 64 creating+peering, 292 active+clean; 584 KiB data, 578 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:33:36.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:35 smithi057 ceph-mon[29805]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:33:36.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:35 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-23"}]': finished 2024-06-11T13:33:36.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:35 smithi057 ceph-mon[40588]: osdmap e217: 8 total, 8 up, 8 in 2024-06-11T13:33:36.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:35 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-23", "mode": "writeback"}]: dispatch 2024-06-11T13:33:36.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:35 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-23", "mode": "writeback"}]: dispatch 2024-06-11T13:33:36.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:35 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3763978023' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi057-77351-33"}]: dispatch 2024-06-11T13:33:36.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:35 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3891109704' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi057-77362-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:36.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:35 smithi057 ceph-mon[40588]: from='client.26182 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi057-77351-33"}]: dispatch 2024-06-11T13:33:36.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:35 smithi057 ceph-mon[40588]: from='client.36209 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi057-77362-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:36.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:35 smithi057 ceph-mon[40588]: pgmap v294: 420 pgs: 64 unknown, 64 creating+peering, 292 active+clean; 584 KiB data, 578 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:33:36.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:35 smithi057 ceph-mon[40588]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:33:36.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:35 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-23"}]': finished 2024-06-11T13:33:36.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:35 smithi107 ceph-mon[33178]: osdmap e217: 8 total, 8 up, 8 in 2024-06-11T13:33:36.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:35 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-23", "mode": "writeback"}]: dispatch 2024-06-11T13:33:36.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:35 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-23", "mode": "writeback"}]: dispatch 2024-06-11T13:33:36.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:35 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3763978023' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi057-77351-33"}]: dispatch 2024-06-11T13:33:36.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:35 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3891109704' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi057-77362-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:36.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:35 smithi107 ceph-mon[33178]: from='client.26182 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi057-77351-33"}]: dispatch 2024-06-11T13:33:36.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:35 smithi107 ceph-mon[33178]: from='client.36209 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi057-77362-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:36.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:35 smithi107 ceph-mon[33178]: pgmap v294: 420 pgs: 64 unknown, 64 creating+peering, 292 active+clean; 584 KiB data, 578 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:33:36.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:35 smithi107 ceph-mon[33178]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:33:37.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:36 smithi057 ceph-mon[40588]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:33:37.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:36 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-23", "mode": "writeback"}]': finished 2024-06-11T13:33:37.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:36 smithi057 ceph-mon[40588]: from='client.26182 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi057-77351-33"}]': finished 2024-06-11T13:33:37.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:36 smithi057 ceph-mon[40588]: from='client.36209 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi057-77362-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:37.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:36 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3763978023' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi057-77351-33"}]: dispatch 2024-06-11T13:33:37.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:36 smithi057 ceph-mon[40588]: osdmap e218: 8 total, 8 up, 8 in 2024-06-11T13:33:37.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:36 smithi057 ceph-mon[40588]: from='client.26182 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi057-77351-33"}]: dispatch 2024-06-11T13:33:37.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:36 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:33:37.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:36 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:33:37.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:36 smithi057 ceph-mon[29805]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:33:37.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:36 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-23", "mode": "writeback"}]': finished 2024-06-11T13:33:37.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:36 smithi057 ceph-mon[29805]: from='client.26182 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi057-77351-33"}]': finished 2024-06-11T13:33:37.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:36 smithi057 ceph-mon[29805]: from='client.36209 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi057-77362-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:37.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:36 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3763978023' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi057-77351-33"}]: dispatch 2024-06-11T13:33:37.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:36 smithi057 ceph-mon[29805]: osdmap e218: 8 total, 8 up, 8 in 2024-06-11T13:33:37.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:36 smithi057 ceph-mon[29805]: from='client.26182 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi057-77351-33"}]: dispatch 2024-06-11T13:33:37.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:36 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:33:37.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:36 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:33:37.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:36 smithi107 ceph-mon[33178]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:33:37.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:36 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-23", "mode": "writeback"}]': finished 2024-06-11T13:33:37.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:36 smithi107 ceph-mon[33178]: from='client.26182 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi057-77351-33"}]': finished 2024-06-11T13:33:37.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:36 smithi107 ceph-mon[33178]: from='client.36209 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi057-77362-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:37.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:36 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3763978023' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi057-77351-33"}]: dispatch 2024-06-11T13:33:37.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:36 smithi107 ceph-mon[33178]: osdmap e218: 8 total, 8 up, 8 in 2024-06-11T13:33:37.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:36 smithi107 ceph-mon[33178]: from='client.26182 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi057-77351-33"}]: dispatch 2024-06-11T13:33:37.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:36 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:33:37.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:36 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:33:38.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:37 smithi057 ceph-mon[29805]: from='client.26182 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi057-77351-33"}]': finished 2024-06-11T13:33:38.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:37 smithi057 ceph-mon[29805]: osdmap e219: 8 total, 8 up, 8 in 2024-06-11T13:33:38.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:37 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/361769779' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi057-77362-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:38.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:37 smithi057 ceph-mon[29805]: from='client.26203 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi057-77362-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:38.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:37 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/358653508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi057-77351-34"}]: dispatch 2024-06-11T13:33:38.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:37 smithi057 ceph-mon[29805]: from='client.26209 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi057-77351-34"}]: dispatch 2024-06-11T13:33:38.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:37 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/358653508' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi057-77351-34"}]: dispatch 2024-06-11T13:33:38.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:37 smithi057 ceph-mon[29805]: from='client.26209 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi057-77351-34"}]: dispatch 2024-06-11T13:33:38.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:37 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/358653508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi057-77351-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:38.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:37 smithi057 ceph-mon[29805]: from='client.26209 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi057-77351-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:38.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:37 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:33:38.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:37 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:33:38.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:37 smithi057 ceph-mon[29805]: pgmap v297: 452 pgs: 32 unknown, 420 active+clean; 584 KiB data, 583 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-06-11T13:33:38.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:37 smithi057 ceph-mon[40588]: from='client.26182 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi057-77351-33"}]': finished 2024-06-11T13:33:38.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:37 smithi057 ceph-mon[40588]: osdmap e219: 8 total, 8 up, 8 in 2024-06-11T13:33:38.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:37 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/361769779' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi057-77362-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:38.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:37 smithi057 ceph-mon[40588]: from='client.26203 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi057-77362-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:38.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:37 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/358653508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi057-77351-34"}]: dispatch 2024-06-11T13:33:38.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:37 smithi057 ceph-mon[40588]: from='client.26209 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi057-77351-34"}]: dispatch 2024-06-11T13:33:38.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:37 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/358653508' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi057-77351-34"}]: dispatch 2024-06-11T13:33:38.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:37 smithi057 ceph-mon[40588]: from='client.26209 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi057-77351-34"}]: dispatch 2024-06-11T13:33:38.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:37 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/358653508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi057-77351-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:38.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:37 smithi057 ceph-mon[40588]: from='client.26209 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi057-77351-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:38.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:37 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:33:38.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:37 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:33:38.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:37 smithi057 ceph-mon[40588]: pgmap v297: 452 pgs: 32 unknown, 420 active+clean; 584 KiB data, 583 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-06-11T13:33:38.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:37 smithi107 ceph-mon[33178]: from='client.26182 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi057-77351-33"}]': finished 2024-06-11T13:33:38.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:37 smithi107 ceph-mon[33178]: osdmap e219: 8 total, 8 up, 8 in 2024-06-11T13:33:38.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:37 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/361769779' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi057-77362-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:38.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:37 smithi107 ceph-mon[33178]: from='client.26203 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi057-77362-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:38.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:37 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/358653508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi057-77351-34"}]: dispatch 2024-06-11T13:33:38.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:37 smithi107 ceph-mon[33178]: from='client.26209 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi057-77351-34"}]: dispatch 2024-06-11T13:33:38.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:37 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/358653508' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi057-77351-34"}]: dispatch 2024-06-11T13:33:38.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:37 smithi107 ceph-mon[33178]: from='client.26209 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi057-77351-34"}]: dispatch 2024-06-11T13:33:38.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:37 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/358653508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi057-77351-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:38.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:37 smithi107 ceph-mon[33178]: from='client.26209 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi057-77351-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:38.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:37 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:33:38.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:37 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:33:38.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:37 smithi107 ceph-mon[33178]: pgmap v297: 452 pgs: 32 unknown, 420 active+clean; 584 KiB data, 583 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-06-11T13:33:39.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:39 smithi057 ceph-mon[40588]: from='client.26203 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi057-77362-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:39.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:39 smithi057 ceph-mon[40588]: from='client.26209 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi057-77351-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:33:39.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:39 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:33:39.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:39 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-23"}]: dispatch 2024-06-11T13:33:39.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:39 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/358653508' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi057-77351-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi057-77351-34"}]: dispatch 2024-06-11T13:33:39.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:39 smithi057 ceph-mon[40588]: osdmap e220: 8 total, 8 up, 8 in 2024-06-11T13:33:39.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:39 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-23"}]: dispatch 2024-06-11T13:33:39.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:39 smithi057 ceph-mon[40588]: from='client.26209 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi057-77351-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi057-77351-34"}]: dispatch 2024-06-11T13:33:39.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:39 smithi057 ceph-mon[29805]: from='client.26203 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi057-77362-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:39.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:39 smithi057 ceph-mon[29805]: from='client.26209 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi057-77351-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:33:39.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:39 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:33:39.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:39 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-23"}]: dispatch 2024-06-11T13:33:39.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:39 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/358653508' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi057-77351-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi057-77351-34"}]: dispatch 2024-06-11T13:33:39.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:39 smithi057 ceph-mon[29805]: osdmap e220: 8 total, 8 up, 8 in 2024-06-11T13:33:39.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:39 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-23"}]: dispatch 2024-06-11T13:33:39.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:39 smithi057 ceph-mon[29805]: from='client.26209 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi057-77351-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi057-77351-34"}]: dispatch 2024-06-11T13:33:39.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:39 smithi107 ceph-mon[33178]: from='client.26203 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi057-77362-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:39.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:39 smithi107 ceph-mon[33178]: from='client.26209 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi057-77351-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:33:39.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:39 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:33:39.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:39 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-23"}]: dispatch 2024-06-11T13:33:39.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:39 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/358653508' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi057-77351-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi057-77351-34"}]: dispatch 2024-06-11T13:33:39.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:39 smithi107 ceph-mon[33178]: osdmap e220: 8 total, 8 up, 8 in 2024-06-11T13:33:39.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:39 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-23"}]: dispatch 2024-06-11T13:33:39.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:39 smithi107 ceph-mon[33178]: from='client.26209 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi057-77351-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi057-77351-34"}]: dispatch 2024-06-11T13:33:40.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:40 smithi057 ceph-mon[29805]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:33:40.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:40 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-23"}]': finished 2024-06-11T13:33:40.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:40 smithi057 ceph-mon[29805]: pgmap v300: 484 pgs: 64 unknown, 420 active+clean; 584 KiB data, 583 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-06-11T13:33:40.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:40 smithi057 ceph-mon[29805]: osdmap e221: 8 total, 8 up, 8 in 2024-06-11T13:33:40.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:40 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/288144966' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi057-77362-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:40.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:40 smithi057 ceph-mon[29805]: from='client.36227 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi057-77362-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:40.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:40 smithi057 ceph-mon[29805]: from='client.26209 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi057-77351-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi057-77351-34"}]': finished 2024-06-11T13:33:40.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:40 smithi057 ceph-mon[29805]: from='client.36227 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi057-77362-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:40.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:40 smithi057 ceph-mon[29805]: osdmap e222: 8 total, 8 up, 8 in 2024-06-11T13:33:40.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:40 smithi057 ceph-mon[40588]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:33:40.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:40 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-23"}]': finished 2024-06-11T13:33:40.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:40 smithi057 ceph-mon[40588]: pgmap v300: 484 pgs: 64 unknown, 420 active+clean; 584 KiB data, 583 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-06-11T13:33:40.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:40 smithi057 ceph-mon[40588]: osdmap e221: 8 total, 8 up, 8 in 2024-06-11T13:33:40.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:40 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/288144966' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi057-77362-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:40.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:40 smithi057 ceph-mon[40588]: from='client.36227 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi057-77362-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:40.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:40 smithi057 ceph-mon[40588]: from='client.26209 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi057-77351-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi057-77351-34"}]': finished 2024-06-11T13:33:40.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:40 smithi057 ceph-mon[40588]: from='client.36227 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi057-77362-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:40.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:40 smithi057 ceph-mon[40588]: osdmap e222: 8 total, 8 up, 8 in 2024-06-11T13:33:40.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:40 smithi107 ceph-mon[33178]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:33:40.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:40 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-23"}]': finished 2024-06-11T13:33:40.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:40 smithi107 ceph-mon[33178]: pgmap v300: 484 pgs: 64 unknown, 420 active+clean; 584 KiB data, 583 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-06-11T13:33:40.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:40 smithi107 ceph-mon[33178]: osdmap e221: 8 total, 8 up, 8 in 2024-06-11T13:33:40.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:40 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/288144966' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi057-77362-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:40.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:40 smithi107 ceph-mon[33178]: from='client.36227 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi057-77362-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:40.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:40 smithi107 ceph-mon[33178]: from='client.26209 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi057-77351-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi057-77351-34"}]': finished 2024-06-11T13:33:40.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:40 smithi107 ceph-mon[33178]: from='client.36227 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi057-77362-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:40.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:40 smithi107 ceph-mon[33178]: osdmap e222: 8 total, 8 up, 8 in 2024-06-11T13:33:41.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:41 smithi107 ceph-mon[33178]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:33:41.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:41 smithi107 ceph-mon[33178]: osdmap e223: 8 total, 8 up, 8 in 2024-06-11T13:33:41.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:41 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:41.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:41 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:41.548 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:33:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:33:41.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:41 smithi057 ceph-mon[29805]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:33:41.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:41 smithi057 ceph-mon[29805]: osdmap e223: 8 total, 8 up, 8 in 2024-06-11T13:33:41.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:41 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:41.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:41 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:41.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:41 smithi057 ceph-mon[40588]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:33:41.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:41 smithi057 ceph-mon[40588]: osdmap e223: 8 total, 8 up, 8 in 2024-06-11T13:33:41.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:41 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:41.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:41 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:42.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:42 smithi057 ceph-mon[29805]: pgmap v303: 460 pgs: 32 creating+peering, 428 active+clean; 584 KiB data, 571 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:33:42.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:42 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:33:42.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:42 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:42.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:42 smithi057 ceph-mon[29805]: osdmap e224: 8 total, 8 up, 8 in 2024-06-11T13:33:42.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:42 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/358653508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi057-77351-34"}]: dispatch 2024-06-11T13:33:42.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:42 smithi057 ceph-mon[29805]: from='client.26209 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi057-77351-34"}]: dispatch 2024-06-11T13:33:42.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:42 smithi057 ceph-mon[40588]: pgmap v303: 460 pgs: 32 creating+peering, 428 active+clean; 584 KiB data, 571 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:33:42.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:42 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:33:42.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:42 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:42.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:42 smithi057 ceph-mon[40588]: osdmap e224: 8 total, 8 up, 8 in 2024-06-11T13:33:42.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:42 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/358653508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi057-77351-34"}]: dispatch 2024-06-11T13:33:42.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:42 smithi057 ceph-mon[40588]: from='client.26209 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi057-77351-34"}]: dispatch 2024-06-11T13:33:42.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:42 smithi107 ceph-mon[33178]: pgmap v303: 460 pgs: 32 creating+peering, 428 active+clean; 584 KiB data, 571 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:33:42.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:42 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:33:42.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:42 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:42.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:42 smithi107 ceph-mon[33178]: osdmap e224: 8 total, 8 up, 8 in 2024-06-11T13:33:42.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:42 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/358653508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi057-77351-34"}]: dispatch 2024-06-11T13:33:42.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:42 smithi107 ceph-mon[33178]: from='client.26209 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi057-77351-34"}]: dispatch 2024-06-11T13:33:44.034 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:33:43 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:33:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:33:44.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:43 smithi057 ceph-mon[40588]: from='client.26209 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi057-77351-34"}]': finished 2024-06-11T13:33:44.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:43 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/358653508' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi057-77351-34"}]: dispatch 2024-06-11T13:33:44.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:43 smithi057 ceph-mon[40588]: osdmap e225: 8 total, 8 up, 8 in 2024-06-11T13:33:44.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:43 smithi057 ceph-mon[40588]: from='client.26209 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi057-77351-34"}]: dispatch 2024-06-11T13:33:44.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:43 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:33:44.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:43 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:33:44.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:43 smithi057 ceph-mon[40588]: pgmap v306: 388 pgs: 32 creating+peering, 356 active+clean; 584 KiB data, 571 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:33:44.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:43 smithi057 ceph-mon[29805]: from='client.26209 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi057-77351-34"}]': finished 2024-06-11T13:33:44.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:43 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/358653508' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi057-77351-34"}]: dispatch 2024-06-11T13:33:44.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:43 smithi057 ceph-mon[29805]: osdmap e225: 8 total, 8 up, 8 in 2024-06-11T13:33:44.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:43 smithi057 ceph-mon[29805]: from='client.26209 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi057-77351-34"}]: dispatch 2024-06-11T13:33:44.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:43 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:33:44.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:43 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:33:44.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:43 smithi057 ceph-mon[29805]: pgmap v306: 388 pgs: 32 creating+peering, 356 active+clean; 584 KiB data, 571 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:33:44.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:43 smithi107 ceph-mon[33178]: from='client.26209 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi057-77351-34"}]': finished 2024-06-11T13:33:44.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:43 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/358653508' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi057-77351-34"}]: dispatch 2024-06-11T13:33:44.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:43 smithi107 ceph-mon[33178]: osdmap e225: 8 total, 8 up, 8 in 2024-06-11T13:33:44.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:43 smithi107 ceph-mon[33178]: from='client.26209 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi057-77351-34"}]: dispatch 2024-06-11T13:33:44.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:43 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:33:44.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:43 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:33:44.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:43 smithi107 ceph-mon[33178]: pgmap v306: 388 pgs: 32 creating+peering, 356 active+clean; 584 KiB data, 571 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:33:45.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:44 smithi057 ceph-mon[29805]: from='client.26209 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi057-77351-34"}]': finished 2024-06-11T13:33:45.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:44 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-25", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:33:45.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:44 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-25"}]: dispatch 2024-06-11T13:33:45.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:44 smithi057 ceph-mon[29805]: osdmap e226: 8 total, 8 up, 8 in 2024-06-11T13:33:45.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:44 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-25"}]: dispatch 2024-06-11T13:33:45.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:44 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2949788271' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi057-77351-35"}]: dispatch 2024-06-11T13:33:45.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:44 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2949788271' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi057-77351-35"}]: dispatch 2024-06-11T13:33:45.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:44 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2949788271' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi057-77351-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:45.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:44 smithi057 ceph-mon[40588]: from='client.26209 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi057-77351-34"}]': finished 2024-06-11T13:33:45.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:44 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-25", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:33:45.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:44 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-25"}]: dispatch 2024-06-11T13:33:45.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:44 smithi057 ceph-mon[40588]: osdmap e226: 8 total, 8 up, 8 in 2024-06-11T13:33:45.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:44 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-25"}]: dispatch 2024-06-11T13:33:45.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:44 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2949788271' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi057-77351-35"}]: dispatch 2024-06-11T13:33:45.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:44 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2949788271' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi057-77351-35"}]: dispatch 2024-06-11T13:33:45.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:44 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2949788271' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi057-77351-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:45.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:44 smithi107 ceph-mon[33178]: from='client.26209 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi057-77351-34"}]': finished 2024-06-11T13:33:45.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:44 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-25", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:33:45.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:44 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-25"}]: dispatch 2024-06-11T13:33:45.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:44 smithi107 ceph-mon[33178]: osdmap e226: 8 total, 8 up, 8 in 2024-06-11T13:33:45.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:44 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-25"}]: dispatch 2024-06-11T13:33:45.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:44 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2949788271' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi057-77351-35"}]: dispatch 2024-06-11T13:33:45.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:44 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2949788271' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi057-77351-35"}]: dispatch 2024-06-11T13:33:45.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:44 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2949788271' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi057-77351-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:46.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:46 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-25"}]': finished 2024-06-11T13:33:46.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:46 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2949788271' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi057-77351-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:33:46.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:46 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-25", "mode": "writeback"}]: dispatch 2024-06-11T13:33:46.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:46 smithi057 ceph-mon[29805]: osdmap e227: 8 total, 8 up, 8 in 2024-06-11T13:33:46.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:46 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2949788271' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi057-77351-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi057-77351-35"}]: dispatch 2024-06-11T13:33:46.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:46 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-25", "mode": "writeback"}]: dispatch 2024-06-11T13:33:46.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:46 smithi057 ceph-mon[29805]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:33:46.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:46 smithi057 ceph-mon[29805]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:33:46.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:46 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-25", "mode": "writeback"}]': finished 2024-06-11T13:33:46.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:46 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-25"}]': finished 2024-06-11T13:33:46.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:46 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2949788271' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi057-77351-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:33:46.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:46 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-25", "mode": "writeback"}]: dispatch 2024-06-11T13:33:46.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:46 smithi057 ceph-mon[40588]: osdmap e227: 8 total, 8 up, 8 in 2024-06-11T13:33:46.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:46 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2949788271' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi057-77351-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi057-77351-35"}]: dispatch 2024-06-11T13:33:46.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:46 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-25", "mode": "writeback"}]: dispatch 2024-06-11T13:33:46.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:46 smithi057 ceph-mon[40588]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:33:46.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:46 smithi057 ceph-mon[40588]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:33:46.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:46 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-25", "mode": "writeback"}]': finished 2024-06-11T13:33:46.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:46 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-25"}]': finished 2024-06-11T13:33:46.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:46 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2949788271' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi057-77351-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:33:46.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:46 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-25", "mode": "writeback"}]: dispatch 2024-06-11T13:33:46.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:46 smithi107 ceph-mon[33178]: osdmap e227: 8 total, 8 up, 8 in 2024-06-11T13:33:46.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:46 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2949788271' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi057-77351-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi057-77351-35"}]: dispatch 2024-06-11T13:33:46.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:46 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-25", "mode": "writeback"}]: dispatch 2024-06-11T13:33:46.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:46 smithi107 ceph-mon[33178]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:33:46.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:46 smithi107 ceph-mon[33178]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:33:46.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:46 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-25", "mode": "writeback"}]': finished 2024-06-11T13:33:47.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:47 smithi057 ceph-mon[29805]: pgmap v309: 324 pgs: 32 creating+peering, 292 active+clean; 584 KiB data, 571 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:33:47.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:47 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:33:47.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:47 smithi057 ceph-mon[29805]: osdmap e228: 8 total, 8 up, 8 in 2024-06-11T13:33:47.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:47 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/33302034' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi057-77362-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:47.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:47 smithi057 ceph-mon[29805]: from='client.36239 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi057-77362-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:47.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:47 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:33:47.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:47 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2949788271' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi057-77351-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi057-77351-35"}]': finished 2024-06-11T13:33:47.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:47 smithi057 ceph-mon[29805]: from='client.36239 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi057-77362-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:47.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:47 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:33:47.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:47 smithi057 ceph-mon[29805]: osdmap e229: 8 total, 8 up, 8 in 2024-06-11T13:33:47.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:47 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-25"}]: dispatch 2024-06-11T13:33:47.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:47 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-25"}]: dispatch 2024-06-11T13:33:47.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:47 smithi057 ceph-mon[40588]: pgmap v309: 324 pgs: 32 creating+peering, 292 active+clean; 584 KiB data, 571 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:33:47.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:47 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:33:47.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:47 smithi057 ceph-mon[40588]: osdmap e228: 8 total, 8 up, 8 in 2024-06-11T13:33:47.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:47 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/33302034' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi057-77362-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:47.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:47 smithi057 ceph-mon[40588]: from='client.36239 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi057-77362-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:47.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:47 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:33:47.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:47 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2949788271' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi057-77351-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi057-77351-35"}]': finished 2024-06-11T13:33:47.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:47 smithi057 ceph-mon[40588]: from='client.36239 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi057-77362-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:47.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:47 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:33:47.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:47 smithi057 ceph-mon[40588]: osdmap e229: 8 total, 8 up, 8 in 2024-06-11T13:33:47.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:47 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-25"}]: dispatch 2024-06-11T13:33:47.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:47 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-25"}]: dispatch 2024-06-11T13:33:47.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:47 smithi107 ceph-mon[33178]: pgmap v309: 324 pgs: 32 creating+peering, 292 active+clean; 584 KiB data, 571 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:33:47.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:47 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:33:47.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:47 smithi107 ceph-mon[33178]: osdmap e228: 8 total, 8 up, 8 in 2024-06-11T13:33:47.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:47 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/33302034' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi057-77362-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:47.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:47 smithi107 ceph-mon[33178]: from='client.36239 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi057-77362-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:47.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:47 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:33:47.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:47 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2949788271' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi057-77351-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi057-77351-35"}]': finished 2024-06-11T13:33:47.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:47 smithi107 ceph-mon[33178]: from='client.36239 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi057-77362-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:47.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:47 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:33:47.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:47 smithi107 ceph-mon[33178]: osdmap e229: 8 total, 8 up, 8 in 2024-06-11T13:33:47.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:47 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-25"}]: dispatch 2024-06-11T13:33:47.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:47 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-25"}]: dispatch 2024-06-11T13:33:48.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:48 smithi057 ceph-mon[29805]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:33:48.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:48 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-25"}]': finished 2024-06-11T13:33:48.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:48 smithi057 ceph-mon[29805]: osdmap e230: 8 total, 8 up, 8 in 2024-06-11T13:33:48.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:48 smithi057 ceph-mon[40588]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:33:48.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:48 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-25"}]': finished 2024-06-11T13:33:48.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:48 smithi057 ceph-mon[40588]: osdmap e230: 8 total, 8 up, 8 in 2024-06-11T13:33:48.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:48 smithi107 ceph-mon[33178]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:33:48.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:48 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-25"}]': finished 2024-06-11T13:33:48.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:48 smithi107 ceph-mon[33178]: osdmap e230: 8 total, 8 up, 8 in 2024-06-11T13:33:49.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:49 smithi057 ceph-mon[29805]: pgmap v312: 364 pgs: 24 creating+peering, 16 unknown, 324 active+clean; 584 KiB data, 572 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-06-11T13:33:49.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:49 smithi057 ceph-mon[29805]: osdmap e231: 8 total, 8 up, 8 in 2024-06-11T13:33:49.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:49 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2949788271' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi057-77351-35"}]: dispatch 2024-06-11T13:33:49.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:49 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/201652934' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi057-77362-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:49.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:49 smithi057 ceph-mon[40588]: pgmap v312: 364 pgs: 24 creating+peering, 16 unknown, 324 active+clean; 584 KiB data, 572 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-06-11T13:33:49.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:49 smithi057 ceph-mon[40588]: osdmap e231: 8 total, 8 up, 8 in 2024-06-11T13:33:49.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:49 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2949788271' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi057-77351-35"}]: dispatch 2024-06-11T13:33:49.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:49 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/201652934' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi057-77362-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:49.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:49 smithi107 ceph-mon[33178]: pgmap v312: 364 pgs: 24 creating+peering, 16 unknown, 324 active+clean; 584 KiB data, 572 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-06-11T13:33:49.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:49 smithi107 ceph-mon[33178]: osdmap e231: 8 total, 8 up, 8 in 2024-06-11T13:33:49.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:49 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2949788271' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi057-77351-35"}]: dispatch 2024-06-11T13:33:49.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:49 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/201652934' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi057-77362-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:51.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:50 smithi057 ceph-mon[40588]: pgmap v315: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 572 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-11T13:33:51.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:50 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2949788271' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi057-77351-35"}]': finished 2024-06-11T13:33:51.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:50 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/201652934' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi057-77362-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:51.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:50 smithi057 ceph-mon[40588]: osdmap e232: 8 total, 8 up, 8 in 2024-06-11T13:33:51.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:50 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2949788271' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi057-77351-35"}]: dispatch 2024-06-11T13:33:51.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:50 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:51.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:50 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:51.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:50 smithi057 ceph-mon[40588]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:33:51.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:50 smithi057 ceph-mon[29805]: pgmap v315: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 572 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-11T13:33:51.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:50 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2949788271' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi057-77351-35"}]': finished 2024-06-11T13:33:51.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:50 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/201652934' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi057-77362-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:51.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:50 smithi057 ceph-mon[29805]: osdmap e232: 8 total, 8 up, 8 in 2024-06-11T13:33:51.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:50 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2949788271' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi057-77351-35"}]: dispatch 2024-06-11T13:33:51.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:50 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:51.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:50 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:51.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:50 smithi057 ceph-mon[29805]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:33:51.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:50 smithi107 ceph-mon[33178]: pgmap v315: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 572 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-11T13:33:51.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:50 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2949788271' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi057-77351-35"}]': finished 2024-06-11T13:33:51.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:50 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/201652934' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi057-77362-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:51.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:50 smithi107 ceph-mon[33178]: osdmap e232: 8 total, 8 up, 8 in 2024-06-11T13:33:51.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:50 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2949788271' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi057-77351-35"}]: dispatch 2024-06-11T13:33:51.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:50 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:51.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:50 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:33:51.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:50 smithi107 ceph-mon[33178]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:33:51.547 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:33:51 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:33:52.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:51 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2949788271' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi057-77351-35"}]': finished 2024-06-11T13:33:52.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:51 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:52.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:51 smithi057 ceph-mon[40588]: osdmap e233: 8 total, 8 up, 8 in 2024-06-11T13:33:52.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:51 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/546569324' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi057-77362-46"}]: dispatch 2024-06-11T13:33:52.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:51 smithi057 ceph-mon[40588]: from='client.26242 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi057-77362-46"}]: dispatch 2024-06-11T13:33:52.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:51 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2950463589' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi057-77351-36"}]: dispatch 2024-06-11T13:33:52.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:51 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/546569324' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi057-77362-46"}]: dispatch 2024-06-11T13:33:52.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:51 smithi057 ceph-mon[40588]: from='client.26245 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi057-77351-36"}]: dispatch 2024-06-11T13:33:52.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:51 smithi057 ceph-mon[40588]: from='client.26242 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi057-77362-46"}]: dispatch 2024-06-11T13:33:52.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:51 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/546569324' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi057-77362-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:52.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:51 smithi057 ceph-mon[40588]: from='client.26242 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi057-77362-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:52.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:51 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2950463589' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi057-77351-36"}]: dispatch 2024-06-11T13:33:52.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:51 smithi057 ceph-mon[40588]: from='client.26245 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi057-77351-36"}]: dispatch 2024-06-11T13:33:52.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:51 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2950463589' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi057-77351-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:52.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:51 smithi057 ceph-mon[40588]: from='client.26245 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi057-77351-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:52.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:51 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:33:52.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:51 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:33:52.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:51 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2949788271' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi057-77351-35"}]': finished 2024-06-11T13:33:52.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:51 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:52.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:51 smithi057 ceph-mon[29805]: osdmap e233: 8 total, 8 up, 8 in 2024-06-11T13:33:52.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:51 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/546569324' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi057-77362-46"}]: dispatch 2024-06-11T13:33:52.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:51 smithi057 ceph-mon[29805]: from='client.26242 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi057-77362-46"}]: dispatch 2024-06-11T13:33:52.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:51 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2950463589' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi057-77351-36"}]: dispatch 2024-06-11T13:33:52.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:51 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/546569324' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi057-77362-46"}]: dispatch 2024-06-11T13:33:52.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:51 smithi057 ceph-mon[29805]: from='client.26245 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi057-77351-36"}]: dispatch 2024-06-11T13:33:52.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:51 smithi057 ceph-mon[29805]: from='client.26242 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi057-77362-46"}]: dispatch 2024-06-11T13:33:52.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:51 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/546569324' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi057-77362-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:52.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:51 smithi057 ceph-mon[29805]: from='client.26242 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi057-77362-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:52.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:51 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2950463589' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi057-77351-36"}]: dispatch 2024-06-11T13:33:52.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:51 smithi057 ceph-mon[29805]: from='client.26245 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi057-77351-36"}]: dispatch 2024-06-11T13:33:52.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:51 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2950463589' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi057-77351-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:52.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:51 smithi057 ceph-mon[29805]: from='client.26245 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi057-77351-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:52.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:51 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:33:52.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:51 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:33:52.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:51 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2949788271' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi057-77351-35"}]': finished 2024-06-11T13:33:52.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:51 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:33:52.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:51 smithi107 ceph-mon[33178]: osdmap e233: 8 total, 8 up, 8 in 2024-06-11T13:33:52.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:51 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/546569324' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi057-77362-46"}]: dispatch 2024-06-11T13:33:52.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:51 smithi107 ceph-mon[33178]: from='client.26242 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi057-77362-46"}]: dispatch 2024-06-11T13:33:52.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:51 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2950463589' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi057-77351-36"}]: dispatch 2024-06-11T13:33:52.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:51 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/546569324' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi057-77362-46"}]: dispatch 2024-06-11T13:33:52.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:51 smithi107 ceph-mon[33178]: from='client.26245 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi057-77351-36"}]: dispatch 2024-06-11T13:33:52.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:51 smithi107 ceph-mon[33178]: from='client.26242 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi057-77362-46"}]: dispatch 2024-06-11T13:33:52.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:51 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/546569324' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi057-77362-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:52.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:51 smithi107 ceph-mon[33178]: from='client.26242 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi057-77362-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:52.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:51 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2950463589' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi057-77351-36"}]: dispatch 2024-06-11T13:33:52.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:51 smithi107 ceph-mon[33178]: from='client.26245 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi057-77351-36"}]: dispatch 2024-06-11T13:33:52.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:51 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2950463589' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi057-77351-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:52.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:51 smithi107 ceph-mon[33178]: from='client.26245 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi057-77351-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:52.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:51 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:33:52.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:51 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:33:53.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:52 smithi057 ceph-mon[40588]: pgmap v318: 324 pgs: 324 active+clean; 584 KiB data, 573 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1023 B/s wr, 1 op/s 2024-06-11T13:33:53.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:52 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:33:53.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:52 smithi057 ceph-mon[40588]: from='client.26242 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi057-77362-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:33:53.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:52 smithi057 ceph-mon[40588]: from='client.26245 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi057-77351-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:33:53.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:52 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/546569324' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi057-77362-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi057-77362-46"}]: dispatch 2024-06-11T13:33:53.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:52 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2950463589' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi057-77351-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi057-77351-36"}]: dispatch 2024-06-11T13:33:53.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:52 smithi057 ceph-mon[40588]: osdmap e234: 8 total, 8 up, 8 in 2024-06-11T13:33:53.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:52 smithi057 ceph-mon[40588]: from='client.26242 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi057-77362-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi057-77362-46"}]: dispatch 2024-06-11T13:33:53.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:52 smithi057 ceph-mon[40588]: from='client.26245 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi057-77351-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi057-77351-36"}]: dispatch 2024-06-11T13:33:53.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:52 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:33:53.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:52 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:33:53.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:52 smithi057 ceph-mon[29805]: pgmap v318: 324 pgs: 324 active+clean; 584 KiB data, 573 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1023 B/s wr, 1 op/s 2024-06-11T13:33:53.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:52 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:33:53.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:52 smithi057 ceph-mon[29805]: from='client.26242 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi057-77362-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:33:53.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:52 smithi057 ceph-mon[29805]: from='client.26245 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi057-77351-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:33:53.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:52 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/546569324' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi057-77362-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi057-77362-46"}]: dispatch 2024-06-11T13:33:53.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:52 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2950463589' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi057-77351-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi057-77351-36"}]: dispatch 2024-06-11T13:33:53.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:52 smithi057 ceph-mon[29805]: osdmap e234: 8 total, 8 up, 8 in 2024-06-11T13:33:53.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:52 smithi057 ceph-mon[29805]: from='client.26242 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi057-77362-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi057-77362-46"}]: dispatch 2024-06-11T13:33:53.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:52 smithi057 ceph-mon[29805]: from='client.26245 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi057-77351-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi057-77351-36"}]: dispatch 2024-06-11T13:33:53.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:52 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:33:53.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:52 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:33:53.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:52 smithi107 ceph-mon[33178]: pgmap v318: 324 pgs: 324 active+clean; 584 KiB data, 573 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1023 B/s wr, 1 op/s 2024-06-11T13:33:53.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:52 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:33:53.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:52 smithi107 ceph-mon[33178]: from='client.26242 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi057-77362-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:33:53.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:52 smithi107 ceph-mon[33178]: from='client.26245 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi057-77351-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:33:53.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:52 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/546569324' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi057-77362-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi057-77362-46"}]: dispatch 2024-06-11T13:33:53.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:52 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2950463589' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi057-77351-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi057-77351-36"}]: dispatch 2024-06-11T13:33:53.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:52 smithi107 ceph-mon[33178]: osdmap e234: 8 total, 8 up, 8 in 2024-06-11T13:33:53.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:52 smithi107 ceph-mon[33178]: from='client.26242 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi057-77362-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi057-77362-46"}]: dispatch 2024-06-11T13:33:53.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:52 smithi107 ceph-mon[33178]: from='client.26245 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi057-77351-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi057-77351-36"}]: dispatch 2024-06-11T13:33:53.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:52 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:33:53.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:52 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:33:54.034 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:33:53 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:33:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:33:54.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:53 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-27", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:33:54.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:53 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-27"}]: dispatch 2024-06-11T13:33:54.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:53 smithi057 ceph-mon[40588]: osdmap e235: 8 total, 8 up, 8 in 2024-06-11T13:33:54.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:53 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-27"}]: dispatch 2024-06-11T13:33:54.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:53 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-27", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:33:54.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:53 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-27"}]: dispatch 2024-06-11T13:33:54.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:53 smithi057 ceph-mon[29805]: osdmap e235: 8 total, 8 up, 8 in 2024-06-11T13:33:54.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:53 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-27"}]: dispatch 2024-06-11T13:33:54.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:53 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-27", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:33:54.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:53 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-27"}]: dispatch 2024-06-11T13:33:54.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:53 smithi107 ceph-mon[33178]: osdmap e235: 8 total, 8 up, 8 in 2024-06-11T13:33:54.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:53 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-27"}]: dispatch 2024-06-11T13:33:55.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:54 smithi057 ceph-mon[40588]: pgmap v321: 324 pgs: 324 active+clean; 584 KiB data, 573 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-06-11T13:33:55.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:54 smithi057 ceph-mon[40588]: from='client.26242 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi057-77362-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi057-77362-46"}]': finished 2024-06-11T13:33:55.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:54 smithi057 ceph-mon[40588]: from='client.26245 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi057-77351-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi057-77351-36"}]': finished 2024-06-11T13:33:55.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:54 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-27"}]': finished 2024-06-11T13:33:55.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:54 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-27", "mode": "writeback"}]: dispatch 2024-06-11T13:33:55.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:54 smithi057 ceph-mon[40588]: osdmap e236: 8 total, 8 up, 8 in 2024-06-11T13:33:55.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:54 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-27", "mode": "writeback"}]: dispatch 2024-06-11T13:33:55.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:54 smithi057 ceph-mon[29805]: pgmap v321: 324 pgs: 324 active+clean; 584 KiB data, 573 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-06-11T13:33:55.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:54 smithi057 ceph-mon[29805]: from='client.26242 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi057-77362-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi057-77362-46"}]': finished 2024-06-11T13:33:55.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:54 smithi057 ceph-mon[29805]: from='client.26245 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi057-77351-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi057-77351-36"}]': finished 2024-06-11T13:33:55.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:54 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-27"}]': finished 2024-06-11T13:33:55.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:54 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-27", "mode": "writeback"}]: dispatch 2024-06-11T13:33:55.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:54 smithi057 ceph-mon[29805]: osdmap e236: 8 total, 8 up, 8 in 2024-06-11T13:33:55.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:54 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-27", "mode": "writeback"}]: dispatch 2024-06-11T13:33:55.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:54 smithi107 ceph-mon[33178]: pgmap v321: 324 pgs: 324 active+clean; 584 KiB data, 573 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-06-11T13:33:55.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:54 smithi107 ceph-mon[33178]: from='client.26242 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi057-77362-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi057-77362-46"}]': finished 2024-06-11T13:33:55.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:54 smithi107 ceph-mon[33178]: from='client.26245 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi057-77351-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi057-77351-36"}]': finished 2024-06-11T13:33:55.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:54 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-27"}]': finished 2024-06-11T13:33:55.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:54 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-27", "mode": "writeback"}]: dispatch 2024-06-11T13:33:55.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:54 smithi107 ceph-mon[33178]: osdmap e236: 8 total, 8 up, 8 in 2024-06-11T13:33:55.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:54 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-27", "mode": "writeback"}]: dispatch 2024-06-11T13:33:56.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:55 smithi057 ceph-mon[40588]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:33:56.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:55 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-27", "mode": "writeback"}]': finished 2024-06-11T13:33:56.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:55 smithi057 ceph-mon[40588]: osdmap e237: 8 total, 8 up, 8 in 2024-06-11T13:33:56.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:55 smithi057 ceph-mon[40588]: pgmap v324: 340 pgs: 16 unknown, 324 active+clean; 584 KiB data, 573 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:33:56.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:55 smithi057 ceph-mon[40588]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:33:56.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:55 smithi057 ceph-mon[40588]: osdmap e238: 8 total, 8 up, 8 in 2024-06-11T13:33:56.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:55 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2950463589' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi057-77351-36"}]: dispatch 2024-06-11T13:33:56.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:55 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/546569324' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi057-77362-46"}]: dispatch 2024-06-11T13:33:56.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:55 smithi057 ceph-mon[40588]: from='client.26245 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi057-77351-36"}]: dispatch 2024-06-11T13:33:56.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:55 smithi057 ceph-mon[40588]: from='client.26242 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi057-77362-46"}]: dispatch 2024-06-11T13:33:56.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:55 smithi057 ceph-mon[29805]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:33:56.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:55 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-27", "mode": "writeback"}]': finished 2024-06-11T13:33:56.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:55 smithi057 ceph-mon[29805]: osdmap e237: 8 total, 8 up, 8 in 2024-06-11T13:33:56.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:55 smithi057 ceph-mon[29805]: pgmap v324: 340 pgs: 16 unknown, 324 active+clean; 584 KiB data, 573 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:33:56.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:55 smithi057 ceph-mon[29805]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:33:56.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:55 smithi057 ceph-mon[29805]: osdmap e238: 8 total, 8 up, 8 in 2024-06-11T13:33:56.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:55 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2950463589' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi057-77351-36"}]: dispatch 2024-06-11T13:33:56.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:55 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/546569324' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi057-77362-46"}]: dispatch 2024-06-11T13:33:56.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:55 smithi057 ceph-mon[29805]: from='client.26245 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi057-77351-36"}]: dispatch 2024-06-11T13:33:56.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:55 smithi057 ceph-mon[29805]: from='client.26242 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi057-77362-46"}]: dispatch 2024-06-11T13:33:56.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:55 smithi107 ceph-mon[33178]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:33:56.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:55 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-27", "mode": "writeback"}]': finished 2024-06-11T13:33:56.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:55 smithi107 ceph-mon[33178]: osdmap e237: 8 total, 8 up, 8 in 2024-06-11T13:33:56.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:55 smithi107 ceph-mon[33178]: pgmap v324: 340 pgs: 16 unknown, 324 active+clean; 584 KiB data, 573 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:33:56.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:55 smithi107 ceph-mon[33178]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:33:56.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:55 smithi107 ceph-mon[33178]: osdmap e238: 8 total, 8 up, 8 in 2024-06-11T13:33:56.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:55 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2950463589' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi057-77351-36"}]: dispatch 2024-06-11T13:33:56.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:55 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/546569324' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi057-77362-46"}]: dispatch 2024-06-11T13:33:56.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:55 smithi107 ceph-mon[33178]: from='client.26245 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi057-77351-36"}]: dispatch 2024-06-11T13:33:56.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:55 smithi107 ceph-mon[33178]: from='client.26242 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi057-77362-46"}]: dispatch 2024-06-11T13:33:57.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:56 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:33:57.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:56 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:33:57.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:56 smithi057 ceph-mon[40588]: from='client.26245 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi057-77351-36"}]': finished 2024-06-11T13:33:57.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:56 smithi057 ceph-mon[40588]: from='client.26242 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi057-77362-46"}]': finished 2024-06-11T13:33:57.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:56 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:33:57.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:56 smithi057 ceph-mon[40588]: osdmap e239: 8 total, 8 up, 8 in 2024-06-11T13:33:57.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:56 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/546569324' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi057-77362-46"}]: dispatch 2024-06-11T13:33:57.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:56 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-27"}]: dispatch 2024-06-11T13:33:57.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:56 smithi057 ceph-mon[40588]: from='client.26242 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi057-77362-46"}]: dispatch 2024-06-11T13:33:57.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:56 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2950463589' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi057-77351-36"}]: dispatch 2024-06-11T13:33:57.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:56 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-27"}]: dispatch 2024-06-11T13:33:57.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:56 smithi057 ceph-mon[40588]: from='client.26245 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi057-77351-36"}]: dispatch 2024-06-11T13:33:57.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:56 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:33:57.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:56 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:33:57.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:56 smithi057 ceph-mon[29805]: from='client.26245 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi057-77351-36"}]': finished 2024-06-11T13:33:57.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:56 smithi057 ceph-mon[29805]: from='client.26242 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi057-77362-46"}]': finished 2024-06-11T13:33:57.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:56 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:33:57.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:56 smithi057 ceph-mon[29805]: osdmap e239: 8 total, 8 up, 8 in 2024-06-11T13:33:57.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:56 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/546569324' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi057-77362-46"}]: dispatch 2024-06-11T13:33:57.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:56 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-27"}]: dispatch 2024-06-11T13:33:57.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:56 smithi057 ceph-mon[29805]: from='client.26242 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi057-77362-46"}]: dispatch 2024-06-11T13:33:57.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:56 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2950463589' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi057-77351-36"}]: dispatch 2024-06-11T13:33:57.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:56 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-27"}]: dispatch 2024-06-11T13:33:57.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:56 smithi057 ceph-mon[29805]: from='client.26245 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi057-77351-36"}]: dispatch 2024-06-11T13:33:57.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:56 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:33:57.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:56 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:33:57.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:56 smithi107 ceph-mon[33178]: from='client.26245 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi057-77351-36"}]': finished 2024-06-11T13:33:57.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:56 smithi107 ceph-mon[33178]: from='client.26242 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi057-77362-46"}]': finished 2024-06-11T13:33:57.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:56 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:33:57.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:56 smithi107 ceph-mon[33178]: osdmap e239: 8 total, 8 up, 8 in 2024-06-11T13:33:57.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:56 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/546569324' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi057-77362-46"}]: dispatch 2024-06-11T13:33:57.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:56 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-27"}]: dispatch 2024-06-11T13:33:57.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:56 smithi107 ceph-mon[33178]: from='client.26242 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi057-77362-46"}]: dispatch 2024-06-11T13:33:57.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:56 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2950463589' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi057-77351-36"}]: dispatch 2024-06-11T13:33:57.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:56 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-27"}]: dispatch 2024-06-11T13:33:57.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:56 smithi107 ceph-mon[33178]: from='client.26245 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi057-77351-36"}]: dispatch 2024-06-11T13:33:58.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:57 smithi057 ceph-mon[29805]: pgmap v327: 324 pgs: 324 active+clean; 584 KiB data, 573 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 3 op/s 2024-06-11T13:33:58.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:57 smithi057 ceph-mon[29805]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:33:58.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:57 smithi057 ceph-mon[29805]: from='client.26242 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi057-77362-46"}]': finished 2024-06-11T13:33:58.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:57 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-27"}]': finished 2024-06-11T13:33:58.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:57 smithi057 ceph-mon[29805]: from='client.26245 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi057-77351-36"}]': finished 2024-06-11T13:33:58.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:57 smithi057 ceph-mon[29805]: osdmap e240: 8 total, 8 up, 8 in 2024-06-11T13:33:58.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:57 smithi057 ceph-mon[40588]: pgmap v327: 324 pgs: 324 active+clean; 584 KiB data, 573 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 3 op/s 2024-06-11T13:33:58.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:57 smithi057 ceph-mon[40588]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:33:58.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:57 smithi057 ceph-mon[40588]: from='client.26242 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi057-77362-46"}]': finished 2024-06-11T13:33:58.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:57 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-27"}]': finished 2024-06-11T13:33:58.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:57 smithi057 ceph-mon[40588]: from='client.26245 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi057-77351-36"}]': finished 2024-06-11T13:33:58.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:57 smithi057 ceph-mon[40588]: osdmap e240: 8 total, 8 up, 8 in 2024-06-11T13:33:58.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:57 smithi107 ceph-mon[33178]: pgmap v327: 324 pgs: 324 active+clean; 584 KiB data, 573 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 3 op/s 2024-06-11T13:33:58.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:57 smithi107 ceph-mon[33178]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:33:58.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:57 smithi107 ceph-mon[33178]: from='client.26242 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi057-77362-46"}]': finished 2024-06-11T13:33:58.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:57 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-27"}]': finished 2024-06-11T13:33:58.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:57 smithi107 ceph-mon[33178]: from='client.26245 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi057-77351-36"}]': finished 2024-06-11T13:33:58.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:57 smithi107 ceph-mon[33178]: osdmap e240: 8 total, 8 up, 8 in 2024-06-11T13:33:59.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:58 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2898188831' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi057-77362-47"}]: dispatch 2024-06-11T13:33:59.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:58 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2898188831' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi057-77362-47"}]: dispatch 2024-06-11T13:33:59.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:58 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2898188831' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi057-77362-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:59.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:58 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2792112744' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi057-77351-37"}]: dispatch 2024-06-11T13:33:59.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:58 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2792112744' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi057-77351-37"}]: dispatch 2024-06-11T13:33:59.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:58 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2792112744' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi057-77351-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:59.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:58 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2898188831' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi057-77362-47"}]: dispatch 2024-06-11T13:33:59.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:58 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2898188831' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi057-77362-47"}]: dispatch 2024-06-11T13:33:59.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:58 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2898188831' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi057-77362-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:59.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:58 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2792112744' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi057-77351-37"}]: dispatch 2024-06-11T13:33:59.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:58 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2792112744' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi057-77351-37"}]: dispatch 2024-06-11T13:33:59.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:58 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2792112744' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi057-77351-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:59.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:58 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2898188831' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi057-77362-47"}]: dispatch 2024-06-11T13:33:59.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:58 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2898188831' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi057-77362-47"}]: dispatch 2024-06-11T13:33:59.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:58 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2898188831' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi057-77362-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:33:59.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:58 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2792112744' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi057-77351-37"}]: dispatch 2024-06-11T13:33:59.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:58 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2792112744' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi057-77351-37"}]: dispatch 2024-06-11T13:33:59.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:58 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2792112744' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi057-77351-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:00.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:59 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2898188831' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi057-77362-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:34:00.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:59 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2792112744' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi057-77351-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:34:00.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:59 smithi057 ceph-mon[29805]: osdmap e241: 8 total, 8 up, 8 in 2024-06-11T13:34:00.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:59 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2898188831' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi057-77362-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi057-77362-47"}]: dispatch 2024-06-11T13:34:00.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:59 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2792112744' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi057-77351-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi057-77351-37"}]: dispatch 2024-06-11T13:34:00.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:33:59 smithi057 ceph-mon[29805]: pgmap v330: 292 pgs: 292 active+clean; 584 KiB data, 573 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-06-11T13:34:00.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:59 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2898188831' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi057-77362-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:34:00.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:59 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2792112744' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi057-77351-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:34:00.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:59 smithi057 ceph-mon[40588]: osdmap e241: 8 total, 8 up, 8 in 2024-06-11T13:34:00.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:59 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2898188831' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi057-77362-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi057-77362-47"}]: dispatch 2024-06-11T13:34:00.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:59 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2792112744' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi057-77351-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi057-77351-37"}]: dispatch 2024-06-11T13:34:00.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:33:59 smithi057 ceph-mon[40588]: pgmap v330: 292 pgs: 292 active+clean; 584 KiB data, 573 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-06-11T13:34:00.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:59 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2898188831' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi057-77362-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:34:00.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:59 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2792112744' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi057-77351-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:34:00.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:59 smithi107 ceph-mon[33178]: osdmap e241: 8 total, 8 up, 8 in 2024-06-11T13:34:00.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:59 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2898188831' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi057-77362-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi057-77362-47"}]: dispatch 2024-06-11T13:34:00.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:59 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2792112744' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi057-77351-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi057-77351-37"}]: dispatch 2024-06-11T13:34:00.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:33:59 smithi107 ceph-mon[33178]: pgmap v330: 292 pgs: 292 active+clean; 584 KiB data, 573 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-06-11T13:34:01.199 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:00 smithi107 ceph-mon[33178]: osdmap e242: 8 total, 8 up, 8 in 2024-06-11T13:34:01.199 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:00 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:34:01.199 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:00 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:34:01.199 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:00 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2898188831' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi057-77362-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi057-77362-47"}]': finished 2024-06-11T13:34:01.199 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:00 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2792112744' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi057-77351-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi057-77351-37"}]': finished 2024-06-11T13:34:01.199 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:00 smithi107 ceph-mon[33178]: osdmap e243: 8 total, 8 up, 8 in 2024-06-11T13:34:01.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:00 smithi057 ceph-mon[29805]: osdmap e242: 8 total, 8 up, 8 in 2024-06-11T13:34:01.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:00 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:34:01.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:00 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:34:01.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:00 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2898188831' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi057-77362-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi057-77362-47"}]': finished 2024-06-11T13:34:01.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:00 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2792112744' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi057-77351-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi057-77351-37"}]': finished 2024-06-11T13:34:01.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:00 smithi057 ceph-mon[29805]: osdmap e243: 8 total, 8 up, 8 in 2024-06-11T13:34:01.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:00 smithi057 ceph-mon[40588]: osdmap e242: 8 total, 8 up, 8 in 2024-06-11T13:34:01.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:00 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:34:01.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:00 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:34:01.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:00 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2898188831' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi057-77362-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi057-77362-47"}]': finished 2024-06-11T13:34:01.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:00 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2792112744' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi057-77351-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi057-77351-37"}]': finished 2024-06-11T13:34:01.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:00 smithi057 ceph-mon[40588]: osdmap e243: 8 total, 8 up, 8 in 2024-06-11T13:34:01.547 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:34:01 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:34:03.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:02 smithi107 ceph-mon[33178]: pgmap v333: 340 pgs: 48 creating+peering, 292 active+clean; 584 KiB data, 574 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-11T13:34:03.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:02 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:34:03.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:02 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:34:03.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:02 smithi107 ceph-mon[33178]: osdmap e244: 8 total, 8 up, 8 in 2024-06-11T13:34:03.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:02 smithi107 ceph-mon[33178]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:34:03.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:02 smithi057 ceph-mon[29805]: pgmap v333: 340 pgs: 48 creating+peering, 292 active+clean; 584 KiB data, 574 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-11T13:34:03.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:02 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:34:03.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:02 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:34:03.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:02 smithi057 ceph-mon[29805]: osdmap e244: 8 total, 8 up, 8 in 2024-06-11T13:34:03.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:02 smithi057 ceph-mon[29805]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:34:03.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:02 smithi057 ceph-mon[40588]: pgmap v333: 340 pgs: 48 creating+peering, 292 active+clean; 584 KiB data, 574 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-11T13:34:03.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:02 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:34:03.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:02 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:34:03.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:02 smithi057 ceph-mon[40588]: osdmap e244: 8 total, 8 up, 8 in 2024-06-11T13:34:03.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:02 smithi057 ceph-mon[40588]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:34:04.034 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:34:03 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:34:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:34:04.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:03 smithi057 ceph-mon[29805]: osdmap e245: 8 total, 8 up, 8 in 2024-06-11T13:34:04.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:03 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2792112744' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi057-77351-37"}]: dispatch 2024-06-11T13:34:04.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:03 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2898188831' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi057-77362-47"}]: dispatch 2024-06-11T13:34:04.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:03 smithi057 ceph-mon[29805]: pgmap v336: 324 pgs: 32 creating+peering, 292 active+clean; 584 KiB data, 574 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:34:04.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:03 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2792112744' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi057-77351-37"}]': finished 2024-06-11T13:34:04.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:03 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2898188831' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi057-77362-47"}]': finished 2024-06-11T13:34:04.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:03 smithi057 ceph-mon[29805]: osdmap e246: 8 total, 8 up, 8 in 2024-06-11T13:34:04.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:03 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2792112744' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi057-77351-37"}]: dispatch 2024-06-11T13:34:04.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:03 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2898188831' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi057-77362-47"}]: dispatch 2024-06-11T13:34:04.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:03 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:34:04.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:03 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:34:04.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:03 smithi057 ceph-mon[40588]: osdmap e245: 8 total, 8 up, 8 in 2024-06-11T13:34:04.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:03 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2792112744' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi057-77351-37"}]: dispatch 2024-06-11T13:34:04.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:03 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2898188831' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi057-77362-47"}]: dispatch 2024-06-11T13:34:04.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:03 smithi057 ceph-mon[40588]: pgmap v336: 324 pgs: 32 creating+peering, 292 active+clean; 584 KiB data, 574 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:34:04.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:03 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2792112744' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi057-77351-37"}]': finished 2024-06-11T13:34:04.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:03 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2898188831' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi057-77362-47"}]': finished 2024-06-11T13:34:04.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:03 smithi057 ceph-mon[40588]: osdmap e246: 8 total, 8 up, 8 in 2024-06-11T13:34:04.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:03 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2792112744' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi057-77351-37"}]: dispatch 2024-06-11T13:34:04.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:03 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2898188831' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi057-77362-47"}]: dispatch 2024-06-11T13:34:04.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:03 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:34:04.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:03 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:34:04.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:03 smithi107 ceph-mon[33178]: osdmap e245: 8 total, 8 up, 8 in 2024-06-11T13:34:04.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:03 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2792112744' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi057-77351-37"}]: dispatch 2024-06-11T13:34:04.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:03 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2898188831' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi057-77362-47"}]: dispatch 2024-06-11T13:34:04.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:03 smithi107 ceph-mon[33178]: pgmap v336: 324 pgs: 32 creating+peering, 292 active+clean; 584 KiB data, 574 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:34:04.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:03 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2792112744' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi057-77351-37"}]': finished 2024-06-11T13:34:04.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:03 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2898188831' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi057-77362-47"}]': finished 2024-06-11T13:34:04.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:03 smithi107 ceph-mon[33178]: osdmap e246: 8 total, 8 up, 8 in 2024-06-11T13:34:04.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:03 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2792112744' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi057-77351-37"}]: dispatch 2024-06-11T13:34:04.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:03 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2898188831' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi057-77362-47"}]: dispatch 2024-06-11T13:34:04.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:03 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:34:04.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:03 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:34:06.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:05 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2792112744' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi057-77351-37"}]': finished 2024-06-11T13:34:06.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:05 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2898188831' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi057-77362-47"}]': finished 2024-06-11T13:34:06.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:05 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-29", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:34:06.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:05 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-29"}]: dispatch 2024-06-11T13:34:06.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:05 smithi107 ceph-mon[33178]: osdmap e247: 8 total, 8 up, 8 in 2024-06-11T13:34:06.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:05 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-29"}]: dispatch 2024-06-11T13:34:06.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:05 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3140173519' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi057-77351-38"}]: dispatch 2024-06-11T13:34:06.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:05 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/10240340' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi057-77362-48"}]: dispatch 2024-06-11T13:34:06.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:05 smithi107 ceph-mon[33178]: from='client.36275 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi057-77362-48"}]: dispatch 2024-06-11T13:34:06.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:05 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3140173519' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi057-77351-38"}]: dispatch 2024-06-11T13:34:06.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:05 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/10240340' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi057-77362-48"}]: dispatch 2024-06-11T13:34:06.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:05 smithi107 ceph-mon[33178]: from='client.36275 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi057-77362-48"}]: dispatch 2024-06-11T13:34:06.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:05 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3140173519' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi057-77351-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:06.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:05 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/10240340' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi057-77362-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:06.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:05 smithi107 ceph-mon[33178]: from='client.36275 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi057-77362-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:06.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:05 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2792112744' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi057-77351-37"}]': finished 2024-06-11T13:34:06.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:05 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2898188831' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi057-77362-47"}]': finished 2024-06-11T13:34:06.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:05 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-29", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:34:06.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:05 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-29"}]: dispatch 2024-06-11T13:34:06.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:05 smithi057 ceph-mon[29805]: osdmap e247: 8 total, 8 up, 8 in 2024-06-11T13:34:06.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:05 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-29"}]: dispatch 2024-06-11T13:34:06.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:05 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3140173519' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi057-77351-38"}]: dispatch 2024-06-11T13:34:06.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:05 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/10240340' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi057-77362-48"}]: dispatch 2024-06-11T13:34:06.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:05 smithi057 ceph-mon[29805]: from='client.36275 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi057-77362-48"}]: dispatch 2024-06-11T13:34:06.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:05 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3140173519' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi057-77351-38"}]: dispatch 2024-06-11T13:34:06.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:05 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/10240340' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi057-77362-48"}]: dispatch 2024-06-11T13:34:06.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:05 smithi057 ceph-mon[29805]: from='client.36275 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi057-77362-48"}]: dispatch 2024-06-11T13:34:06.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:05 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3140173519' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi057-77351-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:06.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:05 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/10240340' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi057-77362-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:06.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:05 smithi057 ceph-mon[29805]: from='client.36275 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi057-77362-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:06.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:05 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2792112744' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi057-77351-37"}]': finished 2024-06-11T13:34:06.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:05 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2898188831' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi057-77362-47"}]': finished 2024-06-11T13:34:06.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:05 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-29", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:34:06.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:05 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-29"}]: dispatch 2024-06-11T13:34:06.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:05 smithi057 ceph-mon[40588]: osdmap e247: 8 total, 8 up, 8 in 2024-06-11T13:34:06.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:05 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-29"}]: dispatch 2024-06-11T13:34:06.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:05 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3140173519' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi057-77351-38"}]: dispatch 2024-06-11T13:34:06.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:05 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/10240340' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi057-77362-48"}]: dispatch 2024-06-11T13:34:06.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:05 smithi057 ceph-mon[40588]: from='client.36275 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi057-77362-48"}]: dispatch 2024-06-11T13:34:06.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:05 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3140173519' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi057-77351-38"}]: dispatch 2024-06-11T13:34:06.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:05 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/10240340' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi057-77362-48"}]: dispatch 2024-06-11T13:34:06.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:05 smithi057 ceph-mon[40588]: from='client.36275 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi057-77362-48"}]: dispatch 2024-06-11T13:34:06.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:05 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3140173519' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi057-77351-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:06.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:05 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/10240340' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi057-77362-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:06.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:05 smithi057 ceph-mon[40588]: from='client.36275 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi057-77362-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:08.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:07 smithi057 ceph-mon[29805]: pgmap v339: 324 pgs: 32 creating+peering, 292 active+clean; 584 KiB data, 574 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:34:08.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:07 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-29"}]': finished 2024-06-11T13:34:08.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:07 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3140173519' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi057-77351-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:34:08.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:07 smithi057 ceph-mon[29805]: from='client.36275 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi057-77362-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:34:08.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:07 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-29", "mode": "writeback"}]: dispatch 2024-06-11T13:34:08.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:07 smithi057 ceph-mon[29805]: osdmap e248: 8 total, 8 up, 8 in 2024-06-11T13:34:08.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:07 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/10240340' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi057-77362-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi057-77362-48"}]: dispatch 2024-06-11T13:34:08.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:07 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3140173519' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi057-77351-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi057-77351-38"}]: dispatch 2024-06-11T13:34:08.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:07 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-29", "mode": "writeback"}]: dispatch 2024-06-11T13:34:08.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:07 smithi057 ceph-mon[29805]: from='client.36275 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi057-77362-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi057-77362-48"}]: dispatch 2024-06-11T13:34:08.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:07 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:34:08.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:07 smithi057 ceph-mon[40588]: pgmap v339: 324 pgs: 32 creating+peering, 292 active+clean; 584 KiB data, 574 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:34:08.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:07 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-29"}]': finished 2024-06-11T13:34:08.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:07 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3140173519' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi057-77351-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:34:08.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:07 smithi057 ceph-mon[40588]: from='client.36275 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi057-77362-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:34:08.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:07 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-29", "mode": "writeback"}]: dispatch 2024-06-11T13:34:08.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:07 smithi057 ceph-mon[40588]: osdmap e248: 8 total, 8 up, 8 in 2024-06-11T13:34:08.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:07 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/10240340' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi057-77362-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi057-77362-48"}]: dispatch 2024-06-11T13:34:08.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:07 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3140173519' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi057-77351-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi057-77351-38"}]: dispatch 2024-06-11T13:34:08.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:07 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-29", "mode": "writeback"}]: dispatch 2024-06-11T13:34:08.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:07 smithi057 ceph-mon[40588]: from='client.36275 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi057-77362-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi057-77362-48"}]: dispatch 2024-06-11T13:34:08.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:07 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:34:08.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:07 smithi107 ceph-mon[33178]: pgmap v339: 324 pgs: 32 creating+peering, 292 active+clean; 584 KiB data, 574 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:34:08.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:07 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-29"}]': finished 2024-06-11T13:34:08.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:07 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3140173519' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi057-77351-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:34:08.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:07 smithi107 ceph-mon[33178]: from='client.36275 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi057-77362-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:34:08.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:07 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-29", "mode": "writeback"}]: dispatch 2024-06-11T13:34:08.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:07 smithi107 ceph-mon[33178]: osdmap e248: 8 total, 8 up, 8 in 2024-06-11T13:34:08.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:07 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/10240340' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi057-77362-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi057-77362-48"}]: dispatch 2024-06-11T13:34:08.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:07 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3140173519' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi057-77351-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi057-77351-38"}]: dispatch 2024-06-11T13:34:08.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:07 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-29", "mode": "writeback"}]: dispatch 2024-06-11T13:34:08.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:07 smithi107 ceph-mon[33178]: from='client.36275 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi057-77362-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi057-77362-48"}]: dispatch 2024-06-11T13:34:08.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:07 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:34:09.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:08 smithi057 ceph-mon[40588]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:34:09.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:08 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-29", "mode": "writeback"}]': finished 2024-06-11T13:34:09.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:08 smithi057 ceph-mon[40588]: pgmap v342: 324 pgs: 324 active+clean; 4.6 MiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-06-11T13:34:09.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:08 smithi057 ceph-mon[40588]: osdmap e249: 8 total, 8 up, 8 in 2024-06-11T13:34:09.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:08 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3140173519' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi057-77351-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi057-77351-38"}]': finished 2024-06-11T13:34:09.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:08 smithi057 ceph-mon[40588]: from='client.36275 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi057-77362-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi057-77362-48"}]': finished 2024-06-11T13:34:09.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:08 smithi057 ceph-mon[40588]: osdmap e250: 8 total, 8 up, 8 in 2024-06-11T13:34:09.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:08 smithi057 ceph-mon[29805]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:34:09.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:08 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-29", "mode": "writeback"}]': finished 2024-06-11T13:34:09.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:08 smithi057 ceph-mon[29805]: pgmap v342: 324 pgs: 324 active+clean; 4.6 MiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-06-11T13:34:09.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:08 smithi057 ceph-mon[29805]: osdmap e249: 8 total, 8 up, 8 in 2024-06-11T13:34:09.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:08 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3140173519' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi057-77351-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi057-77351-38"}]': finished 2024-06-11T13:34:09.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:08 smithi057 ceph-mon[29805]: from='client.36275 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi057-77362-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi057-77362-48"}]': finished 2024-06-11T13:34:09.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:08 smithi057 ceph-mon[29805]: osdmap e250: 8 total, 8 up, 8 in 2024-06-11T13:34:09.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:08 smithi107 ceph-mon[33178]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:34:09.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:08 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-29", "mode": "writeback"}]': finished 2024-06-11T13:34:09.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:08 smithi107 ceph-mon[33178]: pgmap v342: 324 pgs: 324 active+clean; 4.6 MiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-06-11T13:34:09.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:08 smithi107 ceph-mon[33178]: osdmap e249: 8 total, 8 up, 8 in 2024-06-11T13:34:09.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:08 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3140173519' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi057-77351-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi057-77351-38"}]': finished 2024-06-11T13:34:09.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:08 smithi107 ceph-mon[33178]: from='client.36275 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi057-77362-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi057-77362-48"}]': finished 2024-06-11T13:34:09.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:08 smithi107 ceph-mon[33178]: osdmap e250: 8 total, 8 up, 8 in 2024-06-11T13:34:10.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:09 smithi057 ceph-mon[40588]: osdmap e251: 8 total, 8 up, 8 in 2024-06-11T13:34:10.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:09 smithi057 ceph-mon[40588]: pgmap v345: 340 pgs: 16 unknown, 324 active+clean; 4.6 MiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-06-11T13:34:10.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:09 smithi057 ceph-mon[29805]: osdmap e251: 8 total, 8 up, 8 in 2024-06-11T13:34:10.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:09 smithi057 ceph-mon[29805]: pgmap v345: 340 pgs: 16 unknown, 324 active+clean; 4.6 MiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-06-11T13:34:10.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:09 smithi107 ceph-mon[33178]: osdmap e251: 8 total, 8 up, 8 in 2024-06-11T13:34:10.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:09 smithi107 ceph-mon[33178]: pgmap v345: 340 pgs: 16 unknown, 324 active+clean; 4.6 MiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-06-11T13:34:11.274 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:10 smithi107 ceph-mon[33178]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:34:11.275 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:10 smithi107 ceph-mon[33178]: osdmap e252: 8 total, 8 up, 8 in 2024-06-11T13:34:11.275 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:10 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3140173519' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi057-77351-38"}]: dispatch 2024-06-11T13:34:11.275 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:10 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/10240340' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi057-77362-48"}]: dispatch 2024-06-11T13:34:11.275 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:10 smithi107 ceph-mon[33178]: from='client.36275 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi057-77362-48"}]: dispatch 2024-06-11T13:34:11.275 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:10 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:34:11.275 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:10 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:34:11.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:10 smithi057 ceph-mon[40588]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:34:11.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:10 smithi057 ceph-mon[40588]: osdmap e252: 8 total, 8 up, 8 in 2024-06-11T13:34:11.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:10 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3140173519' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi057-77351-38"}]: dispatch 2024-06-11T13:34:11.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:10 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/10240340' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi057-77362-48"}]: dispatch 2024-06-11T13:34:11.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:10 smithi057 ceph-mon[40588]: from='client.36275 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi057-77362-48"}]: dispatch 2024-06-11T13:34:11.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:10 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:34:11.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:10 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:34:11.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:10 smithi057 ceph-mon[29805]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:34:11.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:10 smithi057 ceph-mon[29805]: osdmap e252: 8 total, 8 up, 8 in 2024-06-11T13:34:11.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:10 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3140173519' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi057-77351-38"}]: dispatch 2024-06-11T13:34:11.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:10 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/10240340' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi057-77362-48"}]: dispatch 2024-06-11T13:34:11.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:10 smithi057 ceph-mon[29805]: from='client.36275 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi057-77362-48"}]: dispatch 2024-06-11T13:34:11.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:10 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:34:11.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:10 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:34:11.547 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:34:11 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:34:12.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:11 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3140173519' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi057-77351-38"}]': finished 2024-06-11T13:34:12.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:11 smithi057 ceph-mon[29805]: from='client.36275 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi057-77362-48"}]': finished 2024-06-11T13:34:12.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:11 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:34:12.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:11 smithi057 ceph-mon[29805]: osdmap e253: 8 total, 8 up, 8 in 2024-06-11T13:34:12.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:11 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-29"}]: dispatch 2024-06-11T13:34:12.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:11 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3140173519' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi057-77351-38"}]: dispatch 2024-06-11T13:34:12.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:11 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/10240340' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi057-77362-48"}]: dispatch 2024-06-11T13:34:12.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:11 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-29"}]: dispatch 2024-06-11T13:34:12.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:11 smithi057 ceph-mon[29805]: from='client.36275 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi057-77362-48"}]: dispatch 2024-06-11T13:34:12.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:11 smithi057 ceph-mon[29805]: pgmap v348: 324 pgs: 324 active+clean; 4.6 MiB data, 595 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 1024 KiB/s wr, 27 op/s 2024-06-11T13:34:12.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:11 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:34:12.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:11 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3140173519' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi057-77351-38"}]': finished 2024-06-11T13:34:12.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:11 smithi057 ceph-mon[40588]: from='client.36275 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi057-77362-48"}]': finished 2024-06-11T13:34:12.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:11 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:34:12.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:11 smithi057 ceph-mon[40588]: osdmap e253: 8 total, 8 up, 8 in 2024-06-11T13:34:12.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:11 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-29"}]: dispatch 2024-06-11T13:34:12.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:11 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3140173519' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi057-77351-38"}]: dispatch 2024-06-11T13:34:12.288 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:11 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/10240340' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi057-77362-48"}]: dispatch 2024-06-11T13:34:12.288 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:11 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-29"}]: dispatch 2024-06-11T13:34:12.288 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:11 smithi057 ceph-mon[40588]: from='client.36275 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi057-77362-48"}]: dispatch 2024-06-11T13:34:12.288 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:11 smithi057 ceph-mon[40588]: pgmap v348: 324 pgs: 324 active+clean; 4.6 MiB data, 595 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 1024 KiB/s wr, 27 op/s 2024-06-11T13:34:12.288 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:11 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:34:12.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:11 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3140173519' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi057-77351-38"}]': finished 2024-06-11T13:34:12.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:11 smithi107 ceph-mon[33178]: from='client.36275 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi057-77362-48"}]': finished 2024-06-11T13:34:12.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:11 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:34:12.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:11 smithi107 ceph-mon[33178]: osdmap e253: 8 total, 8 up, 8 in 2024-06-11T13:34:12.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:11 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-29"}]: dispatch 2024-06-11T13:34:12.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:11 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3140173519' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi057-77351-38"}]: dispatch 2024-06-11T13:34:12.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:11 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/10240340' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi057-77362-48"}]: dispatch 2024-06-11T13:34:12.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:11 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-29"}]: dispatch 2024-06-11T13:34:12.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:11 smithi107 ceph-mon[33178]: from='client.36275 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi057-77362-48"}]: dispatch 2024-06-11T13:34:12.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:11 smithi107 ceph-mon[33178]: pgmap v348: 324 pgs: 324 active+clean; 4.6 MiB data, 595 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 1024 KiB/s wr, 27 op/s 2024-06-11T13:34:12.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:11 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:34:13.733 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:13 smithi057 ceph-mon[29805]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:34:13.733 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:13 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3140173519' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi057-77351-38"}]': finished 2024-06-11T13:34:13.733 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:13 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-29"}]': finished 2024-06-11T13:34:13.733 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:13 smithi057 ceph-mon[29805]: from='client.36275 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi057-77362-48"}]': finished 2024-06-11T13:34:13.733 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:13 smithi057 ceph-mon[29805]: osdmap e254: 8 total, 8 up, 8 in 2024-06-11T13:34:13.733 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:13 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2484651234' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi057-77362-49"}]: dispatch 2024-06-11T13:34:13.733 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:13 smithi057 ceph-mon[29805]: from='client.36287 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi057-77362-49"}]: dispatch 2024-06-11T13:34:13.733 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:13 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2931032504' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi057-77351-39"}]: dispatch 2024-06-11T13:34:13.733 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:13 smithi057 ceph-mon[29805]: from='client.36284 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi057-77351-39"}]: dispatch 2024-06-11T13:34:13.734 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:13 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2484651234' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi057-77362-49"}]: dispatch 2024-06-11T13:34:13.734 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:13 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2931032504' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi057-77351-39"}]: dispatch 2024-06-11T13:34:13.734 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:13 smithi057 ceph-mon[29805]: from='client.36287 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi057-77362-49"}]: dispatch 2024-06-11T13:34:13.734 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:13 smithi057 ceph-mon[29805]: from='client.36284 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi057-77351-39"}]: dispatch 2024-06-11T13:34:13.734 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:13 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2931032504' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi057-77351-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:13.734 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:13 smithi057 ceph-mon[29805]: from='client.36284 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi057-77351-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:13.734 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:13 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2484651234' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi057-77362-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:13.734 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:13 smithi057 ceph-mon[29805]: from='client.36287 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi057-77362-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:13.734 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:13 smithi057 ceph-mon[40588]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:34:13.734 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:13 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3140173519' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi057-77351-38"}]': finished 2024-06-11T13:34:13.734 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:13 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-29"}]': finished 2024-06-11T13:34:13.735 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:13 smithi057 ceph-mon[40588]: from='client.36275 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi057-77362-48"}]': finished 2024-06-11T13:34:13.735 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:13 smithi057 ceph-mon[40588]: osdmap e254: 8 total, 8 up, 8 in 2024-06-11T13:34:13.735 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:13 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2484651234' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi057-77362-49"}]: dispatch 2024-06-11T13:34:13.735 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:13 smithi057 ceph-mon[40588]: from='client.36287 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi057-77362-49"}]: dispatch 2024-06-11T13:34:13.735 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:13 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2931032504' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi057-77351-39"}]: dispatch 2024-06-11T13:34:13.735 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:13 smithi057 ceph-mon[40588]: from='client.36284 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi057-77351-39"}]: dispatch 2024-06-11T13:34:13.735 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:13 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2484651234' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi057-77362-49"}]: dispatch 2024-06-11T13:34:13.735 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:13 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2931032504' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi057-77351-39"}]: dispatch 2024-06-11T13:34:13.735 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:13 smithi057 ceph-mon[40588]: from='client.36287 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi057-77362-49"}]: dispatch 2024-06-11T13:34:13.735 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:13 smithi057 ceph-mon[40588]: from='client.36284 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi057-77351-39"}]: dispatch 2024-06-11T13:34:13.735 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:13 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2931032504' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi057-77351-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:13.735 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:13 smithi057 ceph-mon[40588]: from='client.36284 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi057-77351-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:13.736 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:13 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2484651234' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi057-77362-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:13.736 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:13 smithi057 ceph-mon[40588]: from='client.36287 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi057-77362-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:13.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:13 smithi107 ceph-mon[33178]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:34:13.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:13 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3140173519' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi057-77351-38"}]': finished 2024-06-11T13:34:13.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:13 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-29"}]': finished 2024-06-11T13:34:13.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:13 smithi107 ceph-mon[33178]: from='client.36275 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi057-77362-48"}]': finished 2024-06-11T13:34:13.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:13 smithi107 ceph-mon[33178]: osdmap e254: 8 total, 8 up, 8 in 2024-06-11T13:34:13.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:13 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2484651234' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi057-77362-49"}]: dispatch 2024-06-11T13:34:13.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:13 smithi107 ceph-mon[33178]: from='client.36287 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi057-77362-49"}]: dispatch 2024-06-11T13:34:13.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:13 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2931032504' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi057-77351-39"}]: dispatch 2024-06-11T13:34:13.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:13 smithi107 ceph-mon[33178]: from='client.36284 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi057-77351-39"}]: dispatch 2024-06-11T13:34:13.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:13 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2484651234' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi057-77362-49"}]: dispatch 2024-06-11T13:34:13.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:13 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2931032504' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi057-77351-39"}]: dispatch 2024-06-11T13:34:13.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:13 smithi107 ceph-mon[33178]: from='client.36287 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi057-77362-49"}]: dispatch 2024-06-11T13:34:13.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:13 smithi107 ceph-mon[33178]: from='client.36284 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi057-77351-39"}]: dispatch 2024-06-11T13:34:13.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:13 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2931032504' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi057-77351-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:13.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:13 smithi107 ceph-mon[33178]: from='client.36284 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi057-77351-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:13.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:13 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2484651234' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi057-77362-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:13.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:13 smithi107 ceph-mon[33178]: from='client.36287 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi057-77362-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:14.034 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:34:13 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:34:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:34:15.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:14 smithi057 ceph-mon[29805]: from='client.36284 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi057-77351-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:34:15.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:14 smithi057 ceph-mon[29805]: from='client.36287 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi057-77362-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:34:15.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:14 smithi057 ceph-mon[29805]: osdmap e255: 8 total, 8 up, 8 in 2024-06-11T13:34:15.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:14 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2931032504' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi057-77351-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi057-77351-39"}]: dispatch 2024-06-11T13:34:15.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:14 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2484651234' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi057-77362-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi057-77362-49"}]: dispatch 2024-06-11T13:34:15.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:14 smithi057 ceph-mon[29805]: pgmap v351: 292 pgs: 292 active+clean; 4.6 MiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-06-11T13:34:15.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:14 smithi057 ceph-mon[29805]: from='client.36284 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi057-77351-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi057-77351-39"}]: dispatch 2024-06-11T13:34:15.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:14 smithi057 ceph-mon[29805]: from='client.36287 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi057-77362-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi057-77362-49"}]: dispatch 2024-06-11T13:34:15.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:14 smithi057 ceph-mon[40588]: from='client.36284 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi057-77351-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:34:15.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:14 smithi057 ceph-mon[40588]: from='client.36287 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi057-77362-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:34:15.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:14 smithi057 ceph-mon[40588]: osdmap e255: 8 total, 8 up, 8 in 2024-06-11T13:34:15.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:14 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2931032504' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi057-77351-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi057-77351-39"}]: dispatch 2024-06-11T13:34:15.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:14 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2484651234' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi057-77362-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi057-77362-49"}]: dispatch 2024-06-11T13:34:15.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:14 smithi057 ceph-mon[40588]: pgmap v351: 292 pgs: 292 active+clean; 4.6 MiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-06-11T13:34:15.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:14 smithi057 ceph-mon[40588]: from='client.36284 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi057-77351-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi057-77351-39"}]: dispatch 2024-06-11T13:34:15.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:14 smithi057 ceph-mon[40588]: from='client.36287 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi057-77362-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi057-77362-49"}]: dispatch 2024-06-11T13:34:15.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:14 smithi107 ceph-mon[33178]: from='client.36284 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi057-77351-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:34:15.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:14 smithi107 ceph-mon[33178]: from='client.36287 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi057-77362-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:34:15.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:14 smithi107 ceph-mon[33178]: osdmap e255: 8 total, 8 up, 8 in 2024-06-11T13:34:15.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:14 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2931032504' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi057-77351-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi057-77351-39"}]: dispatch 2024-06-11T13:34:15.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:14 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2484651234' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi057-77362-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi057-77362-49"}]: dispatch 2024-06-11T13:34:15.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:14 smithi107 ceph-mon[33178]: pgmap v351: 292 pgs: 292 active+clean; 4.6 MiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-06-11T13:34:15.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:14 smithi107 ceph-mon[33178]: from='client.36284 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi057-77351-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi057-77351-39"}]: dispatch 2024-06-11T13:34:15.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:14 smithi107 ceph-mon[33178]: from='client.36287 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi057-77362-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi057-77362-49"}]: dispatch 2024-06-11T13:34:16.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:15 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:34:16.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:15 smithi057 ceph-mon[29805]: osdmap e256: 8 total, 8 up, 8 in 2024-06-11T13:34:16.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:15 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:34:16.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:15 smithi057 ceph-mon[29805]: from='client.36284 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi057-77351-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi057-77351-39"}]': finished 2024-06-11T13:34:16.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:15 smithi057 ceph-mon[29805]: from='client.36287 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi057-77362-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi057-77362-49"}]': finished 2024-06-11T13:34:16.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:15 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:34:16.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:15 smithi057 ceph-mon[29805]: osdmap e257: 8 total, 8 up, 8 in 2024-06-11T13:34:16.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:15 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:34:16.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:15 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:34:16.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:15 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:34:16.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:15 smithi057 ceph-mon[40588]: osdmap e256: 8 total, 8 up, 8 in 2024-06-11T13:34:16.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:15 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:34:16.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:15 smithi057 ceph-mon[40588]: from='client.36284 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi057-77351-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi057-77351-39"}]': finished 2024-06-11T13:34:16.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:15 smithi057 ceph-mon[40588]: from='client.36287 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi057-77362-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi057-77362-49"}]': finished 2024-06-11T13:34:16.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:15 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:34:16.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:15 smithi057 ceph-mon[40588]: osdmap e257: 8 total, 8 up, 8 in 2024-06-11T13:34:16.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:15 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:34:16.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:15 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:34:16.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:15 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:34:16.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:15 smithi107 ceph-mon[33178]: osdmap e256: 8 total, 8 up, 8 in 2024-06-11T13:34:16.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:15 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:34:16.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:15 smithi107 ceph-mon[33178]: from='client.36284 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi057-77351-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi057-77351-39"}]': finished 2024-06-11T13:34:16.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:15 smithi107 ceph-mon[33178]: from='client.36287 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi057-77362-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi057-77362-49"}]': finished 2024-06-11T13:34:16.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:15 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:34:16.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:15 smithi107 ceph-mon[33178]: osdmap e257: 8 total, 8 up, 8 in 2024-06-11T13:34:16.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:15 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:34:16.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:15 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:34:17.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:16 smithi057 ceph-mon[29805]: pgmap v354: 340 pgs: 48 unknown, 292 active+clean; 4.6 MiB data, 595 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:34:17.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:16 smithi057 ceph-mon[29805]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:34:17.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:16 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-31", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:34:17.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:16 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-31"}]: dispatch 2024-06-11T13:34:17.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:16 smithi057 ceph-mon[29805]: osdmap e258: 8 total, 8 up, 8 in 2024-06-11T13:34:17.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:16 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-31"}]: dispatch 2024-06-11T13:34:17.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:16 smithi057 ceph-mon[40588]: pgmap v354: 340 pgs: 48 unknown, 292 active+clean; 4.6 MiB data, 595 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:34:17.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:16 smithi057 ceph-mon[40588]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:34:17.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:16 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-31", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:34:17.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:16 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-31"}]: dispatch 2024-06-11T13:34:17.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:16 smithi057 ceph-mon[40588]: osdmap e258: 8 total, 8 up, 8 in 2024-06-11T13:34:17.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:16 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-31"}]: dispatch 2024-06-11T13:34:17.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:16 smithi107 ceph-mon[33178]: pgmap v354: 340 pgs: 48 unknown, 292 active+clean; 4.6 MiB data, 595 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:34:17.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:16 smithi107 ceph-mon[33178]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:34:17.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:16 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-31", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:34:17.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:16 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-31"}]: dispatch 2024-06-11T13:34:17.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:16 smithi107 ceph-mon[33178]: osdmap e258: 8 total, 8 up, 8 in 2024-06-11T13:34:17.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:16 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-31"}]: dispatch 2024-06-11T13:34:18.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:17 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-31"}]': finished 2024-06-11T13:34:18.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:17 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-31", "mode": "writeback"}]: dispatch 2024-06-11T13:34:18.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:17 smithi057 ceph-mon[29805]: osdmap e259: 8 total, 8 up, 8 in 2024-06-11T13:34:18.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:17 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-31", "mode": "writeback"}]: dispatch 2024-06-11T13:34:18.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:17 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2931032504' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi057-77351-39"}]: dispatch 2024-06-11T13:34:18.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:17 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2484651234' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi057-77362-49"}]: dispatch 2024-06-11T13:34:18.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:17 smithi057 ceph-mon[29805]: from='client.36284 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi057-77351-39"}]: dispatch 2024-06-11T13:34:18.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:17 smithi057 ceph-mon[29805]: from='client.36287 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi057-77362-49"}]: dispatch 2024-06-11T13:34:18.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:17 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-31"}]': finished 2024-06-11T13:34:18.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:17 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-31", "mode": "writeback"}]: dispatch 2024-06-11T13:34:18.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:17 smithi057 ceph-mon[40588]: osdmap e259: 8 total, 8 up, 8 in 2024-06-11T13:34:18.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:17 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-31", "mode": "writeback"}]: dispatch 2024-06-11T13:34:18.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:17 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2931032504' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi057-77351-39"}]: dispatch 2024-06-11T13:34:18.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:17 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2484651234' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi057-77362-49"}]: dispatch 2024-06-11T13:34:18.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:17 smithi057 ceph-mon[40588]: from='client.36284 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi057-77351-39"}]: dispatch 2024-06-11T13:34:18.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:17 smithi057 ceph-mon[40588]: from='client.36287 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi057-77362-49"}]: dispatch 2024-06-11T13:34:18.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:17 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-31"}]': finished 2024-06-11T13:34:18.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:17 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-31", "mode": "writeback"}]: dispatch 2024-06-11T13:34:18.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:17 smithi107 ceph-mon[33178]: osdmap e259: 8 total, 8 up, 8 in 2024-06-11T13:34:18.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:17 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-31", "mode": "writeback"}]: dispatch 2024-06-11T13:34:18.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:17 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2931032504' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi057-77351-39"}]: dispatch 2024-06-11T13:34:18.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:17 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2484651234' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi057-77362-49"}]: dispatch 2024-06-11T13:34:18.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:17 smithi107 ceph-mon[33178]: from='client.36284 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi057-77351-39"}]: dispatch 2024-06-11T13:34:18.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:17 smithi107 ceph-mon[33178]: from='client.36287 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi057-77362-49"}]: dispatch 2024-06-11T13:34:19.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:18 smithi107 ceph-mon[33178]: pgmap v357: 324 pgs: 324 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:34:19.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:18 smithi107 ceph-mon[33178]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:34:19.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:18 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-31", "mode": "writeback"}]': finished 2024-06-11T13:34:19.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:18 smithi107 ceph-mon[33178]: from='client.36284 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi057-77351-39"}]': finished 2024-06-11T13:34:19.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:18 smithi107 ceph-mon[33178]: from='client.36287 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi057-77362-49"}]': finished 2024-06-11T13:34:19.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:18 smithi107 ceph-mon[33178]: osdmap e260: 8 total, 8 up, 8 in 2024-06-11T13:34:19.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:18 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2931032504' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi057-77351-39"}]: dispatch 2024-06-11T13:34:19.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:18 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2484651234' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi057-77362-49"}]: dispatch 2024-06-11T13:34:19.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:18 smithi107 ceph-mon[33178]: from='client.36284 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi057-77351-39"}]: dispatch 2024-06-11T13:34:19.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:18 smithi107 ceph-mon[33178]: from='client.36287 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi057-77362-49"}]: dispatch 2024-06-11T13:34:19.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:18 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:34:19.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:18 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:34:19.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:18 smithi057 ceph-mon[29805]: pgmap v357: 324 pgs: 324 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:34:19.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:18 smithi057 ceph-mon[29805]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:34:19.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:18 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-31", "mode": "writeback"}]': finished 2024-06-11T13:34:19.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:18 smithi057 ceph-mon[29805]: from='client.36284 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi057-77351-39"}]': finished 2024-06-11T13:34:19.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:18 smithi057 ceph-mon[29805]: from='client.36287 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi057-77362-49"}]': finished 2024-06-11T13:34:19.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:18 smithi057 ceph-mon[29805]: osdmap e260: 8 total, 8 up, 8 in 2024-06-11T13:34:19.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:18 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2931032504' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi057-77351-39"}]: dispatch 2024-06-11T13:34:19.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:18 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2484651234' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi057-77362-49"}]: dispatch 2024-06-11T13:34:19.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:18 smithi057 ceph-mon[29805]: from='client.36284 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi057-77351-39"}]: dispatch 2024-06-11T13:34:19.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:18 smithi057 ceph-mon[29805]: from='client.36287 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi057-77362-49"}]: dispatch 2024-06-11T13:34:19.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:18 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:34:19.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:18 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:34:19.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:18 smithi057 ceph-mon[40588]: pgmap v357: 324 pgs: 324 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:34:19.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:18 smithi057 ceph-mon[40588]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:34:19.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:18 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-31", "mode": "writeback"}]': finished 2024-06-11T13:34:19.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:18 smithi057 ceph-mon[40588]: from='client.36284 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi057-77351-39"}]': finished 2024-06-11T13:34:19.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:18 smithi057 ceph-mon[40588]: from='client.36287 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi057-77362-49"}]': finished 2024-06-11T13:34:19.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:18 smithi057 ceph-mon[40588]: osdmap e260: 8 total, 8 up, 8 in 2024-06-11T13:34:19.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:18 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2931032504' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi057-77351-39"}]: dispatch 2024-06-11T13:34:19.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:18 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2484651234' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi057-77362-49"}]: dispatch 2024-06-11T13:34:19.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:18 smithi057 ceph-mon[40588]: from='client.36284 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi057-77351-39"}]: dispatch 2024-06-11T13:34:19.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:18 smithi057 ceph-mon[40588]: from='client.36287 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi057-77362-49"}]: dispatch 2024-06-11T13:34:19.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:18 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:34:19.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:18 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:34:20.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:19 smithi057 ceph-mon[29805]: from='client.36284 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi057-77351-39"}]': finished 2024-06-11T13:34:20.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:19 smithi057 ceph-mon[29805]: from='client.36287 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi057-77362-49"}]': finished 2024-06-11T13:34:20.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:19 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:34:20.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:19 smithi057 ceph-mon[29805]: osdmap e261: 8 total, 8 up, 8 in 2024-06-11T13:34:20.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:19 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-31"}]: dispatch 2024-06-11T13:34:20.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:19 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-31"}]: dispatch 2024-06-11T13:34:20.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:19 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/239651899' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi057-77351-40"}]: dispatch 2024-06-11T13:34:20.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:19 smithi057 ceph-mon[29805]: from='client.36296 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi057-77351-40"}]: dispatch 2024-06-11T13:34:20.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:19 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/61824103' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi057-77362-50"}]: dispatch 2024-06-11T13:34:20.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:19 smithi057 ceph-mon[29805]: from='client.26293 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi057-77362-50"}]: dispatch 2024-06-11T13:34:20.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:19 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/239651899' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi057-77351-40"}]: dispatch 2024-06-11T13:34:20.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:19 smithi057 ceph-mon[29805]: from='client.36296 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi057-77351-40"}]: dispatch 2024-06-11T13:34:20.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:19 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/239651899' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi057-77351-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:20.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:19 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/61824103' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi057-77362-50"}]: dispatch 2024-06-11T13:34:20.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:19 smithi057 ceph-mon[29805]: from='client.36296 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi057-77351-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:20.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:19 smithi057 ceph-mon[29805]: from='client.26293 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi057-77362-50"}]: dispatch 2024-06-11T13:34:20.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:19 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/61824103' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi057-77362-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:20.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:19 smithi057 ceph-mon[29805]: from='client.26293 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi057-77362-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:20.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:19 smithi057 ceph-mon[29805]: pgmap v360: 324 pgs: 324 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:34:20.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:19 smithi057 ceph-mon[40588]: from='client.36284 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi057-77351-39"}]': finished 2024-06-11T13:34:20.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:19 smithi057 ceph-mon[40588]: from='client.36287 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi057-77362-49"}]': finished 2024-06-11T13:34:20.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:19 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:34:20.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:19 smithi057 ceph-mon[40588]: osdmap e261: 8 total, 8 up, 8 in 2024-06-11T13:34:20.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:19 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-31"}]: dispatch 2024-06-11T13:34:20.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:19 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-31"}]: dispatch 2024-06-11T13:34:20.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:19 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/239651899' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi057-77351-40"}]: dispatch 2024-06-11T13:34:20.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:19 smithi057 ceph-mon[40588]: from='client.36296 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi057-77351-40"}]: dispatch 2024-06-11T13:34:20.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:19 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/61824103' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi057-77362-50"}]: dispatch 2024-06-11T13:34:20.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:19 smithi057 ceph-mon[40588]: from='client.26293 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi057-77362-50"}]: dispatch 2024-06-11T13:34:20.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:19 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/239651899' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi057-77351-40"}]: dispatch 2024-06-11T13:34:20.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:19 smithi057 ceph-mon[40588]: from='client.36296 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi057-77351-40"}]: dispatch 2024-06-11T13:34:20.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:19 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/239651899' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi057-77351-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:20.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:19 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/61824103' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi057-77362-50"}]: dispatch 2024-06-11T13:34:20.288 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:19 smithi057 ceph-mon[40588]: from='client.36296 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi057-77351-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:20.288 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:19 smithi057 ceph-mon[40588]: from='client.26293 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi057-77362-50"}]: dispatch 2024-06-11T13:34:20.288 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:19 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/61824103' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi057-77362-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:20.288 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:19 smithi057 ceph-mon[40588]: from='client.26293 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi057-77362-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:20.288 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:19 smithi057 ceph-mon[40588]: pgmap v360: 324 pgs: 324 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:34:20.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:19 smithi107 ceph-mon[33178]: from='client.36284 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi057-77351-39"}]': finished 2024-06-11T13:34:20.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:19 smithi107 ceph-mon[33178]: from='client.36287 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi057-77362-49"}]': finished 2024-06-11T13:34:20.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:19 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:34:20.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:19 smithi107 ceph-mon[33178]: osdmap e261: 8 total, 8 up, 8 in 2024-06-11T13:34:20.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:19 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-31"}]: dispatch 2024-06-11T13:34:20.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:19 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-31"}]: dispatch 2024-06-11T13:34:20.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:19 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/239651899' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi057-77351-40"}]: dispatch 2024-06-11T13:34:20.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:19 smithi107 ceph-mon[33178]: from='client.36296 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi057-77351-40"}]: dispatch 2024-06-11T13:34:20.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:19 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/61824103' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi057-77362-50"}]: dispatch 2024-06-11T13:34:20.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:19 smithi107 ceph-mon[33178]: from='client.26293 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi057-77362-50"}]: dispatch 2024-06-11T13:34:20.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:19 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/239651899' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi057-77351-40"}]: dispatch 2024-06-11T13:34:20.302 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:19 smithi107 ceph-mon[33178]: from='client.36296 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi057-77351-40"}]: dispatch 2024-06-11T13:34:20.302 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:19 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/239651899' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi057-77351-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:20.302 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:19 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/61824103' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi057-77362-50"}]: dispatch 2024-06-11T13:34:20.303 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:19 smithi107 ceph-mon[33178]: from='client.36296 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi057-77351-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:20.303 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:19 smithi107 ceph-mon[33178]: from='client.26293 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi057-77362-50"}]: dispatch 2024-06-11T13:34:20.303 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:19 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/61824103' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi057-77362-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:20.303 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:19 smithi107 ceph-mon[33178]: from='client.26293 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi057-77362-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:20.303 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:19 smithi107 ceph-mon[33178]: pgmap v360: 324 pgs: 324 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:34:21.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:20 smithi057 ceph-mon[40588]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:34:21.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:20 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-31"}]': finished 2024-06-11T13:34:21.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:20 smithi057 ceph-mon[40588]: from='client.36296 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi057-77351-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:34:21.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:20 smithi057 ceph-mon[40588]: from='client.26293 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi057-77362-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:34:21.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:20 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/61824103' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi057-77362-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi057-77362-50"}]: dispatch 2024-06-11T13:34:21.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:20 smithi057 ceph-mon[40588]: osdmap e262: 8 total, 8 up, 8 in 2024-06-11T13:34:21.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:20 smithi057 ceph-mon[40588]: from='client.26293 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi057-77362-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi057-77362-50"}]: dispatch 2024-06-11T13:34:21.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:20 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/239651899' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi057-77351-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi057-77351-40"}]: dispatch 2024-06-11T13:34:21.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:20 smithi057 ceph-mon[40588]: from='client.36296 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi057-77351-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi057-77351-40"}]: dispatch 2024-06-11T13:34:21.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:20 smithi057 ceph-mon[29805]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:34:21.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:20 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-31"}]': finished 2024-06-11T13:34:21.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:20 smithi057 ceph-mon[29805]: from='client.36296 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi057-77351-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:34:21.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:20 smithi057 ceph-mon[29805]: from='client.26293 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi057-77362-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:34:21.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:20 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/61824103' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi057-77362-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi057-77362-50"}]: dispatch 2024-06-11T13:34:21.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:20 smithi057 ceph-mon[29805]: osdmap e262: 8 total, 8 up, 8 in 2024-06-11T13:34:21.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:20 smithi057 ceph-mon[29805]: from='client.26293 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi057-77362-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi057-77362-50"}]: dispatch 2024-06-11T13:34:21.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:20 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/239651899' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi057-77351-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi057-77351-40"}]: dispatch 2024-06-11T13:34:21.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:20 smithi057 ceph-mon[29805]: from='client.36296 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi057-77351-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi057-77351-40"}]: dispatch 2024-06-11T13:34:21.286 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:20 smithi107 ceph-mon[33178]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:34:21.286 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:20 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-31"}]': finished 2024-06-11T13:34:21.286 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:20 smithi107 ceph-mon[33178]: from='client.36296 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi057-77351-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:34:21.286 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:20 smithi107 ceph-mon[33178]: from='client.26293 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi057-77362-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:34:21.286 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:20 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/61824103' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi057-77362-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi057-77362-50"}]: dispatch 2024-06-11T13:34:21.286 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:20 smithi107 ceph-mon[33178]: osdmap e262: 8 total, 8 up, 8 in 2024-06-11T13:34:21.286 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:20 smithi107 ceph-mon[33178]: from='client.26293 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi057-77362-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi057-77362-50"}]: dispatch 2024-06-11T13:34:21.287 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:20 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/239651899' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi057-77351-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi057-77351-40"}]: dispatch 2024-06-11T13:34:21.287 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:20 smithi107 ceph-mon[33178]: from='client.36296 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi057-77351-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi057-77351-40"}]: dispatch 2024-06-11T13:34:21.547 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:34:21 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:34:22.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:22 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:34:22.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:22 smithi057 ceph-mon[40588]: osdmap e263: 8 total, 8 up, 8 in 2024-06-11T13:34:22.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:22 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:34:22.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:22 smithi057 ceph-mon[29805]: osdmap e263: 8 total, 8 up, 8 in 2024-06-11T13:34:22.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:22 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:34:22.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:22 smithi107 ceph-mon[33178]: osdmap e263: 8 total, 8 up, 8 in 2024-06-11T13:34:23.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:23 smithi057 ceph-mon[29805]: pgmap v363: 292 pgs: 292 active+clean; 4.6 MiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:34:23.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:23 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:34:23.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:23 smithi057 ceph-mon[29805]: from='client.26293 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi057-77362-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi057-77362-50"}]': finished 2024-06-11T13:34:23.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:23 smithi057 ceph-mon[29805]: from='client.36296 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi057-77351-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi057-77351-40"}]': finished 2024-06-11T13:34:23.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:23 smithi057 ceph-mon[29805]: osdmap e264: 8 total, 8 up, 8 in 2024-06-11T13:34:23.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:23 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:34:23.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:23 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:34:23.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:23 smithi057 ceph-mon[40588]: pgmap v363: 292 pgs: 292 active+clean; 4.6 MiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:34:23.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:23 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:34:23.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:23 smithi057 ceph-mon[40588]: from='client.26293 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi057-77362-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi057-77362-50"}]': finished 2024-06-11T13:34:23.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:23 smithi057 ceph-mon[40588]: from='client.36296 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi057-77351-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi057-77351-40"}]': finished 2024-06-11T13:34:23.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:23 smithi057 ceph-mon[40588]: osdmap e264: 8 total, 8 up, 8 in 2024-06-11T13:34:23.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:23 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:34:23.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:23 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:34:23.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:23 smithi107 ceph-mon[33178]: pgmap v363: 292 pgs: 292 active+clean; 4.6 MiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:34:23.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:23 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:34:23.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:23 smithi107 ceph-mon[33178]: from='client.26293 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi057-77362-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi057-77362-50"}]': finished 2024-06-11T13:34:23.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:23 smithi107 ceph-mon[33178]: from='client.36296 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi057-77351-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi057-77351-40"}]': finished 2024-06-11T13:34:23.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:23 smithi107 ceph-mon[33178]: osdmap e264: 8 total, 8 up, 8 in 2024-06-11T13:34:23.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:23 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:34:23.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:23 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:34:24.034 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:34:23 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:34:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:34:24.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:24 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:34:24.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:24 smithi057 ceph-mon[40588]: osdmap e265: 8 total, 8 up, 8 in 2024-06-11T13:34:24.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:24 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:34:24.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:24 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:34:24.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:24 smithi057 ceph-mon[40588]: pgmap v366: 340 pgs: 48 unknown, 292 active+clean; 4.6 MiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:34:24.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:24 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:34:24.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:24 smithi057 ceph-mon[29805]: osdmap e265: 8 total, 8 up, 8 in 2024-06-11T13:34:24.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:24 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:34:24.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:24 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:34:24.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:24 smithi057 ceph-mon[29805]: pgmap v366: 340 pgs: 48 unknown, 292 active+clean; 4.6 MiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:34:24.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:24 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:34:24.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:24 smithi107 ceph-mon[33178]: osdmap e265: 8 total, 8 up, 8 in 2024-06-11T13:34:24.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:24 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:34:24.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:24 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:34:24.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:24 smithi107 ceph-mon[33178]: pgmap v366: 340 pgs: 48 unknown, 292 active+clean; 4.6 MiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:34:25.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:25 smithi057 ceph-mon[29805]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:34:25.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:25 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-33", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:34:25.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:25 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-33"}]: dispatch 2024-06-11T13:34:25.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:25 smithi057 ceph-mon[29805]: osdmap e266: 8 total, 8 up, 8 in 2024-06-11T13:34:25.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:25 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-33"}]: dispatch 2024-06-11T13:34:25.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:25 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/61824103' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi057-77362-50"}]: dispatch 2024-06-11T13:34:25.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:25 smithi057 ceph-mon[29805]: from='client.26293 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi057-77362-50"}]: dispatch 2024-06-11T13:34:25.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:25 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/239651899' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi057-77351-40"}]: dispatch 2024-06-11T13:34:25.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:25 smithi057 ceph-mon[29805]: from='client.36296 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi057-77351-40"}]: dispatch 2024-06-11T13:34:25.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:25 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:34:25.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:25 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:34:25.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:25 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:34:25.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:25 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:34:25.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:25 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:34:25.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:25 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:34:25.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:25 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:34:25.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:25 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:34:25.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:25 smithi057 ceph-mon[40588]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:34:25.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:25 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-33", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:34:25.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:25 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-33"}]: dispatch 2024-06-11T13:34:25.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:25 smithi057 ceph-mon[40588]: osdmap e266: 8 total, 8 up, 8 in 2024-06-11T13:34:25.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:25 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-33"}]: dispatch 2024-06-11T13:34:25.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:25 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/61824103' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi057-77362-50"}]: dispatch 2024-06-11T13:34:25.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:25 smithi057 ceph-mon[40588]: from='client.26293 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi057-77362-50"}]: dispatch 2024-06-11T13:34:25.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:25 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/239651899' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi057-77351-40"}]: dispatch 2024-06-11T13:34:25.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:25 smithi057 ceph-mon[40588]: from='client.36296 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi057-77351-40"}]: dispatch 2024-06-11T13:34:25.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:25 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:34:25.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:25 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:34:25.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:25 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:34:25.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:25 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:34:25.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:25 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:34:25.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:25 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:34:25.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:25 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:34:25.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:25 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:34:25.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:25 smithi107 ceph-mon[33178]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:34:25.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:25 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-33", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:34:25.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:25 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-33"}]: dispatch 2024-06-11T13:34:25.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:25 smithi107 ceph-mon[33178]: osdmap e266: 8 total, 8 up, 8 in 2024-06-11T13:34:25.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:25 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-33"}]: dispatch 2024-06-11T13:34:25.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:25 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/61824103' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi057-77362-50"}]: dispatch 2024-06-11T13:34:25.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:25 smithi107 ceph-mon[33178]: from='client.26293 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi057-77362-50"}]: dispatch 2024-06-11T13:34:25.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:25 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/239651899' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi057-77351-40"}]: dispatch 2024-06-11T13:34:25.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:25 smithi107 ceph-mon[33178]: from='client.36296 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi057-77351-40"}]: dispatch 2024-06-11T13:34:25.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:25 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:34:25.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:25 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:34:25.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:25 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:34:25.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:25 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:34:25.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:25 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:34:25.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:25 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:34:25.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:25 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:34:25.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:25 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:34:26.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:26 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-33"}]': finished 2024-06-11T13:34:26.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:26 smithi057 ceph-mon[29805]: from='client.26293 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi057-77362-50"}]': finished 2024-06-11T13:34:26.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:26 smithi057 ceph-mon[29805]: from='client.36296 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi057-77351-40"}]': finished 2024-06-11T13:34:26.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:26 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-33", "mode": "writeback"}]: dispatch 2024-06-11T13:34:26.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:26 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/61824103' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi057-77362-50"}]: dispatch 2024-06-11T13:34:26.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:26 smithi057 ceph-mon[29805]: osdmap e267: 8 total, 8 up, 8 in 2024-06-11T13:34:26.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:26 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/239651899' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi057-77351-40"}]: dispatch 2024-06-11T13:34:26.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:26 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-33", "mode": "writeback"}]: dispatch 2024-06-11T13:34:26.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:26 smithi057 ceph-mon[29805]: from='client.36296 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi057-77351-40"}]: dispatch 2024-06-11T13:34:26.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:26 smithi057 ceph-mon[29805]: from='client.26293 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi057-77362-50"}]: dispatch 2024-06-11T13:34:26.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:26 smithi057 ceph-mon[29805]: pgmap v369: 324 pgs: 32 unknown, 292 active+clean; 4.6 MiB data, 605 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:34:26.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:26 smithi057 ceph-mon[29805]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:34:26.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:26 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-33", "mode": "writeback"}]': finished 2024-06-11T13:34:26.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:26 smithi057 ceph-mon[29805]: from='client.36296 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi057-77351-40"}]': finished 2024-06-11T13:34:26.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:26 smithi057 ceph-mon[29805]: from='client.26293 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi057-77362-50"}]': finished 2024-06-11T13:34:26.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:26 smithi057 ceph-mon[29805]: osdmap e268: 8 total, 8 up, 8 in 2024-06-11T13:34:26.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:26 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-33"}]': finished 2024-06-11T13:34:26.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:26 smithi057 ceph-mon[40588]: from='client.26293 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi057-77362-50"}]': finished 2024-06-11T13:34:26.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:26 smithi057 ceph-mon[40588]: from='client.36296 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi057-77351-40"}]': finished 2024-06-11T13:34:26.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:26 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-33", "mode": "writeback"}]: dispatch 2024-06-11T13:34:26.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:26 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/61824103' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi057-77362-50"}]: dispatch 2024-06-11T13:34:26.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:26 smithi057 ceph-mon[40588]: osdmap e267: 8 total, 8 up, 8 in 2024-06-11T13:34:26.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:26 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/239651899' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi057-77351-40"}]: dispatch 2024-06-11T13:34:26.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:26 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-33", "mode": "writeback"}]: dispatch 2024-06-11T13:34:26.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:26 smithi057 ceph-mon[40588]: from='client.36296 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi057-77351-40"}]: dispatch 2024-06-11T13:34:26.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:26 smithi057 ceph-mon[40588]: from='client.26293 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi057-77362-50"}]: dispatch 2024-06-11T13:34:26.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:26 smithi057 ceph-mon[40588]: pgmap v369: 324 pgs: 32 unknown, 292 active+clean; 4.6 MiB data, 605 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:34:26.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:26 smithi057 ceph-mon[40588]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:34:26.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:26 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-33", "mode": "writeback"}]': finished 2024-06-11T13:34:26.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:26 smithi057 ceph-mon[40588]: from='client.36296 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi057-77351-40"}]': finished 2024-06-11T13:34:26.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:26 smithi057 ceph-mon[40588]: from='client.26293 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi057-77362-50"}]': finished 2024-06-11T13:34:26.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:26 smithi057 ceph-mon[40588]: osdmap e268: 8 total, 8 up, 8 in 2024-06-11T13:34:26.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:26 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-33"}]': finished 2024-06-11T13:34:26.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:26 smithi107 ceph-mon[33178]: from='client.26293 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi057-77362-50"}]': finished 2024-06-11T13:34:26.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:26 smithi107 ceph-mon[33178]: from='client.36296 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi057-77351-40"}]': finished 2024-06-11T13:34:26.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:26 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-33", "mode": "writeback"}]: dispatch 2024-06-11T13:34:26.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:26 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/61824103' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi057-77362-50"}]: dispatch 2024-06-11T13:34:26.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:26 smithi107 ceph-mon[33178]: osdmap e267: 8 total, 8 up, 8 in 2024-06-11T13:34:26.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:26 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/239651899' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi057-77351-40"}]: dispatch 2024-06-11T13:34:26.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:26 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-33", "mode": "writeback"}]: dispatch 2024-06-11T13:34:26.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:26 smithi107 ceph-mon[33178]: from='client.36296 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi057-77351-40"}]: dispatch 2024-06-11T13:34:26.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:26 smithi107 ceph-mon[33178]: from='client.26293 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi057-77362-50"}]: dispatch 2024-06-11T13:34:26.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:26 smithi107 ceph-mon[33178]: pgmap v369: 324 pgs: 32 unknown, 292 active+clean; 4.6 MiB data, 605 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:34:26.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:26 smithi107 ceph-mon[33178]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:34:26.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:26 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-33", "mode": "writeback"}]': finished 2024-06-11T13:34:26.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:26 smithi107 ceph-mon[33178]: from='client.36296 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi057-77351-40"}]': finished 2024-06-11T13:34:26.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:26 smithi107 ceph-mon[33178]: from='client.26293 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi057-77362-50"}]': finished 2024-06-11T13:34:26.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:26 smithi107 ceph-mon[33178]: osdmap e268: 8 total, 8 up, 8 in 2024-06-11T13:34:27.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:27 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2940126986' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi057-77351-41"}]: dispatch 2024-06-11T13:34:27.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:27 smithi057 ceph-mon[29805]: from='client.26302 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi057-77351-41"}]: dispatch 2024-06-11T13:34:27.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:27 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2940126986' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi057-77351-41"}]: dispatch 2024-06-11T13:34:27.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:27 smithi057 ceph-mon[29805]: from='client.26302 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi057-77351-41"}]: dispatch 2024-06-11T13:34:27.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:27 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2940126986' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi057-77351-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:27.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:27 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:34:27.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:27 smithi057 ceph-mon[29805]: from='client.26302 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi057-77351-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:27.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:27 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:34:27.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:27 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2940126986' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi057-77351-41"}]: dispatch 2024-06-11T13:34:27.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:27 smithi057 ceph-mon[40588]: from='client.26302 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi057-77351-41"}]: dispatch 2024-06-11T13:34:27.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:27 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2940126986' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi057-77351-41"}]: dispatch 2024-06-11T13:34:27.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:27 smithi057 ceph-mon[40588]: from='client.26302 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi057-77351-41"}]: dispatch 2024-06-11T13:34:27.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:27 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2940126986' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi057-77351-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:27.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:27 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:34:27.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:27 smithi057 ceph-mon[40588]: from='client.26302 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi057-77351-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:27.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:27 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:34:27.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:27 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2940126986' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi057-77351-41"}]: dispatch 2024-06-11T13:34:27.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:27 smithi107 ceph-mon[33178]: from='client.26302 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi057-77351-41"}]: dispatch 2024-06-11T13:34:27.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:27 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2940126986' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi057-77351-41"}]: dispatch 2024-06-11T13:34:27.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:27 smithi107 ceph-mon[33178]: from='client.26302 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi057-77351-41"}]: dispatch 2024-06-11T13:34:27.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:27 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2940126986' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi057-77351-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:27.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:27 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:34:27.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:27 smithi107 ceph-mon[33178]: from='client.26302 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi057-77351-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:27.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:27 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:34:29.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:28 smithi057 ceph-mon[29805]: from='client.26302 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi057-77351-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:34:29.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:28 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:34:29.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:28 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-33"}]: dispatch 2024-06-11T13:34:29.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:28 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2940126986' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi057-77351-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi057-77351-41"}]: dispatch 2024-06-11T13:34:29.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:28 smithi057 ceph-mon[29805]: osdmap e269: 8 total, 8 up, 8 in 2024-06-11T13:34:29.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:28 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3831989589' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi057-77362-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:34:29.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:28 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-33"}]: dispatch 2024-06-11T13:34:29.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:28 smithi057 ceph-mon[29805]: from='client.26302 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi057-77351-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi057-77351-41"}]: dispatch 2024-06-11T13:34:29.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:28 smithi057 ceph-mon[29805]: pgmap v372: 356 pgs: 32 unknown, 324 active+clean; 4.6 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:34:29.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:28 smithi057 ceph-mon[29805]: from='client.26305 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi057-77362-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:34:29.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:28 smithi057 ceph-mon[29805]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:34:29.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:28 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-33"}]': finished 2024-06-11T13:34:29.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:28 smithi057 ceph-mon[29805]: from='client.26305 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi057-77362-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:34:29.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:28 smithi057 ceph-mon[40588]: from='client.26302 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi057-77351-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:34:29.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:28 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:34:29.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:28 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-33"}]: dispatch 2024-06-11T13:34:29.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:28 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2940126986' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi057-77351-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi057-77351-41"}]: dispatch 2024-06-11T13:34:29.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:28 smithi057 ceph-mon[40588]: osdmap e269: 8 total, 8 up, 8 in 2024-06-11T13:34:29.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:28 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3831989589' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi057-77362-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:34:29.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:28 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-33"}]: dispatch 2024-06-11T13:34:29.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:28 smithi057 ceph-mon[40588]: from='client.26302 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi057-77351-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi057-77351-41"}]: dispatch 2024-06-11T13:34:29.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:28 smithi057 ceph-mon[40588]: pgmap v372: 356 pgs: 32 unknown, 324 active+clean; 4.6 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:34:29.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:28 smithi057 ceph-mon[40588]: from='client.26305 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi057-77362-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:34:29.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:28 smithi057 ceph-mon[40588]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:34:29.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:28 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-33"}]': finished 2024-06-11T13:34:29.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:28 smithi057 ceph-mon[40588]: from='client.26305 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi057-77362-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:34:29.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:28 smithi107 ceph-mon[33178]: from='client.26302 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi057-77351-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:34:29.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:28 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:34:29.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:28 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-33"}]: dispatch 2024-06-11T13:34:29.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:28 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2940126986' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi057-77351-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi057-77351-41"}]: dispatch 2024-06-11T13:34:29.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:28 smithi107 ceph-mon[33178]: osdmap e269: 8 total, 8 up, 8 in 2024-06-11T13:34:29.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:28 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3831989589' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi057-77362-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:34:29.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:28 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-33"}]: dispatch 2024-06-11T13:34:29.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:28 smithi107 ceph-mon[33178]: from='client.26302 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi057-77351-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi057-77351-41"}]: dispatch 2024-06-11T13:34:29.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:28 smithi107 ceph-mon[33178]: pgmap v372: 356 pgs: 32 unknown, 324 active+clean; 4.6 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:34:29.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:28 smithi107 ceph-mon[33178]: from='client.26305 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi057-77362-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:34:29.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:28 smithi107 ceph-mon[33178]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:34:29.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:28 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-33"}]': finished 2024-06-11T13:34:29.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:28 smithi107 ceph-mon[33178]: from='client.26305 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi057-77362-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:34:30.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:29 smithi057 ceph-mon[29805]: osdmap e270: 8 total, 8 up, 8 in 2024-06-11T13:34:30.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:29 smithi057 ceph-mon[29805]: from='client.26302 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi057-77351-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi057-77351-41"}]': finished 2024-06-11T13:34:30.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:29 smithi057 ceph-mon[29805]: osdmap e271: 8 total, 8 up, 8 in 2024-06-11T13:34:30.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:29 smithi057 ceph-mon[40588]: osdmap e270: 8 total, 8 up, 8 in 2024-06-11T13:34:30.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:29 smithi057 ceph-mon[40588]: from='client.26302 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi057-77351-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi057-77351-41"}]': finished 2024-06-11T13:34:30.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:29 smithi057 ceph-mon[40588]: osdmap e271: 8 total, 8 up, 8 in 2024-06-11T13:34:30.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:29 smithi107 ceph-mon[33178]: osdmap e270: 8 total, 8 up, 8 in 2024-06-11T13:34:30.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:29 smithi107 ceph-mon[33178]: from='client.26302 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi057-77351-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi057-77351-41"}]': finished 2024-06-11T13:34:30.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:29 smithi107 ceph-mon[33178]: osdmap e271: 8 total, 8 up, 8 in 2024-06-11T13:34:31.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:30 smithi057 ceph-mon[29805]: pgmap v375: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:34:31.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:30 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3327132090' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi057-77362-52"}]: dispatch 2024-06-11T13:34:31.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:30 smithi057 ceph-mon[29805]: from='client.36317 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi057-77362-52"}]: dispatch 2024-06-11T13:34:31.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:30 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3327132090' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi057-77362-52"}]: dispatch 2024-06-11T13:34:31.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:30 smithi057 ceph-mon[29805]: from='client.36317 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi057-77362-52"}]: dispatch 2024-06-11T13:34:31.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:30 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3327132090' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi057-77362-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:31.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:30 smithi057 ceph-mon[29805]: from='client.36317 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi057-77362-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:31.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:30 smithi057 ceph-mon[29805]: from='client.36317 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi057-77362-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:34:31.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:30 smithi057 ceph-mon[29805]: osdmap e272: 8 total, 8 up, 8 in 2024-06-11T13:34:31.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:30 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3327132090' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi057-77362-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi057-77362-52"}]: dispatch 2024-06-11T13:34:31.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:30 smithi057 ceph-mon[29805]: from='client.36317 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi057-77362-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi057-77362-52"}]: dispatch 2024-06-11T13:34:31.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:30 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:34:31.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:30 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:34:31.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:30 smithi057 ceph-mon[40588]: pgmap v375: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:34:31.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:30 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3327132090' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi057-77362-52"}]: dispatch 2024-06-11T13:34:31.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:30 smithi057 ceph-mon[40588]: from='client.36317 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi057-77362-52"}]: dispatch 2024-06-11T13:34:31.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:30 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3327132090' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi057-77362-52"}]: dispatch 2024-06-11T13:34:31.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:30 smithi057 ceph-mon[40588]: from='client.36317 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi057-77362-52"}]: dispatch 2024-06-11T13:34:31.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:30 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3327132090' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi057-77362-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:31.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:30 smithi057 ceph-mon[40588]: from='client.36317 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi057-77362-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:31.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:30 smithi057 ceph-mon[40588]: from='client.36317 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi057-77362-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:34:31.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:30 smithi057 ceph-mon[40588]: osdmap e272: 8 total, 8 up, 8 in 2024-06-11T13:34:31.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:30 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3327132090' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi057-77362-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi057-77362-52"}]: dispatch 2024-06-11T13:34:31.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:30 smithi057 ceph-mon[40588]: from='client.36317 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi057-77362-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi057-77362-52"}]: dispatch 2024-06-11T13:34:31.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:30 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:34:31.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:30 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:34:31.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:30 smithi107 ceph-mon[33178]: pgmap v375: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:34:31.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:30 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3327132090' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi057-77362-52"}]: dispatch 2024-06-11T13:34:31.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:30 smithi107 ceph-mon[33178]: from='client.36317 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi057-77362-52"}]: dispatch 2024-06-11T13:34:31.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:30 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3327132090' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi057-77362-52"}]: dispatch 2024-06-11T13:34:31.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:30 smithi107 ceph-mon[33178]: from='client.36317 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi057-77362-52"}]: dispatch 2024-06-11T13:34:31.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:30 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3327132090' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi057-77362-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:31.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:30 smithi107 ceph-mon[33178]: from='client.36317 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi057-77362-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:31.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:30 smithi107 ceph-mon[33178]: from='client.36317 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi057-77362-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:34:31.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:30 smithi107 ceph-mon[33178]: osdmap e272: 8 total, 8 up, 8 in 2024-06-11T13:34:31.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:30 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3327132090' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi057-77362-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi057-77362-52"}]: dispatch 2024-06-11T13:34:31.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:30 smithi107 ceph-mon[33178]: from='client.36317 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi057-77362-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi057-77362-52"}]: dispatch 2024-06-11T13:34:31.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:30 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:34:31.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:30 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:34:31.547 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:34:31 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:34:31.833 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:31 smithi107 ceph-mon[33178]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:34:31.833 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:31 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:34:31.833 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:31 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:34:31.833 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:31 smithi107 ceph-mon[33178]: osdmap e273: 8 total, 8 up, 8 in 2024-06-11T13:34:31.833 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:31 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:34:31.833 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:31 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2940126986' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi057-77351-41"}]: dispatch 2024-06-11T13:34:31.833 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:31 smithi107 ceph-mon[33178]: from='client.26302 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi057-77351-41"}]: dispatch 2024-06-11T13:34:32.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:31 smithi057 ceph-mon[29805]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:34:32.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:31 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:34:32.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:31 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:34:32.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:31 smithi057 ceph-mon[29805]: osdmap e273: 8 total, 8 up, 8 in 2024-06-11T13:34:32.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:31 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:34:32.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:31 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2940126986' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi057-77351-41"}]: dispatch 2024-06-11T13:34:32.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:31 smithi057 ceph-mon[29805]: from='client.26302 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi057-77351-41"}]: dispatch 2024-06-11T13:34:32.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:31 smithi057 ceph-mon[40588]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:34:32.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:31 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:34:32.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:31 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:34:32.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:31 smithi057 ceph-mon[40588]: osdmap e273: 8 total, 8 up, 8 in 2024-06-11T13:34:32.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:31 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:34:32.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:31 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2940126986' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi057-77351-41"}]: dispatch 2024-06-11T13:34:32.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:31 smithi057 ceph-mon[40588]: from='client.26302 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi057-77351-41"}]: dispatch 2024-06-11T13:34:33.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:32 smithi057 ceph-mon[40588]: pgmap v377: 332 pgs: 32 creating+peering, 300 active+clean; 4.6 MiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 510 B/s wr, 2 op/s 2024-06-11T13:34:33.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:32 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:34:33.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:32 smithi057 ceph-mon[40588]: from='client.36317 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi057-77362-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi057-77362-52"}]': finished 2024-06-11T13:34:33.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:32 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-35", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:34:33.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:32 smithi057 ceph-mon[40588]: from='client.26302 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi057-77351-41"}]': finished 2024-06-11T13:34:33.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:32 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-35"}]: dispatch 2024-06-11T13:34:33.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:32 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2940126986' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi057-77351-41"}]: dispatch 2024-06-11T13:34:33.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:32 smithi057 ceph-mon[40588]: osdmap e274: 8 total, 8 up, 8 in 2024-06-11T13:34:33.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:32 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-35"}]: dispatch 2024-06-11T13:34:33.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:32 smithi057 ceph-mon[40588]: from='client.26302 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi057-77351-41"}]: dispatch 2024-06-11T13:34:33.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:32 smithi057 ceph-mon[29805]: pgmap v377: 332 pgs: 32 creating+peering, 300 active+clean; 4.6 MiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 510 B/s wr, 2 op/s 2024-06-11T13:34:33.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:32 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:34:33.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:32 smithi057 ceph-mon[29805]: from='client.36317 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi057-77362-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi057-77362-52"}]': finished 2024-06-11T13:34:33.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:32 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-35", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:34:33.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:32 smithi057 ceph-mon[29805]: from='client.26302 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi057-77351-41"}]': finished 2024-06-11T13:34:33.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:32 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-35"}]: dispatch 2024-06-11T13:34:33.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:32 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2940126986' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi057-77351-41"}]: dispatch 2024-06-11T13:34:33.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:32 smithi057 ceph-mon[29805]: osdmap e274: 8 total, 8 up, 8 in 2024-06-11T13:34:33.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:32 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-35"}]: dispatch 2024-06-11T13:34:33.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:32 smithi057 ceph-mon[29805]: from='client.26302 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi057-77351-41"}]: dispatch 2024-06-11T13:34:33.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:32 smithi107 ceph-mon[33178]: pgmap v377: 332 pgs: 32 creating+peering, 300 active+clean; 4.6 MiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 510 B/s wr, 2 op/s 2024-06-11T13:34:33.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:32 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:34:33.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:32 smithi107 ceph-mon[33178]: from='client.36317 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi057-77362-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi057-77362-52"}]': finished 2024-06-11T13:34:33.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:32 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-35", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:34:33.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:32 smithi107 ceph-mon[33178]: from='client.26302 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi057-77351-41"}]': finished 2024-06-11T13:34:33.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:32 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-35"}]: dispatch 2024-06-11T13:34:33.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:32 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2940126986' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi057-77351-41"}]: dispatch 2024-06-11T13:34:33.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:32 smithi107 ceph-mon[33178]: osdmap e274: 8 total, 8 up, 8 in 2024-06-11T13:34:33.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:32 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-35"}]: dispatch 2024-06-11T13:34:33.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:32 smithi107 ceph-mon[33178]: from='client.26302 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi057-77351-41"}]: dispatch 2024-06-11T13:34:34.034 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:34:33 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:34:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:34:34.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:34 smithi107 ceph-mon[33178]: pgmap v380: 332 pgs: 8 unknown, 32 creating+peering, 292 active+clean; 4.6 MiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:34:34.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:34 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-35"}]': finished 2024-06-11T13:34:34.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:34 smithi107 ceph-mon[33178]: from='client.26302 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi057-77351-41"}]': finished 2024-06-11T13:34:34.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:34 smithi107 ceph-mon[33178]: osdmap e275: 8 total, 8 up, 8 in 2024-06-11T13:34:34.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:34 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-35", "mode": "writeback"}]: dispatch 2024-06-11T13:34:34.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:34 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-35", "mode": "writeback"}]: dispatch 2024-06-11T13:34:34.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:34 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1870887340' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi057-77351-42"}]: dispatch 2024-06-11T13:34:34.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:34 smithi107 ceph-mon[33178]: from='client.36323 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi057-77351-42"}]: dispatch 2024-06-11T13:34:34.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:34 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1870887340' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi057-77351-42"}]: dispatch 2024-06-11T13:34:34.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:34 smithi107 ceph-mon[33178]: from='client.36323 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi057-77351-42"}]: dispatch 2024-06-11T13:34:34.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:34 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1870887340' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi057-77351-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:34.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:34 smithi107 ceph-mon[33178]: from='client.36323 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi057-77351-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:34.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:34 smithi057 ceph-mon[40588]: pgmap v380: 332 pgs: 8 unknown, 32 creating+peering, 292 active+clean; 4.6 MiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:34:34.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:34 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-35"}]': finished 2024-06-11T13:34:34.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:34 smithi057 ceph-mon[40588]: from='client.26302 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi057-77351-41"}]': finished 2024-06-11T13:34:34.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:34 smithi057 ceph-mon[40588]: osdmap e275: 8 total, 8 up, 8 in 2024-06-11T13:34:34.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:34 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-35", "mode": "writeback"}]: dispatch 2024-06-11T13:34:34.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:34 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-35", "mode": "writeback"}]: dispatch 2024-06-11T13:34:34.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:34 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1870887340' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi057-77351-42"}]: dispatch 2024-06-11T13:34:34.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:34 smithi057 ceph-mon[40588]: from='client.36323 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi057-77351-42"}]: dispatch 2024-06-11T13:34:34.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:34 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1870887340' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi057-77351-42"}]: dispatch 2024-06-11T13:34:34.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:34 smithi057 ceph-mon[40588]: from='client.36323 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi057-77351-42"}]: dispatch 2024-06-11T13:34:34.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:34 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1870887340' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi057-77351-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:34.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:34 smithi057 ceph-mon[40588]: from='client.36323 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi057-77351-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:34.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:34 smithi057 ceph-mon[29805]: pgmap v380: 332 pgs: 8 unknown, 32 creating+peering, 292 active+clean; 4.6 MiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:34:34.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:34 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-35"}]': finished 2024-06-11T13:34:34.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:34 smithi057 ceph-mon[29805]: from='client.26302 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi057-77351-41"}]': finished 2024-06-11T13:34:34.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:34 smithi057 ceph-mon[29805]: osdmap e275: 8 total, 8 up, 8 in 2024-06-11T13:34:34.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:34 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-35", "mode": "writeback"}]: dispatch 2024-06-11T13:34:34.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:34 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-35", "mode": "writeback"}]: dispatch 2024-06-11T13:34:34.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:34 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1870887340' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi057-77351-42"}]: dispatch 2024-06-11T13:34:34.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:34 smithi057 ceph-mon[29805]: from='client.36323 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi057-77351-42"}]: dispatch 2024-06-11T13:34:34.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:34 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1870887340' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi057-77351-42"}]: dispatch 2024-06-11T13:34:34.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:34 smithi057 ceph-mon[29805]: from='client.36323 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi057-77351-42"}]: dispatch 2024-06-11T13:34:34.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:34 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1870887340' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi057-77351-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:34.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:34 smithi057 ceph-mon[29805]: from='client.36323 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi057-77351-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:35.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:35 smithi107 ceph-mon[33178]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:34:35.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:35 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-35", "mode": "writeback"}]': finished 2024-06-11T13:34:35.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:35 smithi107 ceph-mon[33178]: from='client.36323 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi057-77351-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:34:35.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:35 smithi107 ceph-mon[33178]: osdmap e276: 8 total, 8 up, 8 in 2024-06-11T13:34:35.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:35 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1870887340' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi057-77351-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi057-77351-42"}]: dispatch 2024-06-11T13:34:35.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:35 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3327132090' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi057-77362-52"}]: dispatch 2024-06-11T13:34:35.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:35 smithi107 ceph-mon[33178]: from='client.36323 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi057-77351-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi057-77351-42"}]: dispatch 2024-06-11T13:34:35.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:35 smithi107 ceph-mon[33178]: from='client.36317 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi057-77362-52"}]: dispatch 2024-06-11T13:34:35.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:35 smithi057 ceph-mon[29805]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:34:35.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:35 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-35", "mode": "writeback"}]': finished 2024-06-11T13:34:35.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:35 smithi057 ceph-mon[29805]: from='client.36323 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi057-77351-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:34:35.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:35 smithi057 ceph-mon[29805]: osdmap e276: 8 total, 8 up, 8 in 2024-06-11T13:34:35.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:35 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1870887340' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi057-77351-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi057-77351-42"}]: dispatch 2024-06-11T13:34:35.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:35 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3327132090' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi057-77362-52"}]: dispatch 2024-06-11T13:34:35.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:35 smithi057 ceph-mon[29805]: from='client.36323 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi057-77351-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi057-77351-42"}]: dispatch 2024-06-11T13:34:35.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:35 smithi057 ceph-mon[29805]: from='client.36317 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi057-77362-52"}]: dispatch 2024-06-11T13:34:35.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:35 smithi057 ceph-mon[40588]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:34:35.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:35 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-35", "mode": "writeback"}]': finished 2024-06-11T13:34:35.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:35 smithi057 ceph-mon[40588]: from='client.36323 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi057-77351-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:34:35.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:35 smithi057 ceph-mon[40588]: osdmap e276: 8 total, 8 up, 8 in 2024-06-11T13:34:35.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:35 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1870887340' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi057-77351-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi057-77351-42"}]: dispatch 2024-06-11T13:34:35.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:35 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3327132090' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi057-77362-52"}]: dispatch 2024-06-11T13:34:35.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:35 smithi057 ceph-mon[40588]: from='client.36323 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi057-77351-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi057-77351-42"}]: dispatch 2024-06-11T13:34:35.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:35 smithi057 ceph-mon[40588]: from='client.36317 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi057-77362-52"}]: dispatch 2024-06-11T13:34:36.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:36 smithi057 ceph-mon[29805]: pgmap v383: 324 pgs: 32 creating+peering, 292 active+clean; 4.6 MiB data, 598 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:34:36.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:36 smithi057 ceph-mon[29805]: from='client.36317 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi057-77362-52"}]': finished 2024-06-11T13:34:36.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:36 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3327132090' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi057-77362-52"}]: dispatch 2024-06-11T13:34:36.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:36 smithi057 ceph-mon[29805]: osdmap e277: 8 total, 8 up, 8 in 2024-06-11T13:34:36.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:36 smithi057 ceph-mon[29805]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:34:36.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:36 smithi057 ceph-mon[29805]: from='client.36317 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi057-77362-52"}]: dispatch 2024-06-11T13:34:36.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:36 smithi057 ceph-mon[40588]: pgmap v383: 324 pgs: 32 creating+peering, 292 active+clean; 4.6 MiB data, 598 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:34:36.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:36 smithi057 ceph-mon[40588]: from='client.36317 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi057-77362-52"}]': finished 2024-06-11T13:34:36.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:36 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3327132090' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi057-77362-52"}]: dispatch 2024-06-11T13:34:36.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:36 smithi057 ceph-mon[40588]: osdmap e277: 8 total, 8 up, 8 in 2024-06-11T13:34:36.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:36 smithi057 ceph-mon[40588]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:34:36.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:36 smithi057 ceph-mon[40588]: from='client.36317 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi057-77362-52"}]: dispatch 2024-06-11T13:34:36.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:36 smithi107 ceph-mon[33178]: pgmap v383: 324 pgs: 32 creating+peering, 292 active+clean; 4.6 MiB data, 598 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:34:36.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:36 smithi107 ceph-mon[33178]: from='client.36317 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi057-77362-52"}]': finished 2024-06-11T13:34:36.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:36 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3327132090' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi057-77362-52"}]: dispatch 2024-06-11T13:34:36.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:36 smithi107 ceph-mon[33178]: osdmap e277: 8 total, 8 up, 8 in 2024-06-11T13:34:36.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:36 smithi107 ceph-mon[33178]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:34:36.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:36 smithi107 ceph-mon[33178]: from='client.36317 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi057-77362-52"}]: dispatch 2024-06-11T13:34:37.756 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a756a0 2024-06-11T13:34:37.756 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a757d0 2024-06-11T13:34:37.756 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a75d90 2024-06-11T13:34:37.756 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a75300 2024-06-11T13:34:37.757 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a75430 2024-06-11T13:34:37.757 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a75560 2024-06-11T13:34:37.757 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a75ee0 2024-06-11T13:34:37.757 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a758e0 2024-06-11T13:34:37.757 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a75a10 2024-06-11T13:34:37.757 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a75b40 2024-06-11T13:34:37.757 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a76030 2024-06-11T13:34:37.757 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a76180 2024-06-11T13:34:37.757 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a76780 2024-06-11T13:34:37.757 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a762d0 2024-06-11T13:34:37.757 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a76400 2024-06-11T13:34:37.758 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a75c50 2024-06-11T13:34:37.758 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a76510 2024-06-11T13:34:37.758 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a76640 2024-06-11T13:34:37.758 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a76890 2024-06-11T13:34:37.758 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a76d80 2024-06-11T13:34:37.758 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a76ed0 2024-06-11T13:34:37.758 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a774d0 2024-06-11T13:34:37.758 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a769e0 2024-06-11T13:34:37.758 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a76b30 2024-06-11T13:34:37.758 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a76c60 2024-06-11T13:34:37.759 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a775e0 2024-06-11T13:34:37.759 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a77020 2024-06-11T13:34:37.759 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a77170 2024-06-11T13:34:37.759 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a772c0 2024-06-11T13:34:37.759 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a77730 2024-06-11T13:34:37.759 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a77c00 2024-06-11T13:34:37.759 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a77d30 2024-06-11T13:34:37.759 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a72080 2024-06-11T13:34:37.759 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a72190 2024-06-11T13:34:37.759 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a722e0 2024-06-11T13:34:37.759 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a77e60 2024-06-11T13:34:37.759 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a78310 2024-06-11T13:34:37.759 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a78460 2024-06-11T13:34:37.760 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a77860 2024-06-11T13:34:37.760 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a77990 2024-06-11T13:34:37.760 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a77ae0 2024-06-11T13:34:37.760 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a78590 2024-06-11T13:34:37.760 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a78a60 2024-06-11T13:34:37.760 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a78b90 2024-06-11T13:34:37.760 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a77f70 2024-06-11T13:34:37.760 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a780a0 2024-06-11T13:34:37.760 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a781d0 2024-06-11T13:34:37.760 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a78cc0 2024-06-11T13:34:37.760 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a79190 2024-06-11T13:34:37.760 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a792e0 2024-06-11T13:34:37.761 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a798c0 2024-06-11T13:34:37.761 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a78df0 2024-06-11T13:34:37.761 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a78f20 2024-06-11T13:34:37.761 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a79050 2024-06-11T13:34:37.761 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a799d0 2024-06-11T13:34:37.761 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a79410 2024-06-11T13:34:37.761 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a79540 2024-06-11T13:34:37.761 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a796b0 2024-06-11T13:34:37.761 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a79b20 2024-06-11T13:34:37.761 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7a010 2024-06-11T13:34:37.761 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7a120 2024-06-11T13:34:37.762 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7a700 2024-06-11T13:34:37.762 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a79c70 2024-06-11T13:34:37.762 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a79dc0 2024-06-11T13:34:37.762 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a79ef0 2024-06-11T13:34:37.762 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7a810 2024-06-11T13:34:37.762 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7a250 2024-06-11T13:34:37.762 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7a380 2024-06-11T13:34:37.762 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7a4d0 2024-06-11T13:34:37.762 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7a920 2024-06-11T13:34:37.762 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7adf0 2024-06-11T13:34:37.762 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7af20 2024-06-11T13:34:37.762 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7a5e0 2024-06-11T13:34:37.763 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7b050 2024-06-11T13:34:37.763 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7b1a0 2024-06-11T13:34:37.763 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7aa30 2024-06-11T13:34:37.763 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7ab60 2024-06-11T13:34:37.763 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7acb0 2024-06-11T13:34:37.763 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7b2f0 2024-06-11T13:34:37.763 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7b7c0 2024-06-11T13:34:37.763 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7b8f0 2024-06-11T13:34:37.763 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7bef0 2024-06-11T13:34:37.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:37 smithi057 ceph-mon[29805]: from='client.36323 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi057-77351-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi057-77351-42"}]': finished 2024-06-11T13:34:37.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:37 smithi057 ceph-mon[29805]: from='client.36317 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi057-77362-52"}]': finished 2024-06-11T13:34:37.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:37 smithi057 ceph-mon[29805]: osdmap e278: 8 total, 8 up, 8 in 2024-06-11T13:34:37.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:37 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2239582370' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi057-77362-53"}]: dispatch 2024-06-11T13:34:37.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:37 smithi057 ceph-mon[29805]: from='client.36329 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi057-77362-53"}]: dispatch 2024-06-11T13:34:37.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:37 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2239582370' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi057-77362-53"}]: dispatch 2024-06-11T13:34:37.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:37 smithi057 ceph-mon[29805]: from='client.36329 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi057-77362-53"}]: dispatch 2024-06-11T13:34:37.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:37 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2239582370' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi057-77362-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:37.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:37 smithi057 ceph-mon[29805]: from='client.36329 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi057-77362-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:37.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:37 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:34:37.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:37 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:34:37.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:37 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:34:37.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:37 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:34:37.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:37 smithi057 ceph-mon[29805]: from='client.36329 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi057-77362-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:34:37.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:37 smithi057 ceph-mon[40588]: from='client.36323 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi057-77351-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi057-77351-42"}]': finished 2024-06-11T13:34:37.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:37 smithi057 ceph-mon[40588]: from='client.36317 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi057-77362-52"}]': finished 2024-06-11T13:34:37.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:37 smithi057 ceph-mon[40588]: osdmap e278: 8 total, 8 up, 8 in 2024-06-11T13:34:37.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:37 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2239582370' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi057-77362-53"}]: dispatch 2024-06-11T13:34:37.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:37 smithi057 ceph-mon[40588]: from='client.36329 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi057-77362-53"}]: dispatch 2024-06-11T13:34:37.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:37 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2239582370' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi057-77362-53"}]: dispatch 2024-06-11T13:34:37.787 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:37 smithi057 ceph-mon[40588]: from='client.36329 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi057-77362-53"}]: dispatch 2024-06-11T13:34:37.787 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:37 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2239582370' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi057-77362-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:37.787 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:37 smithi057 ceph-mon[40588]: from='client.36329 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi057-77362-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:37.787 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:37 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:34:37.787 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:37 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:34:37.787 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:37 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:34:37.787 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:37 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:34:37.787 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:37 smithi057 ceph-mon[40588]: from='client.36329 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi057-77362-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:34:37.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:37 smithi107 ceph-mon[33178]: from='client.36323 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi057-77351-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi057-77351-42"}]': finished 2024-06-11T13:34:37.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:37 smithi107 ceph-mon[33178]: from='client.36317 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi057-77362-52"}]': finished 2024-06-11T13:34:37.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:37 smithi107 ceph-mon[33178]: osdmap e278: 8 total, 8 up, 8 in 2024-06-11T13:34:37.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:37 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2239582370' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi057-77362-53"}]: dispatch 2024-06-11T13:34:37.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:37 smithi107 ceph-mon[33178]: from='client.36329 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi057-77362-53"}]: dispatch 2024-06-11T13:34:37.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:37 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2239582370' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi057-77362-53"}]: dispatch 2024-06-11T13:34:37.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:37 smithi107 ceph-mon[33178]: from='client.36329 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi057-77362-53"}]: dispatch 2024-06-11T13:34:37.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:37 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2239582370' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi057-77362-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:37.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:37 smithi107 ceph-mon[33178]: from='client.36329 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi057-77362-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:37.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:37 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:34:37.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:37 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:34:37.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:37 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:34:37.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:37 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:34:37.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:37 smithi107 ceph-mon[33178]: from='client.36329 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi057-77362-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:34:39.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:38 smithi057 ceph-mon[29805]: pgmap v386: 332 pgs: 8 unknown, 324 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-11T13:34:39.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:38 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:34:39.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:38 smithi057 ceph-mon[29805]: osdmap e279: 8 total, 8 up, 8 in 2024-06-11T13:34:39.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:38 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2239582370' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi057-77362-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi057-77362-53"}]: dispatch 2024-06-11T13:34:39.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:38 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-35"}]: dispatch 2024-06-11T13:34:39.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:38 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-35"}]: dispatch 2024-06-11T13:34:39.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:38 smithi057 ceph-mon[29805]: from='client.36329 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi057-77362-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi057-77362-53"}]: dispatch 2024-06-11T13:34:39.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:38 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-35"}]': finished 2024-06-11T13:34:39.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:38 smithi057 ceph-mon[40588]: pgmap v386: 332 pgs: 8 unknown, 324 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-11T13:34:39.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:38 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:34:39.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:38 smithi057 ceph-mon[40588]: osdmap e279: 8 total, 8 up, 8 in 2024-06-11T13:34:39.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:38 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2239582370' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi057-77362-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi057-77362-53"}]: dispatch 2024-06-11T13:34:39.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:38 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-35"}]: dispatch 2024-06-11T13:34:39.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:38 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-35"}]: dispatch 2024-06-11T13:34:39.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:38 smithi057 ceph-mon[40588]: from='client.36329 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi057-77362-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi057-77362-53"}]: dispatch 2024-06-11T13:34:39.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:38 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-35"}]': finished 2024-06-11T13:34:39.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:38 smithi107 ceph-mon[33178]: pgmap v386: 332 pgs: 8 unknown, 324 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-11T13:34:39.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:38 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:34:39.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:38 smithi107 ceph-mon[33178]: osdmap e279: 8 total, 8 up, 8 in 2024-06-11T13:34:39.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:38 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2239582370' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi057-77362-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi057-77362-53"}]: dispatch 2024-06-11T13:34:39.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:38 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-35"}]: dispatch 2024-06-11T13:34:39.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:38 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-35"}]: dispatch 2024-06-11T13:34:39.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:38 smithi107 ceph-mon[33178]: from='client.36329 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi057-77362-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi057-77362-53"}]: dispatch 2024-06-11T13:34:39.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:38 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-35"}]': finished 2024-06-11T13:34:40.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:39 smithi057 ceph-mon[29805]: osdmap e280: 8 total, 8 up, 8 in 2024-06-11T13:34:40.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:39 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1870887340' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi057-77351-42"}]: dispatch 2024-06-11T13:34:40.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:39 smithi057 ceph-mon[29805]: from='client.36323 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi057-77351-42"}]: dispatch 2024-06-11T13:34:40.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:39 smithi057 ceph-mon[29805]: pgmap v389: 324 pgs: 324 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-11T13:34:40.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:39 smithi057 ceph-mon[29805]: from='client.36329 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi057-77362-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi057-77362-53"}]': finished 2024-06-11T13:34:40.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:39 smithi057 ceph-mon[29805]: from='client.36323 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi057-77351-42"}]': finished 2024-06-11T13:34:40.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:39 smithi057 ceph-mon[29805]: osdmap e281: 8 total, 8 up, 8 in 2024-06-11T13:34:40.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:39 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1870887340' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi057-77351-42"}]: dispatch 2024-06-11T13:34:40.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:39 smithi057 ceph-mon[29805]: from='client.36323 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi057-77351-42"}]: dispatch 2024-06-11T13:34:40.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:39 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:34:40.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:39 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:34:40.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:39 smithi057 ceph-mon[40588]: osdmap e280: 8 total, 8 up, 8 in 2024-06-11T13:34:40.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:39 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1870887340' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi057-77351-42"}]: dispatch 2024-06-11T13:34:40.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:39 smithi057 ceph-mon[40588]: from='client.36323 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi057-77351-42"}]: dispatch 2024-06-11T13:34:40.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:39 smithi057 ceph-mon[40588]: pgmap v389: 324 pgs: 324 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-11T13:34:40.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:39 smithi057 ceph-mon[40588]: from='client.36329 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi057-77362-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi057-77362-53"}]': finished 2024-06-11T13:34:40.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:39 smithi057 ceph-mon[40588]: from='client.36323 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi057-77351-42"}]': finished 2024-06-11T13:34:40.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:39 smithi057 ceph-mon[40588]: osdmap e281: 8 total, 8 up, 8 in 2024-06-11T13:34:40.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:39 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1870887340' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi057-77351-42"}]: dispatch 2024-06-11T13:34:40.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:39 smithi057 ceph-mon[40588]: from='client.36323 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi057-77351-42"}]: dispatch 2024-06-11T13:34:40.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:39 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:34:40.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:39 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:34:40.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:39 smithi107 ceph-mon[33178]: osdmap e280: 8 total, 8 up, 8 in 2024-06-11T13:34:40.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:39 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1870887340' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi057-77351-42"}]: dispatch 2024-06-11T13:34:40.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:39 smithi107 ceph-mon[33178]: from='client.36323 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi057-77351-42"}]: dispatch 2024-06-11T13:34:40.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:39 smithi107 ceph-mon[33178]: pgmap v389: 324 pgs: 324 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-11T13:34:40.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:39 smithi107 ceph-mon[33178]: from='client.36329 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi057-77362-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi057-77362-53"}]': finished 2024-06-11T13:34:40.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:39 smithi107 ceph-mon[33178]: from='client.36323 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi057-77351-42"}]': finished 2024-06-11T13:34:40.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:39 smithi107 ceph-mon[33178]: osdmap e281: 8 total, 8 up, 8 in 2024-06-11T13:34:40.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:39 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1870887340' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi057-77351-42"}]: dispatch 2024-06-11T13:34:40.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:39 smithi107 ceph-mon[33178]: from='client.36323 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi057-77351-42"}]: dispatch 2024-06-11T13:34:40.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:39 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:34:40.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:39 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:34:40.367 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: 2024-06-11T13:34:40.367 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleWrite (7049 ms) 2024-06-11T13:34:40.367 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ RUN ] LibRadosAioEC.WaitForComplete 2024-06-11T13:34:40.367 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ OK ] LibRadosAioEC.WaitForComplete (6914 ms) 2024-06-11T13:34:40.367 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip 2024-06-11T13:34:40.367 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip (7071 ms) 2024-06-11T13:34:40.367 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip2 2024-06-11T13:34:40.367 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip2 (6969 ms) 2024-06-11T13:34:40.367 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripAppend 2024-06-11T13:34:40.367 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripAppend (7045 ms) 2024-06-11T13:34:40.367 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ RUN ] LibRadosAioEC.IsComplete 2024-06-11T13:34:40.368 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ OK ] LibRadosAioEC.IsComplete (7053 ms) 2024-06-11T13:34:40.368 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ RUN ] LibRadosAioEC.IsSafe 2024-06-11T13:34:40.368 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ OK ] LibRadosAioEC.IsSafe (7051 ms) 2024-06-11T13:34:40.368 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ RUN ] LibRadosAioEC.ReturnValue 2024-06-11T13:34:40.368 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ OK ] LibRadosAioEC.ReturnValue (7061 ms) 2024-06-11T13:34:40.368 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ RUN ] LibRadosAioEC.Flush 2024-06-11T13:34:40.368 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ OK ] LibRadosAioEC.Flush (6844 ms) 2024-06-11T13:34:40.368 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ RUN ] LibRadosAioEC.FlushAsync 2024-06-11T13:34:40.368 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ OK ] LibRadosAioEC.FlushAsync (7034 ms) 2024-06-11T13:34:40.368 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripWriteFull 2024-06-11T13:34:40.368 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripWriteFull (6990 ms) 2024-06-11T13:34:40.368 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStat 2024-06-11T13:34:40.368 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStat (7056 ms) 2024-06-11T13:34:40.369 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStatNS 2024-06-11T13:34:40.369 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStatNS (6947 ms) 2024-06-11T13:34:40.369 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ RUN ] LibRadosAioEC.StatRemove 2024-06-11T13:34:40.369 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ OK ] LibRadosAioEC.StatRemove (7448 ms) 2024-06-11T13:34:40.369 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ RUN ] LibRadosAioEC.ExecuteClass 2024-06-11T13:34:40.369 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ OK ] LibRadosAioEC.ExecuteClass (7060 ms) 2024-06-11T13:34:40.369 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ RUN ] LibRadosAioEC.MultiWrite 2024-06-11T13:34:40.369 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ OK ] LibRadosAioEC.MultiWrite (7066 ms) 2024-06-11T13:34:40.369 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [----------] 16 tests from LibRadosAioEC (112658 ms total) 2024-06-11T13:34:40.369 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: 2024-06-11T13:34:40.369 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [----------] Global test environment tear-down 2024-06-11T13:34:40.369 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [==========] 42 tests from 2 test suites ran. (236808 ms total) 2024-06-11T13:34:40.369 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio: [ PASSED ] 42 tests. 2024-06-11T13:34:41.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:41 smithi057 ceph-mon[29805]: from='client.36323 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi057-77351-42"}]': finished 2024-06-11T13:34:41.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:41 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:34:41.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:41 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-35"}]: dispatch 2024-06-11T13:34:41.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:41 smithi057 ceph-mon[29805]: osdmap e282: 8 total, 8 up, 8 in 2024-06-11T13:34:41.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:41 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-35"}]: dispatch 2024-06-11T13:34:41.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:41 smithi057 ceph-mon[40588]: from='client.36323 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi057-77351-42"}]': finished 2024-06-11T13:34:41.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:41 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:34:41.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:41 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-35"}]: dispatch 2024-06-11T13:34:41.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:41 smithi057 ceph-mon[40588]: osdmap e282: 8 total, 8 up, 8 in 2024-06-11T13:34:41.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:41 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-35"}]: dispatch 2024-06-11T13:34:41.797 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:34:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:34:41.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:41 smithi107 ceph-mon[33178]: from='client.36323 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi057-77351-42"}]': finished 2024-06-11T13:34:41.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:41 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:34:41.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:41 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-35"}]: dispatch 2024-06-11T13:34:41.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:41 smithi107 ceph-mon[33178]: osdmap e282: 8 total, 8 up, 8 in 2024-06-11T13:34:41.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:41 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-35"}]: dispatch 2024-06-11T13:34:43.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:42 smithi057 ceph-mon[29805]: pgmap v392: 332 pgs: 332 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 5.7 KiB/s rd, 4.0 KiB/s wr, 4 op/s 2024-06-11T13:34:43.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:42 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:34:43.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:42 smithi057 ceph-mon[29805]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:34:43.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:42 smithi057 ceph-mon[29805]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:34:43.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:42 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-35"}]': finished 2024-06-11T13:34:43.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:42 smithi057 ceph-mon[29805]: osdmap e283: 8 total, 8 up, 8 in 2024-06-11T13:34:43.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:42 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2239582370' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi057-77362-53"}]: dispatch 2024-06-11T13:34:43.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:42 smithi057 ceph-mon[29805]: from='client.36329 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi057-77362-53"}]: dispatch 2024-06-11T13:34:43.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:42 smithi057 ceph-mon[40588]: pgmap v392: 332 pgs: 332 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 5.7 KiB/s rd, 4.0 KiB/s wr, 4 op/s 2024-06-11T13:34:43.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:42 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:34:43.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:42 smithi057 ceph-mon[40588]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:34:43.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:42 smithi057 ceph-mon[40588]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:34:43.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:42 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-35"}]': finished 2024-06-11T13:34:43.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:42 smithi057 ceph-mon[40588]: osdmap e283: 8 total, 8 up, 8 in 2024-06-11T13:34:43.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:42 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2239582370' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi057-77362-53"}]: dispatch 2024-06-11T13:34:43.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:42 smithi057 ceph-mon[40588]: from='client.36329 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi057-77362-53"}]: dispatch 2024-06-11T13:34:43.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:42 smithi107 ceph-mon[33178]: pgmap v392: 332 pgs: 332 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 5.7 KiB/s rd, 4.0 KiB/s wr, 4 op/s 2024-06-11T13:34:43.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:42 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:34:43.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:42 smithi107 ceph-mon[33178]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:34:43.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:42 smithi107 ceph-mon[33178]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:34:43.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:42 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-35"}]': finished 2024-06-11T13:34:43.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:42 smithi107 ceph-mon[33178]: osdmap e283: 8 total, 8 up, 8 in 2024-06-11T13:34:43.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:42 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2239582370' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi057-77362-53"}]: dispatch 2024-06-11T13:34:43.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:42 smithi107 ceph-mon[33178]: from='client.36329 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi057-77362-53"}]: dispatch 2024-06-11T13:34:44.034 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:34:43 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:34:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:34:44.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:43 smithi057 ceph-mon[29805]: from='client.36329 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi057-77362-53"}]': finished 2024-06-11T13:34:44.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:43 smithi057 ceph-mon[29805]: osdmap e284: 8 total, 8 up, 8 in 2024-06-11T13:34:44.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:43 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2239582370' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi057-77362-53"}]: dispatch 2024-06-11T13:34:44.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:43 smithi057 ceph-mon[29805]: from='client.36329 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi057-77362-53"}]: dispatch 2024-06-11T13:34:44.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:43 smithi057 ceph-mon[29805]: pgmap v395: 292 pgs: 292 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 255 B/s wr, 2 op/s 2024-06-11T13:34:44.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:43 smithi057 ceph-mon[29805]: from='client.36329 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi057-77362-53"}]': finished 2024-06-11T13:34:44.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:43 smithi057 ceph-mon[29805]: osdmap e285: 8 total, 8 up, 8 in 2024-06-11T13:34:44.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:43 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:34:44.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:43 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:34:44.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:43 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/825723723' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi057-77362-54"}]: dispatch 2024-06-11T13:34:44.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:43 smithi057 ceph-mon[29805]: from='client.36335 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi057-77362-54"}]: dispatch 2024-06-11T13:34:44.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:43 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/825723723' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi057-77362-54"}]: dispatch 2024-06-11T13:34:44.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:43 smithi057 ceph-mon[29805]: from='client.36335 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi057-77362-54"}]: dispatch 2024-06-11T13:34:44.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:43 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/825723723' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi057-77362-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:44.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:43 smithi057 ceph-mon[29805]: from='client.36335 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi057-77362-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:44.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:43 smithi057 ceph-mon[40588]: from='client.36329 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi057-77362-53"}]': finished 2024-06-11T13:34:44.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:43 smithi057 ceph-mon[40588]: osdmap e284: 8 total, 8 up, 8 in 2024-06-11T13:34:44.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:43 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2239582370' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi057-77362-53"}]: dispatch 2024-06-11T13:34:44.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:43 smithi057 ceph-mon[40588]: from='client.36329 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi057-77362-53"}]: dispatch 2024-06-11T13:34:44.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:43 smithi057 ceph-mon[40588]: pgmap v395: 292 pgs: 292 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 255 B/s wr, 2 op/s 2024-06-11T13:34:44.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:43 smithi057 ceph-mon[40588]: from='client.36329 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi057-77362-53"}]': finished 2024-06-11T13:34:44.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:43 smithi057 ceph-mon[40588]: osdmap e285: 8 total, 8 up, 8 in 2024-06-11T13:34:44.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:43 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:34:44.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:43 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:34:44.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:43 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/825723723' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi057-77362-54"}]: dispatch 2024-06-11T13:34:44.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:43 smithi057 ceph-mon[40588]: from='client.36335 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi057-77362-54"}]: dispatch 2024-06-11T13:34:44.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:43 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/825723723' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi057-77362-54"}]: dispatch 2024-06-11T13:34:44.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:43 smithi057 ceph-mon[40588]: from='client.36335 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi057-77362-54"}]: dispatch 2024-06-11T13:34:44.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:43 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/825723723' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi057-77362-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:44.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:43 smithi057 ceph-mon[40588]: from='client.36335 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi057-77362-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:44.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:43 smithi107 ceph-mon[33178]: from='client.36329 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi057-77362-53"}]': finished 2024-06-11T13:34:44.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:43 smithi107 ceph-mon[33178]: osdmap e284: 8 total, 8 up, 8 in 2024-06-11T13:34:44.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:43 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2239582370' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi057-77362-53"}]: dispatch 2024-06-11T13:34:44.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:43 smithi107 ceph-mon[33178]: from='client.36329 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi057-77362-53"}]: dispatch 2024-06-11T13:34:44.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:43 smithi107 ceph-mon[33178]: pgmap v395: 292 pgs: 292 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 255 B/s wr, 2 op/s 2024-06-11T13:34:44.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:43 smithi107 ceph-mon[33178]: from='client.36329 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi057-77362-53"}]': finished 2024-06-11T13:34:44.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:43 smithi107 ceph-mon[33178]: osdmap e285: 8 total, 8 up, 8 in 2024-06-11T13:34:44.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:43 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:34:44.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:43 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:34:44.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:43 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/825723723' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi057-77362-54"}]: dispatch 2024-06-11T13:34:44.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:43 smithi107 ceph-mon[33178]: from='client.36335 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi057-77362-54"}]: dispatch 2024-06-11T13:34:44.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:43 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/825723723' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi057-77362-54"}]: dispatch 2024-06-11T13:34:44.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:43 smithi107 ceph-mon[33178]: from='client.36335 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi057-77362-54"}]: dispatch 2024-06-11T13:34:44.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:43 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/825723723' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi057-77362-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:44.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:43 smithi107 ceph-mon[33178]: from='client.36335 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi057-77362-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:45.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:45 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:34:45.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:45 smithi057 ceph-mon[29805]: from='client.36335 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi057-77362-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:34:45.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:45 smithi057 ceph-mon[29805]: osdmap e286: 8 total, 8 up, 8 in 2024-06-11T13:34:45.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:45 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/825723723' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi057-77362-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi057-77362-54"}]: dispatch 2024-06-11T13:34:45.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:45 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:34:45.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:45 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:34:45.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:45 smithi057 ceph-mon[29805]: from='client.36335 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi057-77362-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi057-77362-54"}]: dispatch 2024-06-11T13:34:45.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:45 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:34:45.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:45 smithi057 ceph-mon[40588]: from='client.36335 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi057-77362-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:34:45.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:45 smithi057 ceph-mon[40588]: osdmap e286: 8 total, 8 up, 8 in 2024-06-11T13:34:45.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:45 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/825723723' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi057-77362-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi057-77362-54"}]: dispatch 2024-06-11T13:34:45.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:45 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:34:45.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:45 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:34:45.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:45 smithi057 ceph-mon[40588]: from='client.36335 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi057-77362-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi057-77362-54"}]: dispatch 2024-06-11T13:34:45.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:45 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:34:45.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:45 smithi107 ceph-mon[33178]: from='client.36335 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi057-77362-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:34:45.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:45 smithi107 ceph-mon[33178]: osdmap e286: 8 total, 8 up, 8 in 2024-06-11T13:34:45.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:45 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/825723723' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi057-77362-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi057-77362-54"}]: dispatch 2024-06-11T13:34:45.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:45 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:34:45.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:45 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:34:45.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:45 smithi107 ceph-mon[33178]: from='client.36335 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi057-77362-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi057-77362-54"}]: dispatch 2024-06-11T13:34:46.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:46 smithi057 ceph-mon[29805]: pgmap v398: 324 pgs: 32 unknown, 292 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:34:46.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:46 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-37", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:34:46.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:46 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-37"}]: dispatch 2024-06-11T13:34:46.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:46 smithi057 ceph-mon[29805]: osdmap e287: 8 total, 8 up, 8 in 2024-06-11T13:34:46.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:46 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-37"}]: dispatch 2024-06-11T13:34:46.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:46 smithi057 ceph-mon[29805]: from='client.36335 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi057-77362-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi057-77362-54"}]': finished 2024-06-11T13:34:46.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:46 smithi057 ceph-mon[29805]: osdmap e288: 8 total, 8 up, 8 in 2024-06-11T13:34:46.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:46 smithi057 ceph-mon[40588]: pgmap v398: 324 pgs: 32 unknown, 292 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:34:46.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:46 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-37", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:34:46.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:46 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-37"}]: dispatch 2024-06-11T13:34:46.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:46 smithi057 ceph-mon[40588]: osdmap e287: 8 total, 8 up, 8 in 2024-06-11T13:34:46.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:46 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-37"}]: dispatch 2024-06-11T13:34:46.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:46 smithi057 ceph-mon[40588]: from='client.36335 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi057-77362-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi057-77362-54"}]': finished 2024-06-11T13:34:46.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:46 smithi057 ceph-mon[40588]: osdmap e288: 8 total, 8 up, 8 in 2024-06-11T13:34:46.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:46 smithi107 ceph-mon[33178]: pgmap v398: 324 pgs: 32 unknown, 292 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:34:46.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:46 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-37", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:34:46.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:46 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-37"}]: dispatch 2024-06-11T13:34:46.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:46 smithi107 ceph-mon[33178]: osdmap e287: 8 total, 8 up, 8 in 2024-06-11T13:34:46.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:46 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-37"}]: dispatch 2024-06-11T13:34:46.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:46 smithi107 ceph-mon[33178]: from='client.36335 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi057-77362-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi057-77362-54"}]': finished 2024-06-11T13:34:46.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:46 smithi107 ceph-mon[33178]: osdmap e288: 8 total, 8 up, 8 in 2024-06-11T13:34:48.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:47 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-37"}]': finished 2024-06-11T13:34:48.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:47 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-37", "mode": "writeback"}]: dispatch 2024-06-11T13:34:48.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:47 smithi057 ceph-mon[40588]: osdmap e289: 8 total, 8 up, 8 in 2024-06-11T13:34:48.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:47 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-37", "mode": "writeback"}]: dispatch 2024-06-11T13:34:48.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:47 smithi057 ceph-mon[40588]: pgmap v402: 332 pgs: 6 creating+peering, 2 unknown, 324 active+clean; 4.6 MiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-06-11T13:34:48.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:47 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-37"}]': finished 2024-06-11T13:34:48.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:47 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-37", "mode": "writeback"}]: dispatch 2024-06-11T13:34:48.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:47 smithi057 ceph-mon[29805]: osdmap e289: 8 total, 8 up, 8 in 2024-06-11T13:34:48.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:47 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-37", "mode": "writeback"}]: dispatch 2024-06-11T13:34:48.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:47 smithi057 ceph-mon[29805]: pgmap v402: 332 pgs: 6 creating+peering, 2 unknown, 324 active+clean; 4.6 MiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-06-11T13:34:48.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:47 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-37"}]': finished 2024-06-11T13:34:48.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:47 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-37", "mode": "writeback"}]: dispatch 2024-06-11T13:34:48.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:47 smithi107 ceph-mon[33178]: osdmap e289: 8 total, 8 up, 8 in 2024-06-11T13:34:48.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:47 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-37", "mode": "writeback"}]: dispatch 2024-06-11T13:34:48.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:47 smithi107 ceph-mon[33178]: pgmap v402: 332 pgs: 6 creating+peering, 2 unknown, 324 active+clean; 4.6 MiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-06-11T13:34:49.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:48 smithi057 ceph-mon[40588]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:34:49.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:48 smithi057 ceph-mon[40588]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:34:49.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:48 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-37", "mode": "writeback"}]': finished 2024-06-11T13:34:49.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:48 smithi057 ceph-mon[40588]: osdmap e290: 8 total, 8 up, 8 in 2024-06-11T13:34:49.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:48 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/825723723' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi057-77362-54"}]: dispatch 2024-06-11T13:34:49.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:48 smithi057 ceph-mon[40588]: from='client.36335 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi057-77362-54"}]: dispatch 2024-06-11T13:34:49.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:48 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:34:49.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:48 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:34:49.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:48 smithi057 ceph-mon[29805]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:34:49.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:48 smithi057 ceph-mon[29805]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:34:49.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:48 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-37", "mode": "writeback"}]': finished 2024-06-11T13:34:49.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:48 smithi057 ceph-mon[29805]: osdmap e290: 8 total, 8 up, 8 in 2024-06-11T13:34:49.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:48 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/825723723' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi057-77362-54"}]: dispatch 2024-06-11T13:34:49.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:48 smithi057 ceph-mon[29805]: from='client.36335 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi057-77362-54"}]: dispatch 2024-06-11T13:34:49.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:48 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:34:49.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:48 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:34:49.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:48 smithi107 ceph-mon[33178]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:34:49.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:48 smithi107 ceph-mon[33178]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:34:49.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:48 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-37", "mode": "writeback"}]': finished 2024-06-11T13:34:49.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:48 smithi107 ceph-mon[33178]: osdmap e290: 8 total, 8 up, 8 in 2024-06-11T13:34:49.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:48 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/825723723' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi057-77362-54"}]: dispatch 2024-06-11T13:34:49.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:48 smithi107 ceph-mon[33178]: from='client.36335 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi057-77362-54"}]: dispatch 2024-06-11T13:34:49.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:48 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:34:49.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:48 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:34:50.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:49 smithi057 ceph-mon[29805]: from='client.36335 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi057-77362-54"}]': finished 2024-06-11T13:34:50.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:49 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:34:50.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:49 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-37"}]: dispatch 2024-06-11T13:34:50.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:49 smithi057 ceph-mon[29805]: osdmap e291: 8 total, 8 up, 8 in 2024-06-11T13:34:50.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:49 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-37"}]: dispatch 2024-06-11T13:34:50.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:49 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/825723723' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi057-77362-54"}]: dispatch 2024-06-11T13:34:50.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:49 smithi057 ceph-mon[29805]: from='client.36335 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi057-77362-54"}]: dispatch 2024-06-11T13:34:50.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:49 smithi057 ceph-mon[29805]: pgmap v405: 324 pgs: 324 active+clean; 4.6 MiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-06-11T13:34:50.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:49 smithi057 ceph-mon[29805]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:34:50.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:49 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-37"}]': finished 2024-06-11T13:34:50.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:49 smithi057 ceph-mon[40588]: from='client.36335 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi057-77362-54"}]': finished 2024-06-11T13:34:50.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:49 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:34:50.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:49 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-37"}]: dispatch 2024-06-11T13:34:50.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:49 smithi057 ceph-mon[40588]: osdmap e291: 8 total, 8 up, 8 in 2024-06-11T13:34:50.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:49 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-37"}]: dispatch 2024-06-11T13:34:50.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:49 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/825723723' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi057-77362-54"}]: dispatch 2024-06-11T13:34:50.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:49 smithi057 ceph-mon[40588]: from='client.36335 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi057-77362-54"}]: dispatch 2024-06-11T13:34:50.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:49 smithi057 ceph-mon[40588]: pgmap v405: 324 pgs: 324 active+clean; 4.6 MiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-06-11T13:34:50.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:49 smithi057 ceph-mon[40588]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:34:50.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:49 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-37"}]': finished 2024-06-11T13:34:50.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:49 smithi107 ceph-mon[33178]: from='client.36335 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi057-77362-54"}]': finished 2024-06-11T13:34:50.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:49 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:34:50.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:49 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-37"}]: dispatch 2024-06-11T13:34:50.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:49 smithi107 ceph-mon[33178]: osdmap e291: 8 total, 8 up, 8 in 2024-06-11T13:34:50.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:49 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-37"}]: dispatch 2024-06-11T13:34:50.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:49 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/825723723' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi057-77362-54"}]: dispatch 2024-06-11T13:34:50.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:49 smithi107 ceph-mon[33178]: from='client.36335 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi057-77362-54"}]: dispatch 2024-06-11T13:34:50.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:49 smithi107 ceph-mon[33178]: pgmap v405: 324 pgs: 324 active+clean; 4.6 MiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-06-11T13:34:50.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:49 smithi107 ceph-mon[33178]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:34:50.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:49 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-37"}]': finished 2024-06-11T13:34:51.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:50 smithi057 ceph-mon[29805]: from='client.36335 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi057-77362-54"}]': finished 2024-06-11T13:34:51.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:50 smithi057 ceph-mon[29805]: osdmap e292: 8 total, 8 up, 8 in 2024-06-11T13:34:51.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:50 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2793709328' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi057-77362-55"}]: dispatch 2024-06-11T13:34:51.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:50 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2793709328' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi057-77362-55"}]: dispatch 2024-06-11T13:34:51.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:50 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2793709328' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi057-77362-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:51.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:50 smithi057 ceph-mon[40588]: from='client.36335 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi057-77362-54"}]': finished 2024-06-11T13:34:51.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:50 smithi057 ceph-mon[40588]: osdmap e292: 8 total, 8 up, 8 in 2024-06-11T13:34:51.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:50 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2793709328' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi057-77362-55"}]: dispatch 2024-06-11T13:34:51.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:50 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2793709328' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi057-77362-55"}]: dispatch 2024-06-11T13:34:51.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:50 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2793709328' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi057-77362-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:51.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:50 smithi107 ceph-mon[33178]: from='client.36335 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi057-77362-54"}]': finished 2024-06-11T13:34:51.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:50 smithi107 ceph-mon[33178]: osdmap e292: 8 total, 8 up, 8 in 2024-06-11T13:34:51.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:50 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2793709328' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi057-77362-55"}]: dispatch 2024-06-11T13:34:51.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:50 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2793709328' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi057-77362-55"}]: dispatch 2024-06-11T13:34:51.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:50 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2793709328' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi057-77362-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:51.797 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:34:51 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:34:52.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:51 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2793709328' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi057-77362-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:34:52.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:51 smithi057 ceph-mon[40588]: osdmap e293: 8 total, 8 up, 8 in 2024-06-11T13:34:52.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:51 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2793709328' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi057-77362-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi057-77362-55"}]: dispatch 2024-06-11T13:34:52.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:51 smithi057 ceph-mon[40588]: pgmap v408: 292 pgs: 292 active+clean; 4.6 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-06-11T13:34:52.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:51 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:34:52.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:51 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:34:52.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:51 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:34:52.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:51 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2793709328' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi057-77362-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:34:52.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:51 smithi057 ceph-mon[29805]: osdmap e293: 8 total, 8 up, 8 in 2024-06-11T13:34:52.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:51 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2793709328' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi057-77362-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi057-77362-55"}]: dispatch 2024-06-11T13:34:52.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:51 smithi057 ceph-mon[29805]: pgmap v408: 292 pgs: 292 active+clean; 4.6 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-06-11T13:34:52.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:51 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:34:52.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:51 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:34:52.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:51 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:34:52.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:51 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2793709328' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi057-77362-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:34:52.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:51 smithi107 ceph-mon[33178]: osdmap e293: 8 total, 8 up, 8 in 2024-06-11T13:34:52.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:51 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2793709328' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi057-77362-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi057-77362-55"}]: dispatch 2024-06-11T13:34:52.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:51 smithi107 ceph-mon[33178]: pgmap v408: 292 pgs: 292 active+clean; 4.6 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-06-11T13:34:52.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:51 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:34:52.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:51 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:34:52.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:51 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:34:53.185 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:52 smithi057 ceph-mon[29805]: osdmap e294: 8 total, 8 up, 8 in 2024-06-11T13:34:53.185 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:52 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:34:53.186 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:52 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:34:53.186 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:52 smithi057 ceph-mon[40588]: osdmap e294: 8 total, 8 up, 8 in 2024-06-11T13:34:53.186 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:52 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:34:53.186 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:52 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:34:53.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:52 smithi107 ceph-mon[33178]: osdmap e294: 8 total, 8 up, 8 in 2024-06-11T13:34:53.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:52 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:34:53.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:52 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:34:54.034 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:34:53 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:34:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:34:54.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:53 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2793709328' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi057-77362-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi057-77362-55"}]': finished 2024-06-11T13:34:54.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:53 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:34:54.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:53 smithi057 ceph-mon[29805]: osdmap e295: 8 total, 8 up, 8 in 2024-06-11T13:34:54.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:53 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:34:54.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:53 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:34:54.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:53 smithi057 ceph-mon[29805]: pgmap v411: 332 pgs: 40 unknown, 292 active+clean; 4.6 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-06-11T13:34:54.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:53 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2793709328' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi057-77362-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi057-77362-55"}]': finished 2024-06-11T13:34:54.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:53 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:34:54.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:53 smithi057 ceph-mon[40588]: osdmap e295: 8 total, 8 up, 8 in 2024-06-11T13:34:54.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:53 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:34:54.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:53 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:34:54.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:53 smithi057 ceph-mon[40588]: pgmap v411: 332 pgs: 40 unknown, 292 active+clean; 4.6 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-06-11T13:34:54.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:53 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2793709328' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi057-77362-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi057-77362-55"}]': finished 2024-06-11T13:34:54.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:53 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:34:54.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:53 smithi107 ceph-mon[33178]: osdmap e295: 8 total, 8 up, 8 in 2024-06-11T13:34:54.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:53 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:34:54.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:53 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:34:54.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:53 smithi107 ceph-mon[33178]: pgmap v411: 332 pgs: 40 unknown, 292 active+clean; 4.6 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-06-11T13:34:55.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:54 smithi057 ceph-mon[40588]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:34:55.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:54 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-39", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:34:55.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:54 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-39"}]: dispatch 2024-06-11T13:34:55.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:54 smithi057 ceph-mon[40588]: osdmap e296: 8 total, 8 up, 8 in 2024-06-11T13:34:55.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:54 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-39"}]: dispatch 2024-06-11T13:34:55.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:54 smithi057 ceph-mon[29805]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:34:55.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:54 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-39", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:34:55.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:54 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-39"}]: dispatch 2024-06-11T13:34:55.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:54 smithi057 ceph-mon[29805]: osdmap e296: 8 total, 8 up, 8 in 2024-06-11T13:34:55.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:54 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-39"}]: dispatch 2024-06-11T13:34:55.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:54 smithi107 ceph-mon[33178]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:34:55.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:54 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-39", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:34:55.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:54 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-39"}]: dispatch 2024-06-11T13:34:55.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:54 smithi107 ceph-mon[33178]: osdmap e296: 8 total, 8 up, 8 in 2024-06-11T13:34:55.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:54 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-39"}]: dispatch 2024-06-11T13:34:56.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:55 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-39"}]': finished 2024-06-11T13:34:56.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:55 smithi057 ceph-mon[29805]: osdmap e297: 8 total, 8 up, 8 in 2024-06-11T13:34:56.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:55 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-39", "mode": "writeback"}]: dispatch 2024-06-11T13:34:56.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:55 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-39", "mode": "writeback"}]: dispatch 2024-06-11T13:34:56.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:55 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2793709328' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi057-77362-55"}]: dispatch 2024-06-11T13:34:56.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:55 smithi057 ceph-mon[29805]: pgmap v414: 324 pgs: 32 unknown, 292 active+clean; 4.6 MiB data, 609 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:34:56.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:55 smithi057 ceph-mon[29805]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:34:56.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:55 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-39", "mode": "writeback"}]': finished 2024-06-11T13:34:56.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:55 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2793709328' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi057-77362-55"}]': finished 2024-06-11T13:34:56.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:55 smithi057 ceph-mon[29805]: osdmap e298: 8 total, 8 up, 8 in 2024-06-11T13:34:56.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:55 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-39"}]': finished 2024-06-11T13:34:56.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:55 smithi057 ceph-mon[40588]: osdmap e297: 8 total, 8 up, 8 in 2024-06-11T13:34:56.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:55 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-39", "mode": "writeback"}]: dispatch 2024-06-11T13:34:56.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:55 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-39", "mode": "writeback"}]: dispatch 2024-06-11T13:34:56.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:55 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2793709328' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi057-77362-55"}]: dispatch 2024-06-11T13:34:56.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:55 smithi057 ceph-mon[40588]: pgmap v414: 324 pgs: 32 unknown, 292 active+clean; 4.6 MiB data, 609 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:34:56.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:55 smithi057 ceph-mon[40588]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:34:56.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:55 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-39", "mode": "writeback"}]': finished 2024-06-11T13:34:56.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:55 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2793709328' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi057-77362-55"}]': finished 2024-06-11T13:34:56.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:55 smithi057 ceph-mon[40588]: osdmap e298: 8 total, 8 up, 8 in 2024-06-11T13:34:56.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:55 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-39"}]': finished 2024-06-11T13:34:56.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:55 smithi107 ceph-mon[33178]: osdmap e297: 8 total, 8 up, 8 in 2024-06-11T13:34:56.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:55 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-39", "mode": "writeback"}]: dispatch 2024-06-11T13:34:56.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:55 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-39", "mode": "writeback"}]: dispatch 2024-06-11T13:34:56.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:55 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2793709328' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi057-77362-55"}]: dispatch 2024-06-11T13:34:56.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:55 smithi107 ceph-mon[33178]: pgmap v414: 324 pgs: 32 unknown, 292 active+clean; 4.6 MiB data, 609 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:34:56.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:55 smithi107 ceph-mon[33178]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:34:56.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:55 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-39", "mode": "writeback"}]': finished 2024-06-11T13:34:56.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:55 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2793709328' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi057-77362-55"}]': finished 2024-06-11T13:34:56.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:55 smithi107 ceph-mon[33178]: osdmap e298: 8 total, 8 up, 8 in 2024-06-11T13:34:57.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:57 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2793709328' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi057-77362-55"}]: dispatch 2024-06-11T13:34:57.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:57 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:34:57.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:57 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:34:57.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:57 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2793709328' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi057-77362-55"}]': finished 2024-06-11T13:34:57.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:56 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2793709328' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi057-77362-55"}]: dispatch 2024-06-11T13:34:57.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:56 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:34:57.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:56 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:34:57.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:56 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2793709328' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi057-77362-55"}]': finished 2024-06-11T13:34:57.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:56 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2793709328' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi057-77362-55"}]: dispatch 2024-06-11T13:34:57.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:56 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:34:57.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:56 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:34:57.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:56 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2793709328' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi057-77362-55"}]': finished 2024-06-11T13:34:58.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:58 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:34:58.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:58 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-39"}]: dispatch 2024-06-11T13:34:58.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:58 smithi057 ceph-mon[29805]: osdmap e299: 8 total, 8 up, 8 in 2024-06-11T13:34:58.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:58 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-39"}]: dispatch 2024-06-11T13:34:58.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:58 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/25100391' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi057-77362-56"}]: dispatch 2024-06-11T13:34:58.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:58 smithi057 ceph-mon[29805]: from='client.36347 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi057-77362-56"}]: dispatch 2024-06-11T13:34:58.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:58 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/25100391' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi057-77362-56"}]: dispatch 2024-06-11T13:34:58.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:58 smithi057 ceph-mon[29805]: from='client.36347 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi057-77362-56"}]: dispatch 2024-06-11T13:34:58.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:58 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/25100391' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi057-77362-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:58.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:58 smithi057 ceph-mon[29805]: from='client.36347 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi057-77362-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:58.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:58 smithi057 ceph-mon[29805]: pgmap v417: 324 pgs: 324 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 9 op/s 2024-06-11T13:34:58.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:58 smithi057 ceph-mon[29805]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:34:58.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:58 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-39"}]': finished 2024-06-11T13:34:58.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:58 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:34:58.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:58 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-39"}]: dispatch 2024-06-11T13:34:58.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:58 smithi057 ceph-mon[40588]: osdmap e299: 8 total, 8 up, 8 in 2024-06-11T13:34:58.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:58 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-39"}]: dispatch 2024-06-11T13:34:58.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:58 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/25100391' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi057-77362-56"}]: dispatch 2024-06-11T13:34:58.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:58 smithi057 ceph-mon[40588]: from='client.36347 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi057-77362-56"}]: dispatch 2024-06-11T13:34:58.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:58 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/25100391' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi057-77362-56"}]: dispatch 2024-06-11T13:34:58.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:58 smithi057 ceph-mon[40588]: from='client.36347 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi057-77362-56"}]: dispatch 2024-06-11T13:34:58.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:58 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/25100391' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi057-77362-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:58.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:58 smithi057 ceph-mon[40588]: from='client.36347 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi057-77362-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:58.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:58 smithi057 ceph-mon[40588]: pgmap v417: 324 pgs: 324 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 9 op/s 2024-06-11T13:34:58.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:58 smithi057 ceph-mon[40588]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:34:58.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:58 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-39"}]': finished 2024-06-11T13:34:58.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:58 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:34:58.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:58 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-39"}]: dispatch 2024-06-11T13:34:58.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:58 smithi107 ceph-mon[33178]: osdmap e299: 8 total, 8 up, 8 in 2024-06-11T13:34:58.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:58 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-39"}]: dispatch 2024-06-11T13:34:58.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:58 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/25100391' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi057-77362-56"}]: dispatch 2024-06-11T13:34:58.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:58 smithi107 ceph-mon[33178]: from='client.36347 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi057-77362-56"}]: dispatch 2024-06-11T13:34:58.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:58 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/25100391' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi057-77362-56"}]: dispatch 2024-06-11T13:34:58.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:58 smithi107 ceph-mon[33178]: from='client.36347 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi057-77362-56"}]: dispatch 2024-06-11T13:34:58.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:58 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/25100391' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi057-77362-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:58.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:58 smithi107 ceph-mon[33178]: from='client.36347 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi057-77362-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:34:58.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:58 smithi107 ceph-mon[33178]: pgmap v417: 324 pgs: 324 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 9 op/s 2024-06-11T13:34:58.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:58 smithi107 ceph-mon[33178]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:34:58.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:58 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-39"}]': finished 2024-06-11T13:34:59.448 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [==========] Running 77 tests from 4 test suites. 2024-06-11T13:34:59.448 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [----------] Global test environment set-up. 2024-06-11T13:34:59.448 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP 2024-06-11T13:34:59.448 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: seed 77599 2024-06-11T13:34:59.448 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.Dirty 2024-06-11T13:34:59.449 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTierPP.Dirty (253 ms) 2024-06-11T13:34:59.449 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.FlushWriteRaces 2024-06-11T13:34:59.449 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTierPP.FlushWriteRaces (28842 ms) 2024-06-11T13:34:59.449 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.HitSetNone 2024-06-11T13:34:59.449 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTierPP.HitSetNone (33 ms) 2024-06-11T13:34:59.449 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP (29128 ms total) 2024-06-11T13:34:59.449 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: 2024-06-11T13:34:59.449 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP 2024-06-11T13:34:59.449 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Overlay 2024-06-11T13:34:59.449 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Overlay (9946 ms) 2024-06-11T13:34:59.450 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Promote 2024-06-11T13:34:59.450 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Promote (10678 ms) 2024-06-11T13:34:59.450 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnap 2024-06-11T13:34:59.450 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnap (13521 ms) 2024-06-11T13:34:59.450 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapScrub 2024-06-11T13:34:59.450 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: my_snaps [3] 2024-06-11T13:34:59.450 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: my_snaps [4,3] 2024-06-11T13:34:59.450 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: my_snaps [5,4,3] 2024-06-11T13:34:59.450 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: my_snaps [6,5,4,3] 2024-06-11T13:34:59.450 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: promoting some heads 2024-06-11T13:34:59.450 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: promoting from clones for snap 6 2024-06-11T13:34:59.450 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: promoting from clones for snap 5 2024-06-11T13:34:59.451 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: promoting from clones for snap 4 2024-06-11T13:34:59.451 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: promoting from clones for snap 3 2024-06-11T13:34:59.451 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: waiting for scrubs... 2024-06-11T13:34:59.451 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: done waiting 2024-06-11T13:34:59.451 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapScrub (46011 ms) 2024-06-11T13:34:59.451 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2024-06-11T13:34:59.451 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapTrimRace (10070 ms) 2024-06-11T13:34:59.451 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Whiteout 2024-06-11T13:34:59.451 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Whiteout (7970 ms) 2024-06-11T13:34:59.451 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2024-06-11T13:34:59.451 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate (8040 ms) 2024-06-11T13:34:59.451 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Evict 2024-06-11T13:34:59.451 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Evict (8064 ms) 2024-06-11T13:34:59.451 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap 2024-06-11T13:34:59.452 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap (10078 ms) 2024-06-11T13:34:59.452 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap2 2024-06-11T13:34:59.452 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap2 (8856 ms) 2024-06-11T13:34:59.452 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ListSnap 2024-06-11T13:34:59.452 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ListSnap (10109 ms) 2024-06-11T13:34:59.452 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace 2024-06-11T13:34:59.452 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace (14006 ms) 2024-06-11T13:34:59.452 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlush 2024-06-11T13:34:59.452 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlush (8891 ms) 2024-06-11T13:34:59.452 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Flush 2024-06-11T13:34:59.452 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Flush (7529 ms) 2024-06-11T13:34:59.452 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushSnap 2024-06-11T13:34:59.452 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushSnap (13113 ms) 2024-06-11T13:34:59.452 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2024-06-11T13:34:59.453 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushTryFlushRaces (8526 ms) 2024-06-11T13:34:59.453 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlushReadRace 2024-06-11T13:34:59.453 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlushReadRace (8541 ms) 2024-06-11T13:34:59.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:59 smithi057 ceph-mon[29805]: from='client.36347 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi057-77362-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:34:59.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:59 smithi057 ceph-mon[29805]: osdmap e300: 8 total, 8 up, 8 in 2024-06-11T13:34:59.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:59 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/25100391' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi057-77362-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi057-77362-56"}]: dispatch 2024-06-11T13:34:59.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:34:59 smithi057 ceph-mon[29805]: from='client.36347 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi057-77362-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi057-77362-56"}]: dispatch 2024-06-11T13:34:59.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:59 smithi057 ceph-mon[40588]: from='client.36347 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi057-77362-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:34:59.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:59 smithi057 ceph-mon[40588]: osdmap e300: 8 total, 8 up, 8 in 2024-06-11T13:34:59.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:59 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/25100391' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi057-77362-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi057-77362-56"}]: dispatch 2024-06-11T13:34:59.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:34:59 smithi057 ceph-mon[40588]: from='client.36347 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi057-77362-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi057-77362-56"}]: dispatch 2024-06-11T13:34:59.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:59 smithi107 ceph-mon[33178]: from='client.36347 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi057-77362-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:34:59.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:59 smithi107 ceph-mon[33178]: osdmap e300: 8 total, 8 up, 8 in 2024-06-11T13:34:59.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:59 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/25100391' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi057-77362-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi057-77362-56"}]: dispatch 2024-06-11T13:34:59.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:34:59 smithi107 ceph-mon[33178]: from='client.36347 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi057-77362-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi057-77362-56"}]: dispatch 2024-06-11T13:35:00.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:00 smithi057 ceph-mon[29805]: pgmap v420: 292 pgs: 292 active+clean; 4.6 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:35:00.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:00 smithi057 ceph-mon[29805]: osdmap e301: 8 total, 8 up, 8 in 2024-06-11T13:35:00.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:00 smithi057 ceph-mon[29805]: from='client.36347 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi057-77362-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi057-77362-56"}]': finished 2024-06-11T13:35:00.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:00 smithi057 ceph-mon[29805]: osdmap e302: 8 total, 8 up, 8 in 2024-06-11T13:35:00.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:00 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:35:00.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:00 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:35:00.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:00 smithi057 ceph-mon[40588]: pgmap v420: 292 pgs: 292 active+clean; 4.6 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:35:00.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:00 smithi057 ceph-mon[40588]: osdmap e301: 8 total, 8 up, 8 in 2024-06-11T13:35:00.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:00 smithi057 ceph-mon[40588]: from='client.36347 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi057-77362-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi057-77362-56"}]': finished 2024-06-11T13:35:00.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:00 smithi057 ceph-mon[40588]: osdmap e302: 8 total, 8 up, 8 in 2024-06-11T13:35:00.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:00 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:35:00.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:00 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:35:00.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:00 smithi107 ceph-mon[33178]: pgmap v420: 292 pgs: 292 active+clean; 4.6 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:35:00.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:00 smithi107 ceph-mon[33178]: osdmap e301: 8 total, 8 up, 8 in 2024-06-11T13:35:00.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:00 smithi107 ceph-mon[33178]: from='client.36347 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi057-77362-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi057-77362-56"}]': finished 2024-06-11T13:35:00.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:00 smithi107 ceph-mon[33178]: osdmap e302: 8 total, 8 up, 8 in 2024-06-11T13:35:00.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:00 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:35:00.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:00 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:35:01.699 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:35:01 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:35:01.700 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:01 smithi107 ceph-mon[33178]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:35:01.700 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:01 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:35:01.700 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:01 smithi107 ceph-mon[33178]: osdmap e303: 8 total, 8 up, 8 in 2024-06-11T13:35:01.700 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:01 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:35:01.701 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:01 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:35:01.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:01 smithi057 ceph-mon[40588]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:35:01.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:01 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:35:01.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:01 smithi057 ceph-mon[40588]: osdmap e303: 8 total, 8 up, 8 in 2024-06-11T13:35:01.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:01 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:35:01.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:01 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:35:01.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:01 smithi057 ceph-mon[29805]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:35:01.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:01 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:35:01.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:01 smithi057 ceph-mon[29805]: osdmap e303: 8 total, 8 up, 8 in 2024-06-11T13:35:01.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:01 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:35:01.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:01 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:35:02.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:02 smithi057 ceph-mon[29805]: pgmap v423: 332 pgs: 40 creating+peering, 292 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:35:02.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:02 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:35:02.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:02 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-41", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:35:02.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:02 smithi057 ceph-mon[29805]: osdmap e304: 8 total, 8 up, 8 in 2024-06-11T13:35:02.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:02 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-41","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:35:02.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:02 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-41","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:35:02.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:02 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/25100391' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi057-77362-56"}]: dispatch 2024-06-11T13:35:02.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:02 smithi057 ceph-mon[29805]: from='client.36347 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi057-77362-56"}]: dispatch 2024-06-11T13:35:02.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:02 smithi057 ceph-mon[40588]: pgmap v423: 332 pgs: 40 creating+peering, 292 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:35:02.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:02 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:35:02.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:02 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-41", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:35:02.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:02 smithi057 ceph-mon[40588]: osdmap e304: 8 total, 8 up, 8 in 2024-06-11T13:35:02.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:02 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-41","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:35:02.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:02 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-41","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:35:02.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:02 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/25100391' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi057-77362-56"}]: dispatch 2024-06-11T13:35:02.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:02 smithi057 ceph-mon[40588]: from='client.36347 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi057-77362-56"}]: dispatch 2024-06-11T13:35:02.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:02 smithi107 ceph-mon[33178]: pgmap v423: 332 pgs: 40 creating+peering, 292 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:35:02.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:02 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:35:02.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:02 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-41", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:35:02.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:02 smithi107 ceph-mon[33178]: osdmap e304: 8 total, 8 up, 8 in 2024-06-11T13:35:02.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:02 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-41","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:35:02.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:02 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-41","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:35:02.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:02 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/25100391' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi057-77362-56"}]: dispatch 2024-06-11T13:35:02.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:02 smithi107 ceph-mon[33178]: from='client.36347 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi057-77362-56"}]: dispatch 2024-06-11T13:35:04.034 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:35:03 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:35:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:35:04.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:04 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-41","var": "hit_set_count","val": "2"}]': finished 2024-06-11T13:35:04.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:04 smithi107 ceph-mon[33178]: from='client.36347 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi057-77362-56"}]': finished 2024-06-11T13:35:04.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:04 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-41","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:35:04.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:04 smithi107 ceph-mon[33178]: osdmap e305: 8 total, 8 up, 8 in 2024-06-11T13:35:04.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:04 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-41","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:35:04.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:04 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/25100391' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi057-77362-56"}]: dispatch 2024-06-11T13:35:04.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:04 smithi107 ceph-mon[33178]: from='client.36347 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi057-77362-56"}]: dispatch 2024-06-11T13:35:04.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:04 smithi107 ceph-mon[33178]: pgmap v426: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-06-11T13:35:04.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:04 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-41","var": "hit_set_count","val": "2"}]': finished 2024-06-11T13:35:04.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:04 smithi057 ceph-mon[40588]: from='client.36347 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi057-77362-56"}]': finished 2024-06-11T13:35:04.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:04 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-41","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:35:04.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:04 smithi057 ceph-mon[40588]: osdmap e305: 8 total, 8 up, 8 in 2024-06-11T13:35:04.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:04 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-41","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:35:04.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:04 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/25100391' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi057-77362-56"}]: dispatch 2024-06-11T13:35:04.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:04 smithi057 ceph-mon[40588]: from='client.36347 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi057-77362-56"}]: dispatch 2024-06-11T13:35:04.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:04 smithi057 ceph-mon[40588]: pgmap v426: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-06-11T13:35:04.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:04 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-41","var": "hit_set_count","val": "2"}]': finished 2024-06-11T13:35:04.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:04 smithi057 ceph-mon[29805]: from='client.36347 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi057-77362-56"}]': finished 2024-06-11T13:35:04.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:04 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-41","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:35:04.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:04 smithi057 ceph-mon[29805]: osdmap e305: 8 total, 8 up, 8 in 2024-06-11T13:35:04.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:04 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-41","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:35:04.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:04 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/25100391' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi057-77362-56"}]: dispatch 2024-06-11T13:35:04.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:04 smithi057 ceph-mon[29805]: from='client.36347 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi057-77362-56"}]: dispatch 2024-06-11T13:35:04.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:04 smithi057 ceph-mon[29805]: pgmap v426: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-06-11T13:35:05.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:05 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-41","var": "hit_set_period","val": "600"}]': finished 2024-06-11T13:35:05.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:05 smithi057 ceph-mon[29805]: from='client.36347 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi057-77362-56"}]': finished 2024-06-11T13:35:05.538 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:05 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-11T13:35:05.538 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:05 smithi057 ceph-mon[29805]: osdmap e306: 8 total, 8 up, 8 in 2024-06-11T13:35:05.538 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:05 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-11T13:35:05.538 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:05 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/4232513613' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi057-77362-57"}]: dispatch 2024-06-11T13:35:05.538 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:05 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/4232513613' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi057-77362-57"}]: dispatch 2024-06-11T13:35:05.538 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:05 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/4232513613' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi057-77362-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:35:05.538 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:05 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-41","var": "hit_set_period","val": "600"}]': finished 2024-06-11T13:35:05.538 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:05 smithi057 ceph-mon[40588]: from='client.36347 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi057-77362-56"}]': finished 2024-06-11T13:35:05.538 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:05 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-11T13:35:05.538 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:05 smithi057 ceph-mon[40588]: osdmap e306: 8 total, 8 up, 8 in 2024-06-11T13:35:05.539 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:05 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-11T13:35:05.539 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:05 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/4232513613' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi057-77362-57"}]: dispatch 2024-06-11T13:35:05.539 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:05 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/4232513613' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi057-77362-57"}]: dispatch 2024-06-11T13:35:05.539 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:05 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/4232513613' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi057-77362-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:35:05.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:05 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-41","var": "hit_set_period","val": "600"}]': finished 2024-06-11T13:35:05.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:05 smithi107 ceph-mon[33178]: from='client.36347 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi057-77362-56"}]': finished 2024-06-11T13:35:05.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:05 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-11T13:35:05.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:05 smithi107 ceph-mon[33178]: osdmap e306: 8 total, 8 up, 8 in 2024-06-11T13:35:05.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:05 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-11T13:35:05.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:05 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/4232513613' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi057-77362-57"}]: dispatch 2024-06-11T13:35:05.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:05 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/4232513613' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi057-77362-57"}]: dispatch 2024-06-11T13:35:05.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:05 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/4232513613' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi057-77362-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:35:06.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:06 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-06-11T13:35:06.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:06 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/4232513613' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi057-77362-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:35:06.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:06 smithi057 ceph-mon[29805]: osdmap e307: 8 total, 8 up, 8 in 2024-06-11T13:35:06.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:06 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/4232513613' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi057-77362-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi057-77362-57"}]: dispatch 2024-06-11T13:35:06.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:06 smithi057 ceph-mon[29805]: pgmap v429: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:35:06.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:06 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-06-11T13:35:06.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:06 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/4232513613' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi057-77362-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:35:06.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:06 smithi057 ceph-mon[40588]: osdmap e307: 8 total, 8 up, 8 in 2024-06-11T13:35:06.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:06 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/4232513613' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi057-77362-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi057-77362-57"}]: dispatch 2024-06-11T13:35:06.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:06 smithi057 ceph-mon[40588]: pgmap v429: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:35:06.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:06 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-06-11T13:35:06.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:06 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/4232513613' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi057-77362-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:35:06.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:06 smithi107 ceph-mon[33178]: osdmap e307: 8 total, 8 up, 8 in 2024-06-11T13:35:06.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:06 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/4232513613' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi057-77362-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi057-77362-57"}]: dispatch 2024-06-11T13:35:06.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:06 smithi107 ceph-mon[33178]: pgmap v429: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:35:07.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:07 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:35:07.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:07 smithi057 ceph-mon[29805]: osdmap e308: 8 total, 8 up, 8 in 2024-06-11T13:35:07.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:07 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:35:07.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:07 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:35:07.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:07 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-41"}]: dispatch 2024-06-11T13:35:07.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:07 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-41"}]: dispatch 2024-06-11T13:35:07.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:07 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/4232513613' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi057-77362-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi057-77362-57"}]': finished 2024-06-11T13:35:07.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:07 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-41"}]': finished 2024-06-11T13:35:07.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:07 smithi057 ceph-mon[29805]: osdmap e309: 8 total, 8 up, 8 in 2024-06-11T13:35:07.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:07 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:35:07.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:07 smithi057 ceph-mon[40588]: osdmap e308: 8 total, 8 up, 8 in 2024-06-11T13:35:07.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:07 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:35:07.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:07 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:35:07.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:07 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-41"}]: dispatch 2024-06-11T13:35:07.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:07 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-41"}]: dispatch 2024-06-11T13:35:07.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:07 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/4232513613' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi057-77362-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi057-77362-57"}]': finished 2024-06-11T13:35:07.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:07 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-41"}]': finished 2024-06-11T13:35:07.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:07 smithi057 ceph-mon[40588]: osdmap e309: 8 total, 8 up, 8 in 2024-06-11T13:35:07.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:07 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:35:07.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:07 smithi107 ceph-mon[33178]: osdmap e308: 8 total, 8 up, 8 in 2024-06-11T13:35:07.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:07 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:35:07.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:07 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:35:07.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:07 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-41"}]: dispatch 2024-06-11T13:35:07.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:07 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-41"}]: dispatch 2024-06-11T13:35:07.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:07 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/4232513613' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi057-77362-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi057-77362-57"}]': finished 2024-06-11T13:35:07.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:07 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-41"}]': finished 2024-06-11T13:35:07.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:07 smithi107 ceph-mon[33178]: osdmap e309: 8 total, 8 up, 8 in 2024-06-11T13:35:08.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:08 smithi057 ceph-mon[29805]: pgmap v432: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:35:08.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:08 smithi057 ceph-mon[29805]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:35:08.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:08 smithi057 ceph-mon[29805]: osdmap e310: 8 total, 8 up, 8 in 2024-06-11T13:35:08.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:08 smithi057 ceph-mon[40588]: pgmap v432: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:35:08.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:08 smithi057 ceph-mon[40588]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:35:08.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:08 smithi057 ceph-mon[40588]: osdmap e310: 8 total, 8 up, 8 in 2024-06-11T13:35:08.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:08 smithi107 ceph-mon[33178]: pgmap v432: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:35:08.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:08 smithi107 ceph-mon[33178]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:35:08.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:08 smithi107 ceph-mon[33178]: osdmap e310: 8 total, 8 up, 8 in 2024-06-11T13:35:10.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:10 smithi057 ceph-mon[29805]: osdmap e311: 8 total, 8 up, 8 in 2024-06-11T13:35:10.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:10 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:35:10.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:10 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:35:10.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:10 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/4232513613' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi057-77362-57"}]: dispatch 2024-06-11T13:35:10.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:10 smithi057 ceph-mon[29805]: pgmap v435: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:35:10.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:10 smithi057 ceph-mon[40588]: osdmap e311: 8 total, 8 up, 8 in 2024-06-11T13:35:10.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:10 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:35:10.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:10 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:35:10.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:10 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/4232513613' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi057-77362-57"}]: dispatch 2024-06-11T13:35:10.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:10 smithi057 ceph-mon[40588]: pgmap v435: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:35:10.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:10 smithi107 ceph-mon[33178]: osdmap e311: 8 total, 8 up, 8 in 2024-06-11T13:35:10.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:10 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:35:10.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:10 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:35:10.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:10 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/4232513613' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi057-77362-57"}]: dispatch 2024-06-11T13:35:10.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:10 smithi107 ceph-mon[33178]: pgmap v435: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:35:11.335 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:11 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:35:11.336 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:11 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/4232513613' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi057-77362-57"}]': finished 2024-06-11T13:35:11.336 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:11 smithi107 ceph-mon[33178]: osdmap e312: 8 total, 8 up, 8 in 2024-06-11T13:35:11.336 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:11 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi057-77599-6","var": "pg_num","format": "json"}]: dispatch 2024-06-11T13:35:11.336 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:11 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/4232513613' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi057-77362-57"}]: dispatch 2024-06-11T13:35:11.336 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:11 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:35:11.336 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:11 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:35:11.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:11 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:35:11.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:11 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/4232513613' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi057-77362-57"}]': finished 2024-06-11T13:35:11.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:11 smithi057 ceph-mon[40588]: osdmap e312: 8 total, 8 up, 8 in 2024-06-11T13:35:11.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:11 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi057-77599-6","var": "pg_num","format": "json"}]: dispatch 2024-06-11T13:35:11.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:11 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/4232513613' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi057-77362-57"}]: dispatch 2024-06-11T13:35:11.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:11 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:35:11.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:11 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:35:11.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:11 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:35:11.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:11 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/4232513613' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi057-77362-57"}]': finished 2024-06-11T13:35:11.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:11 smithi057 ceph-mon[29805]: osdmap e312: 8 total, 8 up, 8 in 2024-06-11T13:35:11.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:11 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi057-77599-6","var": "pg_num","format": "json"}]: dispatch 2024-06-11T13:35:11.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:11 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/4232513613' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi057-77362-57"}]: dispatch 2024-06-11T13:35:11.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:11 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:35:11.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:11 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:35:11.797 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:35:11 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:35:12.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:12 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/4232513613' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi057-77362-57"}]': finished 2024-06-11T13:35:12.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:12 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-43", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:35:12.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:12 smithi057 ceph-mon[40588]: osdmap e313: 8 total, 8 up, 8 in 2024-06-11T13:35:12.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:12 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-43","var": "hit_set_count","val": "8"}]: dispatch 2024-06-11T13:35:12.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:12 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-43","var": "hit_set_count","val": "8"}]: dispatch 2024-06-11T13:35:12.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:12 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1097988569' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi057-77362-58"}]: dispatch 2024-06-11T13:35:12.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:12 smithi057 ceph-mon[40588]: from='client.36359 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi057-77362-58"}]: dispatch 2024-06-11T13:35:12.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:12 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1097988569' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi057-77362-58"}]: dispatch 2024-06-11T13:35:12.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:12 smithi057 ceph-mon[40588]: from='client.36359 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi057-77362-58"}]: dispatch 2024-06-11T13:35:12.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:12 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1097988569' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi057-77362-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:35:12.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:12 smithi057 ceph-mon[40588]: from='client.36359 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi057-77362-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:35:12.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:12 smithi057 ceph-mon[40588]: pgmap v438: 324 pgs: 324 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:35:12.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:12 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:35:12.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:12 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/4232513613' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi057-77362-57"}]': finished 2024-06-11T13:35:12.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:12 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-43", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:35:12.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:12 smithi057 ceph-mon[29805]: osdmap e313: 8 total, 8 up, 8 in 2024-06-11T13:35:12.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:12 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-43","var": "hit_set_count","val": "8"}]: dispatch 2024-06-11T13:35:12.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:12 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-43","var": "hit_set_count","val": "8"}]: dispatch 2024-06-11T13:35:12.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:12 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1097988569' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi057-77362-58"}]: dispatch 2024-06-11T13:35:12.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:12 smithi057 ceph-mon[29805]: from='client.36359 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi057-77362-58"}]: dispatch 2024-06-11T13:35:12.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:12 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1097988569' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi057-77362-58"}]: dispatch 2024-06-11T13:35:12.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:12 smithi057 ceph-mon[29805]: from='client.36359 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi057-77362-58"}]: dispatch 2024-06-11T13:35:12.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:12 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1097988569' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi057-77362-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:35:12.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:12 smithi057 ceph-mon[29805]: from='client.36359 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi057-77362-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:35:12.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:12 smithi057 ceph-mon[29805]: pgmap v438: 324 pgs: 324 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:35:12.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:12 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:35:12.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:12 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/4232513613' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi057-77362-57"}]': finished 2024-06-11T13:35:12.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:12 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-43", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:35:12.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:12 smithi107 ceph-mon[33178]: osdmap e313: 8 total, 8 up, 8 in 2024-06-11T13:35:12.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:12 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-43","var": "hit_set_count","val": "8"}]: dispatch 2024-06-11T13:35:12.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:12 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-43","var": "hit_set_count","val": "8"}]: dispatch 2024-06-11T13:35:12.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:12 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1097988569' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi057-77362-58"}]: dispatch 2024-06-11T13:35:12.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:12 smithi107 ceph-mon[33178]: from='client.36359 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi057-77362-58"}]: dispatch 2024-06-11T13:35:12.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:12 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1097988569' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi057-77362-58"}]: dispatch 2024-06-11T13:35:12.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:12 smithi107 ceph-mon[33178]: from='client.36359 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi057-77362-58"}]: dispatch 2024-06-11T13:35:12.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:12 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1097988569' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi057-77362-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:35:12.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:12 smithi107 ceph-mon[33178]: from='client.36359 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi057-77362-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:35:12.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:12 smithi107 ceph-mon[33178]: pgmap v438: 324 pgs: 324 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:35:12.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:12 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:35:14.034 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:35:13 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:35:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:35:14.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:13 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-43","var": "hit_set_count","val": "8"}]': finished 2024-06-11T13:35:14.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:13 smithi057 ceph-mon[29805]: from='client.36359 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi057-77362-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:35:14.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:13 smithi057 ceph-mon[29805]: osdmap e314: 8 total, 8 up, 8 in 2024-06-11T13:35:14.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:13 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-43","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:35:14.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:13 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1097988569' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi057-77362-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi057-77362-58"}]: dispatch 2024-06-11T13:35:14.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:13 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-43","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:35:14.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:13 smithi057 ceph-mon[29805]: from='client.36359 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi057-77362-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi057-77362-58"}]: dispatch 2024-06-11T13:35:14.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:13 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-43","var": "hit_set_period","val": "600"}]': finished 2024-06-11T13:35:14.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:13 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-43","var": "hit_set_count","val": "8"}]': finished 2024-06-11T13:35:14.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:13 smithi057 ceph-mon[40588]: from='client.36359 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi057-77362-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:35:14.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:13 smithi057 ceph-mon[40588]: osdmap e314: 8 total, 8 up, 8 in 2024-06-11T13:35:14.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:13 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-43","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:35:14.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:13 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1097988569' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi057-77362-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi057-77362-58"}]: dispatch 2024-06-11T13:35:14.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:13 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-43","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:35:14.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:13 smithi057 ceph-mon[40588]: from='client.36359 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi057-77362-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi057-77362-58"}]: dispatch 2024-06-11T13:35:14.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:13 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-43","var": "hit_set_period","val": "600"}]': finished 2024-06-11T13:35:14.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:13 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-43","var": "hit_set_count","val": "8"}]': finished 2024-06-11T13:35:14.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:13 smithi107 ceph-mon[33178]: from='client.36359 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi057-77362-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:35:14.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:13 smithi107 ceph-mon[33178]: osdmap e314: 8 total, 8 up, 8 in 2024-06-11T13:35:14.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:13 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-43","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:35:14.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:13 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1097988569' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi057-77362-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi057-77362-58"}]: dispatch 2024-06-11T13:35:14.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:13 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-43","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:35:14.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:13 smithi107 ceph-mon[33178]: from='client.36359 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi057-77362-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi057-77362-58"}]: dispatch 2024-06-11T13:35:14.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:13 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-43","var": "hit_set_period","val": "600"}]': finished 2024-06-11T13:35:14.347 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetRead 2024-06-11T13:35:14.347 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: hmm, no HitSet yet 2024-06-11T13:35:14.348 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: ok, hit_set contains 275:602f83fe:::foo:head 2024-06-11T13:35:14.348 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetRead (8617 ms) 2024-06-11T13:35:14.348 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetWrite 2024-06-11T13:35:14.348 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: pg_num = 32 2024-06-11T13:35:14.348 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: pg 0 ls 1718112915,0 2024-06-11T13:35:14.348 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: pg 1 ls 1718112915,0 2024-06-11T13:35:14.348 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: pg 2 ls 1718112915,0 2024-06-11T13:35:14.348 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: pg 3 ls 1718112915,0 2024-06-11T13:35:14.348 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: pg 4 ls 1718112915,0 2024-06-11T13:35:14.348 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: pg 5 ls 1718112915,0 2024-06-11T13:35:14.348 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: pg 6 ls 1718112915,0 2024-06-11T13:35:14.348 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: pg 7 ls 1718112915,0 2024-06-11T13:35:14.348 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: pg 8 ls 1718112915,0 2024-06-11T13:35:14.349 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: pg 9 ls 1718112915,0 2024-06-11T13:35:14.349 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: pg 10 ls 1718112915,0 2024-06-11T13:35:14.349 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: pg 11 ls 1718112915,0 2024-06-11T13:35:14.349 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: pg 12 ls 1718112915,0 2024-06-11T13:35:14.349 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: pg 13 ls 1718112915,0 2024-06-11T13:35:14.349 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: pg 14 ls 1718112915,0 2024-06-11T13:35:14.349 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: pg 15 ls 1718112915,0 2024-06-11T13:35:14.349 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: pg 16 ls 1718112915,0 2024-06-11T13:35:14.349 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: pg 17 ls 1718112915,0 2024-06-11T13:35:14.349 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: pg 18 ls 1718112915,0 2024-06-11T13:35:14.349 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: pg 19 ls 1718112915,0 2024-06-11T13:35:14.349 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: pg 20 ls 1718112915,0 2024-06-11T13:35:14.349 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: pg 21 ls 1718112915,0 2024-06-11T13:35:14.349 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: pg 22 ls 1718112915,0 2024-06-11T13:35:14.350 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: pg 23 ls 1718112915,0 2024-06-11T13:35:14.350 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: pg 24 ls 1718112915,0 2024-06-11T13:35:14.350 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: pg 25 ls 1718112915,0 2024-06-11T13:35:14.350 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: pg 26 ls 1718112915,0 2024-06-11T13:35:14.350 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: pg 27 ls 1718112915,0 2024-06-11T13:35:14.350 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: pg 28 ls 1718112915,0 2024-06-11T13:35:14.350 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: pg 29 ls 1718112915,0 2024-06-11T13:35:14.350 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: pg 30 ls 1718112915,0 2024-06-11T13:35:14.350 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: pg 31 ls 1718112915,0 2024-06-11T13:35:14.350 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: pg_num = 32 2024-06-11T13:35:14.350 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:6cac518f:::0:head 2024-06-11T13:35:14.350 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:02547ec2:::1:head 2024-06-11T13:35:14.350 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:f905c69b:::2:head 2024-06-11T13:35:14.351 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:cfc208b3:::3:head 2024-06-11T13:35:14.351 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:d83876eb:::4:head 2024-06-11T13:35:14.351 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:b29083e3:::5:head 2024-06-11T13:35:14.351 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:c4fdafeb:::6:head 2024-06-11T13:35:14.351 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:5c6b0b28:::7:head 2024-06-11T13:35:14.351 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:bd63b0f1:::8:head 2024-06-11T13:35:14.351 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:e960b815:::9:head 2024-06-11T13:35:14.351 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:52ea6a34:::10:head 2024-06-11T13:35:14.351 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:89d3ae78:::11:head 2024-06-11T13:35:14.351 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:de5d7c5f:::12:head 2024-06-11T13:35:14.351 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:566253c9:::13:head 2024-06-11T13:35:14.351 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:62a1935d:::14:head 2024-06-11T13:35:14.352 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:863748b0:::15:head 2024-06-11T13:35:14.352 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:3958e169:::16:head 2024-06-11T13:35:14.352 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:4d4dabf9:::17:head 2024-06-11T13:35:14.352 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:8391935d:::18:head 2024-06-11T13:35:14.352 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:28883081:::19:head 2024-06-11T13:35:14.352 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:69259c59:::20:head 2024-06-11T13:35:14.352 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:4bdb80b7:::21:head 2024-06-11T13:35:14.352 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:a11c5d71:::22:head 2024-06-11T13:35:14.352 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:271af37b:::23:head 2024-06-11T13:35:14.353 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:95b121be:::24:head 2024-06-11T13:35:14.353 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:58d1031b:::25:head 2024-06-11T13:35:14.353 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:0a050783:::26:head 2024-06-11T13:35:14.353 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:c709704c:::27:head 2024-06-11T13:35:14.353 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:cbe56eaf:::28:head 2024-06-11T13:35:14.353 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:86b4b162:::29:head 2024-06-11T13:35:14.353 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:70d89383:::30:head 2024-06-11T13:35:14.353 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:dd450c7c:::31:head 2024-06-11T13:35:14.353 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:6d5729b1:::32:head 2024-06-11T13:35:14.353 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:c388f3fb:::33:head 2024-06-11T13:35:14.353 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:56cfea31:::34:head 2024-06-11T13:35:14.353 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:9dbc1bf7:::35:head 2024-06-11T13:35:14.353 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:40b74ccd:::36:head 2024-06-11T13:35:14.354 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:4d5aaf42:::37:head 2024-06-11T13:35:14.354 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:920f362c:::38:head 2024-06-11T13:35:14.354 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:6cc53222:::39:head 2024-06-11T13:35:14.354 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:9cad833f:::40:head 2024-06-11T13:35:14.354 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:1ea84d41:::41:head 2024-06-11T13:35:14.354 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:c4480ef6:::42:head 2024-06-11T13:35:14.354 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:a694361e:::43:head 2024-06-11T13:35:14.354 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:d1bd33e9:::44:head 2024-06-11T13:35:14.354 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:ddc2cd5d:::45:head 2024-06-11T13:35:14.354 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:2b782207:::46:head 2024-06-11T13:35:14.354 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:7b187fca:::47:head 2024-06-11T13:35:14.354 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:90ecdf6f:::48:head 2024-06-11T13:35:14.354 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:a5ed95fe:::49:head 2024-06-11T13:35:14.355 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:ea0eaa55:::50:head 2024-06-11T13:35:14.355 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:f33ef17b:::51:head 2024-06-11T13:35:14.355 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:a0d1b2f6:::52:head 2024-06-11T13:35:14.355 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:60c5229e:::53:head 2024-06-11T13:35:14.355 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:edcbc575:::54:head 2024-06-11T13:35:14.355 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:102cf253:::55:head 2024-06-11T13:35:14.355 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:efb7fb0b:::56:head 2024-06-11T13:35:14.355 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:50d0a326:::57:head 2024-06-11T13:35:14.355 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:d4dc5daf:::58:head 2024-06-11T13:35:14.355 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:3a130462:::59:head 2024-06-11T13:35:14.355 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:ec87ed71:::60:head 2024-06-11T13:35:14.355 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:d5bc9454:::61:head 2024-06-11T13:35:14.355 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:3ddfe313:::62:head 2024-06-11T13:35:14.356 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:7c2816b9:::63:head 2024-06-11T13:35:14.356 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:47e00e4d:::64:head 2024-06-11T13:35:14.356 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:c6410c18:::65:head 2024-06-11T13:35:14.356 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:b48ed237:::66:head 2024-06-11T13:35:14.356 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:cd63ad31:::67:head 2024-06-11T13:35:14.356 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:b179e92b:::68:head 2024-06-11T13:35:14.356 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:0d9f741a:::69:head 2024-06-11T13:35:14.356 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:6d3352ae:::70:head 2024-06-11T13:35:14.356 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:c6d5c19e:::71:head 2024-06-11T13:35:14.356 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:bc4729c3:::72:head 2024-06-11T13:35:14.356 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:77e930b9:::73:head 2024-06-11T13:35:14.356 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:0abeecfd:::74:head 2024-06-11T13:35:14.356 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:b7c37e15:::75:head 2024-06-11T13:35:14.356 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:b6378398:::76:head 2024-06-11T13:35:14.357 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:02bd68de:::77:head 2024-06-11T13:35:14.357 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:cc795d2d:::78:head 2024-06-11T13:35:14.357 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:630d4fea:::79:head 2024-06-11T13:35:14.357 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:e0d29ef5:::80:head 2024-06-11T13:35:14.357 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:fd6f13d2:::81:head 2024-06-11T13:35:14.357 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:606461d5:::82:head 2024-06-11T13:35:14.357 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:eadbdc43:::83:head 2024-06-11T13:35:14.357 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:8761d0bb:::84:head 2024-06-11T13:35:14.357 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:9ef0186f:::85:head 2024-06-11T13:35:14.357 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:e0d41294:::86:head 2024-06-11T13:35:14.357 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:961de695:::87:head 2024-06-11T13:35:14.357 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:1423148f:::88:head 2024-06-11T13:35:14.357 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:633a8fa2:::89:head 2024-06-11T13:35:14.357 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:a8653809:::90:head 2024-06-11T13:35:14.358 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:3dac8b33:::91:head 2024-06-11T13:35:14.358 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:35aad435:::92:head 2024-06-11T13:35:14.358 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:f6dcc343:::93:head 2024-06-11T13:35:14.358 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:dbbdad87:::94:head 2024-06-11T13:35:14.358 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:1cb48ce0:::95:head 2024-06-11T13:35:14.358 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:03cd461c:::96:head 2024-06-11T13:35:14.358 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:17a4ea99:::97:head 2024-06-11T13:35:14.358 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:9993c9a7:::98:head 2024-06-11T13:35:14.358 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:6394211c:::99:head 2024-06-11T13:35:14.358 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:94c7ae57:::100:head 2024-06-11T13:35:14.358 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:6fdee5bb:::101:head 2024-06-11T13:35:14.358 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:9a477fd1:::102:head 2024-06-11T13:35:14.358 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:eb850916:::103:head 2024-06-11T13:35:14.358 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:affc56b9:::104:head 2024-06-11T13:35:14.359 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:b42dc814:::105:head 2024-06-11T13:35:14.359 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:f319f8f0:::106:head 2024-06-11T13:35:14.359 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:9a40b9de:::107:head 2024-06-11T13:35:14.359 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:8b524f28:::108:head 2024-06-11T13:35:14.359 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:e3de589f:::109:head 2024-06-11T13:35:14.359 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:90f90a5b:::110:head 2024-06-11T13:35:14.359 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:a7b4f1d7:::111:head 2024-06-11T13:35:14.359 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:af51766e:::112:head 2024-06-11T13:35:14.359 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:b6f90bd1:::113:head 2024-06-11T13:35:14.359 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:e0261208:::114:head 2024-06-11T13:35:14.359 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:c9569ef7:::115:head 2024-06-11T13:35:14.359 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:61bebe50:::116:head 2024-06-11T13:35:14.359 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:fe93412b:::117:head 2024-06-11T13:35:14.359 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:d3d38bee:::118:head 2024-06-11T13:35:14.360 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:3100ba0c:::119:head 2024-06-11T13:35:14.360 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:d0560ada:::120:head 2024-06-11T13:35:14.360 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:f0ea8b35:::121:head 2024-06-11T13:35:14.360 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:766f231a:::122:head 2024-06-11T13:35:14.360 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:a07a2582:::123:head 2024-06-11T13:35:14.360 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:bd7c6b3a:::124:head 2024-06-11T13:35:14.360 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:fb2ddaff:::125:head 2024-06-11T13:35:14.360 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:4408e1fe:::126:head 2024-06-11T13:35:14.360 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:ee1df7a7:::127:head 2024-06-11T13:35:14.360 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:c3002909:::128:head 2024-06-11T13:35:14.360 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:4f48ffa9:::129:head 2024-06-11T13:35:14.360 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:edf38733:::130:head 2024-06-11T13:35:14.360 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:c08425c0:::131:head 2024-06-11T13:35:14.360 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:5f902d98:::132:head 2024-06-11T13:35:14.360 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:41ea2c93:::133:head 2024-06-11T13:35:14.361 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:813cee13:::134:head 2024-06-11T13:35:14.361 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:0131818d:::135:head 2024-06-11T13:35:14.361 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:26ba5a85:::136:head 2024-06-11T13:35:14.361 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:381b8a5a:::137:head 2024-06-11T13:35:14.361 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:28797e47:::138:head 2024-06-11T13:35:14.361 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:bfca7f22:::139:head 2024-06-11T13:35:14.361 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:36807075:::140:head 2024-06-11T13:35:14.361 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:80b03975:::141:head 2024-06-11T13:35:14.361 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:5c15709b:::142:head 2024-06-11T13:35:14.361 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:f39ea15e:::143:head 2024-06-11T13:35:14.361 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:ea992956:::144:head 2024-06-11T13:35:14.361 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:48887b1c:::145:head 2024-06-11T13:35:14.361 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:9f24a9dd:::146:head 2024-06-11T13:35:14.361 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:987f100b:::147:head 2024-06-11T13:35:14.361 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:d2dd3581:::148:head 2024-06-11T13:35:14.362 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:7fed1808:::149:head 2024-06-11T13:35:14.362 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:c80b70e9:::150:head 2024-06-11T13:35:14.362 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:85ed90f9:::151:head 2024-06-11T13:35:14.362 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:36428b24:::152:head 2024-06-11T13:35:14.362 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:d044c34a:::153:head 2024-06-11T13:35:14.362 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:7c18bf58:::154:head 2024-06-11T13:35:14.362 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:d1c21232:::155:head 2024-06-11T13:35:14.362 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:a7a3c575:::156:head 2024-06-11T13:35:14.362 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:87da0633:::157:head 2024-06-11T13:35:14.362 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:d5ac3822:::158:head 2024-06-11T13:35:14.362 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:3f20522d:::159:head 2024-06-11T13:35:14.363 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:6ca26563:::160:head 2024-06-11T13:35:14.363 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:532ce135:::161:head 2024-06-11T13:35:14.363 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:c78863e6:::162:head 2024-06-11T13:35:15.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:14 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-06-11T13:35:15.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:14 smithi057 ceph-mon[29805]: pgmap v441: 324 pgs: 324 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:35:15.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:14 smithi057 ceph-mon[29805]: osdmap e315: 8 total, 8 up, 8 in 2024-06-11T13:35:15.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:14 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-06-11T13:35:15.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:14 smithi057 ceph-mon[29805]: from='client.36359 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi057-77362-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi057-77362-58"}]': finished 2024-06-11T13:35:15.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:14 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-06-11T13:35:15.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:14 smithi057 ceph-mon[29805]: osdmap e316: 8 total, 8 up, 8 in 2024-06-11T13:35:15.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:14 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi057-77599-43","var": "pg_num","format": "json"}]: dispatch 2024-06-11T13:35:15.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:14 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:35:15.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:14 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:35:15.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:14 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-43"}]: dispatch 2024-06-11T13:35:15.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:14 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-43"}]: dispatch 2024-06-11T13:35:15.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:14 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-06-11T13:35:15.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:14 smithi057 ceph-mon[40588]: pgmap v441: 324 pgs: 324 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:35:15.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:14 smithi057 ceph-mon[40588]: osdmap e315: 8 total, 8 up, 8 in 2024-06-11T13:35:15.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:14 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-06-11T13:35:15.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:14 smithi057 ceph-mon[40588]: from='client.36359 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi057-77362-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi057-77362-58"}]': finished 2024-06-11T13:35:15.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:14 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-06-11T13:35:15.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:14 smithi057 ceph-mon[40588]: osdmap e316: 8 total, 8 up, 8 in 2024-06-11T13:35:15.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:14 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi057-77599-43","var": "pg_num","format": "json"}]: dispatch 2024-06-11T13:35:15.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:14 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:35:15.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:14 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:35:15.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:14 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-43"}]: dispatch 2024-06-11T13:35:15.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:14 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-43"}]: dispatch 2024-06-11T13:35:15.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:14 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-06-11T13:35:15.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:14 smithi107 ceph-mon[33178]: pgmap v441: 324 pgs: 324 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:35:15.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:14 smithi107 ceph-mon[33178]: osdmap e315: 8 total, 8 up, 8 in 2024-06-11T13:35:15.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:14 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-06-11T13:35:15.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:14 smithi107 ceph-mon[33178]: from='client.36359 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi057-77362-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi057-77362-58"}]': finished 2024-06-11T13:35:15.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:14 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-06-11T13:35:15.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:14 smithi107 ceph-mon[33178]: osdmap e316: 8 total, 8 up, 8 in 2024-06-11T13:35:15.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:14 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi057-77599-43","var": "pg_num","format": "json"}]: dispatch 2024-06-11T13:35:15.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:14 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:35:15.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:14 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:35:15.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:14 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-43"}]: dispatch 2024-06-11T13:35:15.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:14 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-43"}]: dispatch 2024-06-11T13:35:16.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:16 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-43"}]': finished 2024-06-11T13:35:16.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:16 smithi057 ceph-mon[40588]: osdmap e317: 8 total, 8 up, 8 in 2024-06-11T13:35:16.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:16 smithi057 ceph-mon[40588]: pgmap v444: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:35:16.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:16 smithi057 ceph-mon[40588]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:35:16.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:16 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-43"}]': finished 2024-06-11T13:35:16.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:16 smithi057 ceph-mon[29805]: osdmap e317: 8 total, 8 up, 8 in 2024-06-11T13:35:16.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:16 smithi057 ceph-mon[29805]: pgmap v444: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:35:16.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:16 smithi057 ceph-mon[29805]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:35:16.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:16 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-43"}]': finished 2024-06-11T13:35:16.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:16 smithi107 ceph-mon[33178]: osdmap e317: 8 total, 8 up, 8 in 2024-06-11T13:35:16.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:16 smithi107 ceph-mon[33178]: pgmap v444: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:35:16.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:16 smithi107 ceph-mon[33178]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:35:17.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:17 smithi057 ceph-mon[29805]: osdmap e318: 8 total, 8 up, 8 in 2024-06-11T13:35:17.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:17 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1097988569' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi057-77362-58"}]: dispatch 2024-06-11T13:35:17.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:17 smithi057 ceph-mon[29805]: from='client.36359 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi057-77362-58"}]: dispatch 2024-06-11T13:35:17.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:17 smithi057 ceph-mon[40588]: osdmap e318: 8 total, 8 up, 8 in 2024-06-11T13:35:17.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:17 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1097988569' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi057-77362-58"}]: dispatch 2024-06-11T13:35:17.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:17 smithi057 ceph-mon[40588]: from='client.36359 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi057-77362-58"}]: dispatch 2024-06-11T13:35:17.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:17 smithi107 ceph-mon[33178]: osdmap e318: 8 total, 8 up, 8 in 2024-06-11T13:35:17.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:17 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1097988569' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi057-77362-58"}]: dispatch 2024-06-11T13:35:17.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:17 smithi107 ceph-mon[33178]: from='client.36359 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi057-77362-58"}]: dispatch 2024-06-11T13:35:18.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:18 smithi057 ceph-mon[29805]: from='client.36359 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi057-77362-58"}]': finished 2024-06-11T13:35:18.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:18 smithi057 ceph-mon[29805]: osdmap e319: 8 total, 8 up, 8 in 2024-06-11T13:35:18.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:18 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:35:18.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:18 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:35:18.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:18 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1097988569' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi057-77362-58"}]: dispatch 2024-06-11T13:35:18.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:18 smithi057 ceph-mon[29805]: from='client.36359 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi057-77362-58"}]: dispatch 2024-06-11T13:35:18.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:18 smithi057 ceph-mon[29805]: pgmap v447: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:35:18.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:18 smithi057 ceph-mon[40588]: from='client.36359 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi057-77362-58"}]': finished 2024-06-11T13:35:18.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:18 smithi057 ceph-mon[40588]: osdmap e319: 8 total, 8 up, 8 in 2024-06-11T13:35:18.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:18 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:35:18.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:18 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:35:18.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:18 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1097988569' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi057-77362-58"}]: dispatch 2024-06-11T13:35:18.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:18 smithi057 ceph-mon[40588]: from='client.36359 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi057-77362-58"}]: dispatch 2024-06-11T13:35:18.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:18 smithi057 ceph-mon[40588]: pgmap v447: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:35:18.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:18 smithi107 ceph-mon[33178]: from='client.36359 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi057-77362-58"}]': finished 2024-06-11T13:35:18.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:18 smithi107 ceph-mon[33178]: osdmap e319: 8 total, 8 up, 8 in 2024-06-11T13:35:18.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:18 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:35:18.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:18 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:35:18.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:18 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1097988569' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi057-77362-58"}]: dispatch 2024-06-11T13:35:18.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:18 smithi107 ceph-mon[33178]: from='client.36359 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi057-77362-58"}]: dispatch 2024-06-11T13:35:18.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:18 smithi107 ceph-mon[33178]: pgmap v447: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:35:19.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:19 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:35:19.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:19 smithi057 ceph-mon[40588]: from='client.36359 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi057-77362-58"}]': finished 2024-06-11T13:35:19.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:19 smithi057 ceph-mon[40588]: osdmap e320: 8 total, 8 up, 8 in 2024-06-11T13:35:19.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:19 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:35:19.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:19 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:35:19.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:19 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2641809736' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi057-77362-59"}]: dispatch 2024-06-11T13:35:19.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:19 smithi057 ceph-mon[40588]: from='client.36365 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi057-77362-59"}]: dispatch 2024-06-11T13:35:19.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:19 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2641809736' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi057-77362-59"}]: dispatch 2024-06-11T13:35:19.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:19 smithi057 ceph-mon[40588]: from='client.36365 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi057-77362-59"}]: dispatch 2024-06-11T13:35:19.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:19 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2641809736' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi057-77362-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:35:19.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:19 smithi057 ceph-mon[40588]: from='client.36365 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi057-77362-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:35:19.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:19 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:35:19.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:19 smithi057 ceph-mon[29805]: from='client.36359 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi057-77362-58"}]': finished 2024-06-11T13:35:19.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:19 smithi057 ceph-mon[29805]: osdmap e320: 8 total, 8 up, 8 in 2024-06-11T13:35:19.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:19 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:35:19.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:19 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:35:19.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:19 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2641809736' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi057-77362-59"}]: dispatch 2024-06-11T13:35:19.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:19 smithi057 ceph-mon[29805]: from='client.36365 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi057-77362-59"}]: dispatch 2024-06-11T13:35:19.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:19 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2641809736' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi057-77362-59"}]: dispatch 2024-06-11T13:35:19.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:19 smithi057 ceph-mon[29805]: from='client.36365 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi057-77362-59"}]: dispatch 2024-06-11T13:35:19.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:19 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2641809736' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi057-77362-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:35:19.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:19 smithi057 ceph-mon[29805]: from='client.36365 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi057-77362-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:35:19.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:19 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:35:19.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:19 smithi107 ceph-mon[33178]: from='client.36359 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi057-77362-58"}]': finished 2024-06-11T13:35:19.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:19 smithi107 ceph-mon[33178]: osdmap e320: 8 total, 8 up, 8 in 2024-06-11T13:35:19.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:19 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:35:19.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:19 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:35:19.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:19 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2641809736' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi057-77362-59"}]: dispatch 2024-06-11T13:35:19.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:19 smithi107 ceph-mon[33178]: from='client.36365 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi057-77362-59"}]: dispatch 2024-06-11T13:35:19.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:19 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2641809736' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi057-77362-59"}]: dispatch 2024-06-11T13:35:19.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:19 smithi107 ceph-mon[33178]: from='client.36365 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi057-77362-59"}]: dispatch 2024-06-11T13:35:19.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:19 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2641809736' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi057-77362-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:35:19.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:19 smithi107 ceph-mon[33178]: from='client.36365 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi057-77362-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:35:21.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:20 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-45", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:35:21.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:20 smithi057 ceph-mon[40588]: from='client.36365 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi057-77362-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:35:21.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:20 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-45","var": "hit_set_count","val": "3"}]: dispatch 2024-06-11T13:35:21.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:20 smithi057 ceph-mon[40588]: osdmap e321: 8 total, 8 up, 8 in 2024-06-11T13:35:21.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:20 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-45","var": "hit_set_count","val": "3"}]: dispatch 2024-06-11T13:35:21.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:20 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2641809736' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi057-77362-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi057-77362-59"}]: dispatch 2024-06-11T13:35:21.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:20 smithi057 ceph-mon[40588]: from='client.36365 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi057-77362-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi057-77362-59"}]: dispatch 2024-06-11T13:35:21.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:20 smithi057 ceph-mon[40588]: pgmap v450: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:35:21.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:20 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-45","var": "hit_set_count","val": "3"}]': finished 2024-06-11T13:35:21.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:20 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-45", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:35:21.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:20 smithi057 ceph-mon[29805]: from='client.36365 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi057-77362-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:35:21.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:20 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-45","var": "hit_set_count","val": "3"}]: dispatch 2024-06-11T13:35:21.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:20 smithi057 ceph-mon[29805]: osdmap e321: 8 total, 8 up, 8 in 2024-06-11T13:35:21.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:20 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-45","var": "hit_set_count","val": "3"}]: dispatch 2024-06-11T13:35:21.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:20 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2641809736' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi057-77362-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi057-77362-59"}]: dispatch 2024-06-11T13:35:21.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:20 smithi057 ceph-mon[29805]: from='client.36365 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi057-77362-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi057-77362-59"}]: dispatch 2024-06-11T13:35:21.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:20 smithi057 ceph-mon[29805]: pgmap v450: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:35:21.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:20 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-45","var": "hit_set_count","val": "3"}]': finished 2024-06-11T13:35:21.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:20 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-45", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:35:21.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:20 smithi107 ceph-mon[33178]: from='client.36365 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi057-77362-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:35:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:20 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-45","var": "hit_set_count","val": "3"}]: dispatch 2024-06-11T13:35:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:20 smithi107 ceph-mon[33178]: osdmap e321: 8 total, 8 up, 8 in 2024-06-11T13:35:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:20 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-45","var": "hit_set_count","val": "3"}]: dispatch 2024-06-11T13:35:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:20 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2641809736' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi057-77362-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi057-77362-59"}]: dispatch 2024-06-11T13:35:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:20 smithi107 ceph-mon[33178]: from='client.36365 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi057-77362-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi057-77362-59"}]: dispatch 2024-06-11T13:35:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:20 smithi107 ceph-mon[33178]: pgmap v450: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:35:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:20 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-45","var": "hit_set_count","val": "3"}]': finished 2024-06-11T13:35:21.628 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:35:21 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:35:22.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:21 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-45","var": "hit_set_period","val": "3"}]: dispatch 2024-06-11T13:35:22.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:21 smithi057 ceph-mon[40588]: osdmap e322: 8 total, 8 up, 8 in 2024-06-11T13:35:22.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:21 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-45","var": "hit_set_period","val": "3"}]: dispatch 2024-06-11T13:35:22.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:21 smithi057 ceph-mon[40588]: from='client.36365 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi057-77362-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi057-77362-59"}]': finished 2024-06-11T13:35:22.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:21 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-45","var": "hit_set_period","val": "3"}]': finished 2024-06-11T13:35:22.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:21 smithi057 ceph-mon[40588]: osdmap e323: 8 total, 8 up, 8 in 2024-06-11T13:35:22.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:21 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-11T13:35:22.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:21 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-11T13:35:22.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:21 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:35:22.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:21 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-45","var": "hit_set_period","val": "3"}]: dispatch 2024-06-11T13:35:22.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:21 smithi057 ceph-mon[29805]: osdmap e322: 8 total, 8 up, 8 in 2024-06-11T13:35:22.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:21 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-45","var": "hit_set_period","val": "3"}]: dispatch 2024-06-11T13:35:22.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:21 smithi057 ceph-mon[29805]: from='client.36365 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi057-77362-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi057-77362-59"}]': finished 2024-06-11T13:35:22.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:21 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-45","var": "hit_set_period","val": "3"}]': finished 2024-06-11T13:35:22.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:21 smithi057 ceph-mon[29805]: osdmap e323: 8 total, 8 up, 8 in 2024-06-11T13:35:22.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:21 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-11T13:35:22.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:21 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-11T13:35:22.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:21 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:35:22.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:21 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-45","var": "hit_set_period","val": "3"}]: dispatch 2024-06-11T13:35:22.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:21 smithi107 ceph-mon[33178]: osdmap e322: 8 total, 8 up, 8 in 2024-06-11T13:35:22.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:21 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-45","var": "hit_set_period","val": "3"}]: dispatch 2024-06-11T13:35:22.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:21 smithi107 ceph-mon[33178]: from='client.36365 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi057-77362-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi057-77362-59"}]': finished 2024-06-11T13:35:22.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:21 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-45","var": "hit_set_period","val": "3"}]': finished 2024-06-11T13:35:22.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:21 smithi107 ceph-mon[33178]: osdmap e323: 8 total, 8 up, 8 in 2024-06-11T13:35:22.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:21 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-11T13:35:22.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:21 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-11T13:35:22.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:21 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:35:23.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:22 smithi057 ceph-mon[29805]: pgmap v453: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:35:23.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:22 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:35:23.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:22 smithi057 ceph-mon[29805]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:35:23.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:22 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-45","var": "hit_set_type","val": "bloom"}]': finished 2024-06-11T13:35:23.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:22 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-06-11T13:35:23.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:22 smithi057 ceph-mon[29805]: osdmap e324: 8 total, 8 up, 8 in 2024-06-11T13:35:23.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:22 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-06-11T13:35:23.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:22 smithi057 ceph-mon[40588]: pgmap v453: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:35:23.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:22 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:35:23.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:22 smithi057 ceph-mon[40588]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:35:23.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:22 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-45","var": "hit_set_type","val": "bloom"}]': finished 2024-06-11T13:35:23.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:22 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-06-11T13:35:23.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:22 smithi057 ceph-mon[40588]: osdmap e324: 8 total, 8 up, 8 in 2024-06-11T13:35:23.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:22 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-06-11T13:35:23.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:22 smithi107 ceph-mon[33178]: pgmap v453: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:35:23.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:22 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:35:23.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:22 smithi107 ceph-mon[33178]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:35:23.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:22 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-45","var": "hit_set_type","val": "bloom"}]': finished 2024-06-11T13:35:23.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:22 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-06-11T13:35:23.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:22 smithi107 ceph-mon[33178]: osdmap e324: 8 total, 8 up, 8 in 2024-06-11T13:35:23.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:22 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-06-11T13:35:24.034 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:35:23 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:35:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:35:24.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:24 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-06-11T13:35:24.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:24 smithi057 ceph-mon[29805]: osdmap e325: 8 total, 8 up, 8 in 2024-06-11T13:35:24.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:24 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2641809736' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi057-77362-59"}]: dispatch 2024-06-11T13:35:24.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:24 smithi057 ceph-mon[29805]: from='client.36365 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi057-77362-59"}]: dispatch 2024-06-11T13:35:24.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:24 smithi057 ceph-mon[29805]: pgmap v456: 324 pgs: 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:35:24.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:24 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-06-11T13:35:24.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:24 smithi057 ceph-mon[40588]: osdmap e325: 8 total, 8 up, 8 in 2024-06-11T13:35:24.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:24 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2641809736' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi057-77362-59"}]: dispatch 2024-06-11T13:35:24.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:24 smithi057 ceph-mon[40588]: from='client.36365 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi057-77362-59"}]: dispatch 2024-06-11T13:35:24.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:24 smithi057 ceph-mon[40588]: pgmap v456: 324 pgs: 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:35:24.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:24 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-06-11T13:35:24.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:24 smithi107 ceph-mon[33178]: osdmap e325: 8 total, 8 up, 8 in 2024-06-11T13:35:24.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:24 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2641809736' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi057-77362-59"}]: dispatch 2024-06-11T13:35:24.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:24 smithi107 ceph-mon[33178]: from='client.36365 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi057-77362-59"}]: dispatch 2024-06-11T13:35:24.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:24 smithi107 ceph-mon[33178]: pgmap v456: 324 pgs: 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:35:25.224 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:7462ddf6:io.RoundTripAppendPP (2545 ms) 2024-06-11T13:35:25.224 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAio.RacingRemovePP 2024-06-11T13:35:25.224 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAio.RacingRemovePP (3042 ms) 2024-06-11T13:35:25.224 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP 2024-06-11T13:35:25.224 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP (3019 ms) 2024-06-11T13:35:25.224 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP2 2024-06-11T13:35:25.224 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP2 (3384 ms) 2024-06-11T13:35:25.224 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolEIOFlag 2024-06-11T13:35:25.225 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: setting pool EIO 2024-06-11T13:35:25.225 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: max_success 99, min_failed 100 2024-06-11T13:35:25.225 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolEIOFlag (3597 ms) 2024-06-11T13:35:25.225 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiReads 2024-06-11T13:35:25.225 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiReads (3452 ms) 2024-06-11T13:35:25.225 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio (154605 ms total) 2024-06-11T13:35:25.225 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: 2024-06-11T13:35:25.225 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP 2024-06-11T13:35:25.225 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.ReadIntoBufferlist 2024-06-11T13:35:25.225 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAioPP.ReadIntoBufferlist (2565 ms) 2024-06-11T13:35:25.225 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.XattrsRoundTripPP 2024-06-11T13:35:25.225 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAioPP.XattrsRoundTripPP (9078 ms) 2024-06-11T13:35:25.225 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RmXattrPP 2024-06-11T13:35:25.226 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RmXattrPP (15131 ms) 2024-06-11T13:35:25.226 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RemoveTestPP 2024-06-11T13:35:25.226 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RemoveTestPP (2794 ms) 2024-06-11T13:35:25.226 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP (29568 ms total) 2024-06-11T13:35:25.226 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: 2024-06-11T13:35:25.226 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP 2024-06-11T13:35:25.226 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-06-11T13:35:25.226 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosIoPP.XattrListPP (3021 ms) 2024-06-11T13:35:25.226 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP (3021 ms total) 2024-06-11T13:35:25.226 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: 2024-06-11T13:35:25.226 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC 2024-06-11T13:35:25.226 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleWritePP 2024-06-11T13:35:25.226 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleWritePP (14032 ms) 2024-06-11T13:35:25.226 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.WaitForSafePP 2024-06-11T13:35:25.227 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAioEC.WaitForSafePP (7056 ms) 2024-06-11T13:35:25.227 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP 2024-06-11T13:35:25.227 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP (6948 ms) 2024-06-11T13:35:25.227 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP2 2024-06-11T13:35:25.227 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP2 (7446 ms) 2024-06-11T13:35:25.227 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP3 2024-06-11T13:35:25.227 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP3 (3035 ms) 2024-06-11T13:35:25.227 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripSparseReadPP 2024-06-11T13:35:25.227 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripSparseReadPP (7066 ms) 2024-06-11T13:35:25.227 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripAppendPP 2024-06-11T13:35:25.227 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripAppendPP (7055 ms) 2024-06-11T13:35:25.227 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsCompletePP 2024-06-11T13:35:25.227 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsCompletePP (6503 ms) 2024-06-11T13:35:25.227 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsSafePP 2024-06-11T13:35:25.228 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsSafePP (7095 ms) 2024-06-11T13:35:25.228 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ReturnValuePP 2024-06-11T13:35:25.228 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ReturnValuePP (7056 ms) 2024-06-11T13:35:25.228 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushPP 2024-06-11T13:35:25.228 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushPP (7037 ms) 2024-06-11T13:35:25.228 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushAsyncPP 2024-06-11T13:35:25.228 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushAsyncPP (7085 ms) 2024-06-11T13:35:25.228 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP 2024-06-11T13:35:25.228 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP (7053 ms) 2024-06-11T13:35:25.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:25 smithi057 ceph-mon[40588]: from='client.36365 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi057-77362-59"}]': finished 2024-06-11T13:35:25.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:25 smithi057 ceph-mon[40588]: osdmap e326: 8 total, 8 up, 8 in 2024-06-11T13:35:25.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:25 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2641809736' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi057-77362-59"}]: dispatch 2024-06-11T13:35:25.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:25 smithi057 ceph-mon[40588]: from='client.36365 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi057-77362-59"}]: dispatch 2024-06-11T13:35:25.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:25 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:35:25.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:25 smithi057 ceph-mon[29805]: from='client.36365 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi057-77362-59"}]': finished 2024-06-11T13:35:25.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:25 smithi057 ceph-mon[29805]: osdmap e326: 8 total, 8 up, 8 in 2024-06-11T13:35:25.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:25 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2641809736' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi057-77362-59"}]: dispatch 2024-06-11T13:35:25.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:25 smithi057 ceph-mon[29805]: from='client.36365 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi057-77362-59"}]: dispatch 2024-06-11T13:35:25.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:25 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:35:25.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:25 smithi107 ceph-mon[33178]: from='client.36365 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi057-77362-59"}]': finished 2024-06-11T13:35:25.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:25 smithi107 ceph-mon[33178]: osdmap e326: 8 total, 8 up, 8 in 2024-06-11T13:35:25.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:25 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2641809736' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi057-77362-59"}]: dispatch 2024-06-11T13:35:25.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:25 smithi107 ceph-mon[33178]: from='client.36365 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi057-77362-59"}]: dispatch 2024-06-11T13:35:25.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:25 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:35:26.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:26 smithi057 ceph-mon[29805]: from='client.36365 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi057-77362-59"}]': finished 2024-06-11T13:35:26.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:26 smithi057 ceph-mon[29805]: osdmap e327: 8 total, 8 up, 8 in 2024-06-11T13:35:26.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:26 smithi057 ceph-mon[29805]: pgmap v459: 324 pgs: 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:35:26.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:26 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:35:26.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:26 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:35:26.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:26 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:35:26.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:26 smithi057 ceph-mon[40588]: from='client.36365 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi057-77362-59"}]': finished 2024-06-11T13:35:26.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:26 smithi057 ceph-mon[40588]: osdmap e327: 8 total, 8 up, 8 in 2024-06-11T13:35:26.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:26 smithi057 ceph-mon[40588]: pgmap v459: 324 pgs: 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:35:26.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:26 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:35:26.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:26 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:35:26.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:26 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:35:26.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:26 smithi107 ceph-mon[33178]: from='client.36365 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi057-77362-59"}]': finished 2024-06-11T13:35:26.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:26 smithi107 ceph-mon[33178]: osdmap e327: 8 total, 8 up, 8 in 2024-06-11T13:35:26.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:26 smithi107 ceph-mon[33178]: pgmap v459: 324 pgs: 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:35:26.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:26 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:35:26.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:26 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:35:26.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:26 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:35:27.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:27 smithi057 ceph-mon[40588]: osdmap e328: 8 total, 8 up, 8 in 2024-06-11T13:35:27.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:27 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/4256519934' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi057-77362-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:35:27.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:27 smithi057 ceph-mon[29805]: osdmap e328: 8 total, 8 up, 8 in 2024-06-11T13:35:27.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:27 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/4256519934' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi057-77362-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:35:27.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:27 smithi107 ceph-mon[33178]: osdmap e328: 8 total, 8 up, 8 in 2024-06-11T13:35:27.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:27 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/4256519934' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi057-77362-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:35:28.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:28 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/4256519934' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi057-77362-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:35:28.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:28 smithi057 ceph-mon[40588]: osdmap e329: 8 total, 8 up, 8 in 2024-06-11T13:35:28.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:28 smithi057 ceph-mon[40588]: pgmap v462: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-06-11T13:35:28.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:28 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/4256519934' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi057-77362-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:35:28.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:28 smithi057 ceph-mon[29805]: osdmap e329: 8 total, 8 up, 8 in 2024-06-11T13:35:28.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:28 smithi057 ceph-mon[29805]: pgmap v462: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-06-11T13:35:28.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:28 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/4256519934' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi057-77362-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:35:28.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:28 smithi107 ceph-mon[33178]: osdmap e329: 8 total, 8 up, 8 in 2024-06-11T13:35:28.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:28 smithi107 ceph-mon[33178]: pgmap v462: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-06-11T13:35:29.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:29 smithi057 ceph-mon[29805]: osdmap e330: 8 total, 8 up, 8 in 2024-06-11T13:35:29.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:29 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1729202216' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi057-77362-61"}]: dispatch 2024-06-11T13:35:29.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:29 smithi057 ceph-mon[29805]: from='client.36377 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi057-77362-61"}]: dispatch 2024-06-11T13:35:29.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:29 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1729202216' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi057-77362-61"}]: dispatch 2024-06-11T13:35:29.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:29 smithi057 ceph-mon[29805]: from='client.36377 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi057-77362-61"}]: dispatch 2024-06-11T13:35:29.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:29 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1729202216' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi057-77362-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:35:29.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:29 smithi057 ceph-mon[29805]: from='client.36377 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi057-77362-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:35:29.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:29 smithi057 ceph-mon[40588]: osdmap e330: 8 total, 8 up, 8 in 2024-06-11T13:35:29.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:29 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1729202216' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi057-77362-61"}]: dispatch 2024-06-11T13:35:29.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:29 smithi057 ceph-mon[40588]: from='client.36377 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi057-77362-61"}]: dispatch 2024-06-11T13:35:29.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:29 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1729202216' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi057-77362-61"}]: dispatch 2024-06-11T13:35:29.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:29 smithi057 ceph-mon[40588]: from='client.36377 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi057-77362-61"}]: dispatch 2024-06-11T13:35:29.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:29 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1729202216' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi057-77362-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:35:29.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:29 smithi057 ceph-mon[40588]: from='client.36377 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi057-77362-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:35:29.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:29 smithi107 ceph-mon[33178]: osdmap e330: 8 total, 8 up, 8 in 2024-06-11T13:35:29.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:29 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1729202216' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi057-77362-61"}]: dispatch 2024-06-11T13:35:29.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:29 smithi107 ceph-mon[33178]: from='client.36377 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi057-77362-61"}]: dispatch 2024-06-11T13:35:29.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:29 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1729202216' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi057-77362-61"}]: dispatch 2024-06-11T13:35:29.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:29 smithi107 ceph-mon[33178]: from='client.36377 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi057-77362-61"}]: dispatch 2024-06-11T13:35:29.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:29 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1729202216' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi057-77362-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:35:29.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:29 smithi107 ceph-mon[33178]: from='client.36377 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi057-77362-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:35:31.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:30 smithi057 ceph-mon[29805]: from='client.36377 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi057-77362-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:35:31.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:30 smithi057 ceph-mon[29805]: pgmap v464: 324 pgs: 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 753 B/s wr, 1 op/s 2024-06-11T13:35:31.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:30 smithi057 ceph-mon[29805]: osdmap e331: 8 total, 8 up, 8 in 2024-06-11T13:35:31.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:30 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1729202216' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi057-77362-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi057-77362-61"}]: dispatch 2024-06-11T13:35:31.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:30 smithi057 ceph-mon[29805]: from='client.36377 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi057-77362-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi057-77362-61"}]: dispatch 2024-06-11T13:35:31.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:30 smithi057 ceph-mon[40588]: from='client.36377 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi057-77362-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:35:31.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:30 smithi057 ceph-mon[40588]: pgmap v464: 324 pgs: 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 753 B/s wr, 1 op/s 2024-06-11T13:35:31.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:30 smithi057 ceph-mon[40588]: osdmap e331: 8 total, 8 up, 8 in 2024-06-11T13:35:31.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:30 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1729202216' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi057-77362-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi057-77362-61"}]: dispatch 2024-06-11T13:35:31.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:30 smithi057 ceph-mon[40588]: from='client.36377 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi057-77362-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi057-77362-61"}]: dispatch 2024-06-11T13:35:31.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:30 smithi107 ceph-mon[33178]: from='client.36377 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi057-77362-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:35:31.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:30 smithi107 ceph-mon[33178]: pgmap v464: 324 pgs: 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 753 B/s wr, 1 op/s 2024-06-11T13:35:31.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:30 smithi107 ceph-mon[33178]: osdmap e331: 8 total, 8 up, 8 in 2024-06-11T13:35:31.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:30 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1729202216' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi057-77362-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi057-77362-61"}]: dispatch 2024-06-11T13:35:31.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:30 smithi107 ceph-mon[33178]: from='client.36377 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi057-77362-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi057-77362-61"}]: dispatch 2024-06-11T13:35:31.757 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:35:31 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:35:32.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:31 smithi057 ceph-mon[40588]: osdmap e332: 8 total, 8 up, 8 in 2024-06-11T13:35:32.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:31 smithi057 ceph-mon[40588]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:35:32.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:31 smithi057 ceph-mon[40588]: from='client.36377 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi057-77362-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi057-77362-61"}]': finished 2024-06-11T13:35:32.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:31 smithi057 ceph-mon[40588]: osdmap e333: 8 total, 8 up, 8 in 2024-06-11T13:35:32.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:31 smithi057 ceph-mon[29805]: osdmap e332: 8 total, 8 up, 8 in 2024-06-11T13:35:32.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:31 smithi057 ceph-mon[29805]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:35:32.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:31 smithi057 ceph-mon[29805]: from='client.36377 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi057-77362-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi057-77362-61"}]': finished 2024-06-11T13:35:32.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:31 smithi057 ceph-mon[29805]: osdmap e333: 8 total, 8 up, 8 in 2024-06-11T13:35:32.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:31 smithi107 ceph-mon[33178]: osdmap e332: 8 total, 8 up, 8 in 2024-06-11T13:35:32.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:31 smithi107 ceph-mon[33178]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:35:32.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:31 smithi107 ceph-mon[33178]: from='client.36377 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi057-77362-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi057-77362-61"}]': finished 2024-06-11T13:35:32.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:31 smithi107 ceph-mon[33178]: osdmap e333: 8 total, 8 up, 8 in 2024-06-11T13:35:33.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:32 smithi057 ceph-mon[40588]: pgmap v467: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-06-11T13:35:33.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:32 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:35:33.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:32 smithi057 ceph-mon[40588]: osdmap e334: 8 total, 8 up, 8 in 2024-06-11T13:35:33.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:32 smithi057 ceph-mon[29805]: pgmap v467: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-06-11T13:35:33.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:32 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:35:33.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:32 smithi057 ceph-mon[29805]: osdmap e334: 8 total, 8 up, 8 in 2024-06-11T13:35:33.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:32 smithi107 ceph-mon[33178]: pgmap v467: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-06-11T13:35:33.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:32 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:35:33.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:32 smithi107 ceph-mon[33178]: osdmap e334: 8 total, 8 up, 8 in 2024-06-11T13:35:34.034 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:35:33 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:35:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:35:34.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:34 smithi057 ceph-mon[40588]: pgmap v470: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-06-11T13:35:34.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:34 smithi057 ceph-mon[40588]: osdmap e335: 8 total, 8 up, 8 in 2024-06-11T13:35:34.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:34 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1729202216' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi057-77362-61"}]: dispatch 2024-06-11T13:35:34.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:34 smithi057 ceph-mon[40588]: from='client.36377 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi057-77362-61"}]: dispatch 2024-06-11T13:35:34.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:34 smithi057 ceph-mon[29805]: pgmap v470: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-06-11T13:35:34.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:34 smithi057 ceph-mon[29805]: osdmap e335: 8 total, 8 up, 8 in 2024-06-11T13:35:34.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:34 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1729202216' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi057-77362-61"}]: dispatch 2024-06-11T13:35:34.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:34 smithi057 ceph-mon[29805]: from='client.36377 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi057-77362-61"}]: dispatch 2024-06-11T13:35:34.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:34 smithi107 ceph-mon[33178]: pgmap v470: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-06-11T13:35:34.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:34 smithi107 ceph-mon[33178]: osdmap e335: 8 total, 8 up, 8 in 2024-06-11T13:35:34.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:34 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1729202216' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi057-77362-61"}]: dispatch 2024-06-11T13:35:34.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:34 smithi107 ceph-mon[33178]: from='client.36377 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi057-77362-61"}]: dispatch 2024-06-11T13:35:35.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:35 smithi057 ceph-mon[29805]: from='client.36377 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi057-77362-61"}]': finished 2024-06-11T13:35:35.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:35 smithi057 ceph-mon[29805]: osdmap e336: 8 total, 8 up, 8 in 2024-06-11T13:35:35.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:35 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1729202216' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi057-77362-61"}]: dispatch 2024-06-11T13:35:35.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:35 smithi057 ceph-mon[29805]: from='client.36377 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi057-77362-61"}]: dispatch 2024-06-11T13:35:35.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:35 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:35:35.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:35 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:35:35.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:35 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-45"}]: dispatch 2024-06-11T13:35:35.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:35 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-45"}]: dispatch 2024-06-11T13:35:35.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:35 smithi057 ceph-mon[40588]: from='client.36377 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi057-77362-61"}]': finished 2024-06-11T13:35:35.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:35 smithi057 ceph-mon[40588]: osdmap e336: 8 total, 8 up, 8 in 2024-06-11T13:35:35.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:35 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1729202216' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi057-77362-61"}]: dispatch 2024-06-11T13:35:35.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:35 smithi057 ceph-mon[40588]: from='client.36377 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi057-77362-61"}]: dispatch 2024-06-11T13:35:35.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:35 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:35:35.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:35 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:35:35.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:35 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-45"}]: dispatch 2024-06-11T13:35:35.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:35 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-45"}]: dispatch 2024-06-11T13:35:35.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:35 smithi107 ceph-mon[33178]: from='client.36377 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi057-77362-61"}]': finished 2024-06-11T13:35:35.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:35 smithi107 ceph-mon[33178]: osdmap e336: 8 total, 8 up, 8 in 2024-06-11T13:35:35.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:35 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1729202216' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi057-77362-61"}]: dispatch 2024-06-11T13:35:35.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:35 smithi107 ceph-mon[33178]: from='client.36377 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi057-77362-61"}]: dispatch 2024-06-11T13:35:35.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:35 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:35:35.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:35 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:35:35.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:35 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-45"}]: dispatch 2024-06-11T13:35:35.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:35 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-45"}]: dispatch 2024-06-11T13:35:37.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:36 smithi057 ceph-mon[29805]: pgmap v473: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:35:37.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:36 smithi057 ceph-mon[29805]: from='client.36377 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi057-77362-61"}]': finished 2024-06-11T13:35:37.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:36 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-45"}]': finished 2024-06-11T13:35:37.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:36 smithi057 ceph-mon[29805]: osdmap e337: 8 total, 8 up, 8 in 2024-06-11T13:35:37.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:36 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/251888670' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi057-77362-62"}]: dispatch 2024-06-11T13:35:37.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:36 smithi057 ceph-mon[29805]: from='client.36383 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi057-77362-62"}]: dispatch 2024-06-11T13:35:37.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:36 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/251888670' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi057-77362-62"}]: dispatch 2024-06-11T13:35:37.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:36 smithi057 ceph-mon[29805]: from='client.36383 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi057-77362-62"}]: dispatch 2024-06-11T13:35:37.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:36 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/251888670' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi057-77362-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:35:37.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:36 smithi057 ceph-mon[29805]: from='client.36383 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi057-77362-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:35:37.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:36 smithi057 ceph-mon[40588]: pgmap v473: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:35:37.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:36 smithi057 ceph-mon[40588]: from='client.36377 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi057-77362-61"}]': finished 2024-06-11T13:35:37.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:36 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-45"}]': finished 2024-06-11T13:35:37.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:36 smithi057 ceph-mon[40588]: osdmap e337: 8 total, 8 up, 8 in 2024-06-11T13:35:37.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:36 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/251888670' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi057-77362-62"}]: dispatch 2024-06-11T13:35:37.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:36 smithi057 ceph-mon[40588]: from='client.36383 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi057-77362-62"}]: dispatch 2024-06-11T13:35:37.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:36 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/251888670' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi057-77362-62"}]: dispatch 2024-06-11T13:35:37.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:36 smithi057 ceph-mon[40588]: from='client.36383 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi057-77362-62"}]: dispatch 2024-06-11T13:35:37.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:36 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/251888670' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi057-77362-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:35:37.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:36 smithi057 ceph-mon[40588]: from='client.36383 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi057-77362-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:35:37.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:36 smithi107 ceph-mon[33178]: pgmap v473: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:35:37.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:36 smithi107 ceph-mon[33178]: from='client.36377 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi057-77362-61"}]': finished 2024-06-11T13:35:37.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:36 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-45"}]': finished 2024-06-11T13:35:37.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:36 smithi107 ceph-mon[33178]: osdmap e337: 8 total, 8 up, 8 in 2024-06-11T13:35:37.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:36 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/251888670' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi057-77362-62"}]: dispatch 2024-06-11T13:35:37.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:36 smithi107 ceph-mon[33178]: from='client.36383 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi057-77362-62"}]: dispatch 2024-06-11T13:35:37.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:36 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/251888670' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi057-77362-62"}]: dispatch 2024-06-11T13:35:37.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:36 smithi107 ceph-mon[33178]: from='client.36383 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi057-77362-62"}]: dispatch 2024-06-11T13:35:37.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:36 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/251888670' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi057-77362-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:35:37.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:36 smithi107 ceph-mon[33178]: from='client.36383 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi057-77362-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:35:38.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:37 smithi057 ceph-mon[29805]: from='client.36383 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi057-77362-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:35:38.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:37 smithi057 ceph-mon[29805]: osdmap e338: 8 total, 8 up, 8 in 2024-06-11T13:35:38.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:37 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/251888670' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi057-77362-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi057-77362-62"}]: dispatch 2024-06-11T13:35:38.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:37 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:35:38.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:37 smithi057 ceph-mon[29805]: from='client.36383 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi057-77362-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi057-77362-62"}]: dispatch 2024-06-11T13:35:38.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:37 smithi057 ceph-mon[29805]: pgmap v476: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:35:38.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:37 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:35:38.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:37 smithi057 ceph-mon[29805]: osdmap e339: 8 total, 8 up, 8 in 2024-06-11T13:35:38.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:37 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:35:38.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:37 smithi057 ceph-mon[40588]: from='client.36383 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi057-77362-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:35:38.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:37 smithi057 ceph-mon[40588]: osdmap e338: 8 total, 8 up, 8 in 2024-06-11T13:35:38.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:37 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/251888670' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi057-77362-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi057-77362-62"}]: dispatch 2024-06-11T13:35:38.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:37 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:35:38.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:37 smithi057 ceph-mon[40588]: from='client.36383 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi057-77362-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi057-77362-62"}]: dispatch 2024-06-11T13:35:38.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:37 smithi057 ceph-mon[40588]: pgmap v476: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:35:38.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:37 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:35:38.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:37 smithi057 ceph-mon[40588]: osdmap e339: 8 total, 8 up, 8 in 2024-06-11T13:35:38.288 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:37 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:35:38.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:37 smithi107 ceph-mon[33178]: from='client.36383 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi057-77362-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:35:38.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:37 smithi107 ceph-mon[33178]: osdmap e338: 8 total, 8 up, 8 in 2024-06-11T13:35:38.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:37 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/251888670' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi057-77362-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi057-77362-62"}]: dispatch 2024-06-11T13:35:38.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:37 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:35:38.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:37 smithi107 ceph-mon[33178]: from='client.36383 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi057-77362-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi057-77362-62"}]: dispatch 2024-06-11T13:35:38.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:37 smithi107 ceph-mon[33178]: pgmap v476: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:35:38.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:37 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:35:38.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:37 smithi107 ceph-mon[33178]: osdmap e339: 8 total, 8 up, 8 in 2024-06-11T13:35:38.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:37 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:35:39.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:39 smithi057 ceph-mon[29805]: from='client.36383 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi057-77362-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi057-77362-62"}]': finished 2024-06-11T13:35:39.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:39 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:35:39.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:39 smithi057 ceph-mon[29805]: osdmap e340: 8 total, 8 up, 8 in 2024-06-11T13:35:39.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:39 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:35:39.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:39 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:35:39.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:39 smithi057 ceph-mon[40588]: from='client.36383 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi057-77362-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi057-77362-62"}]': finished 2024-06-11T13:35:39.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:39 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:35:39.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:39 smithi057 ceph-mon[40588]: osdmap e340: 8 total, 8 up, 8 in 2024-06-11T13:35:39.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:39 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:35:39.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:39 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:35:39.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:39 smithi107 ceph-mon[33178]: from='client.36383 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi057-77362-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi057-77362-62"}]': finished 2024-06-11T13:35:39.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:39 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:35:39.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:39 smithi107 ceph-mon[33178]: osdmap e340: 8 total, 8 up, 8 in 2024-06-11T13:35:39.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:39 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:35:39.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:39 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:35:40.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:40 smithi057 ceph-mon[29805]: pgmap v479: 332 pgs: 40 unknown, 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:35:40.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:40 smithi057 ceph-mon[29805]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:35:40.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:40 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-47", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:35:40.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:40 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-47"}]: dispatch 2024-06-11T13:35:40.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:40 smithi057 ceph-mon[29805]: osdmap e341: 8 total, 8 up, 8 in 2024-06-11T13:35:40.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:40 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-47"}]: dispatch 2024-06-11T13:35:40.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:40 smithi057 ceph-mon[40588]: pgmap v479: 332 pgs: 40 unknown, 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:35:40.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:40 smithi057 ceph-mon[40588]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:35:40.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:40 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-47", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:35:40.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:40 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-47"}]: dispatch 2024-06-11T13:35:40.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:40 smithi057 ceph-mon[40588]: osdmap e341: 8 total, 8 up, 8 in 2024-06-11T13:35:40.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:40 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-47"}]: dispatch 2024-06-11T13:35:40.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:40 smithi107 ceph-mon[33178]: pgmap v479: 332 pgs: 40 unknown, 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:35:40.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:40 smithi107 ceph-mon[33178]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:35:40.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:40 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-47", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:35:40.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:40 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-47"}]: dispatch 2024-06-11T13:35:40.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:40 smithi107 ceph-mon[33178]: osdmap e341: 8 total, 8 up, 8 in 2024-06-11T13:35:40.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:40 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-47"}]: dispatch 2024-06-11T13:35:41.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:41 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-47"}]': finished 2024-06-11T13:35:41.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:41 smithi057 ceph-mon[29805]: osdmap e342: 8 total, 8 up, 8 in 2024-06-11T13:35:41.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:41 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-47", "mode": "writeback"}]: dispatch 2024-06-11T13:35:41.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:41 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/251888670' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi057-77362-62"}]: dispatch 2024-06-11T13:35:41.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:41 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-47", "mode": "writeback"}]: dispatch 2024-06-11T13:35:41.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:41 smithi057 ceph-mon[29805]: from='client.36383 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi057-77362-62"}]: dispatch 2024-06-11T13:35:41.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:41 smithi057 ceph-mon[29805]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:35:41.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:41 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-47", "mode": "writeback"}]': finished 2024-06-11T13:35:41.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:41 smithi057 ceph-mon[29805]: from='client.36383 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi057-77362-62"}]': finished 2024-06-11T13:35:41.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:41 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-47"}]': finished 2024-06-11T13:35:41.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:41 smithi057 ceph-mon[40588]: osdmap e342: 8 total, 8 up, 8 in 2024-06-11T13:35:41.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:41 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-47", "mode": "writeback"}]: dispatch 2024-06-11T13:35:41.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:41 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/251888670' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi057-77362-62"}]: dispatch 2024-06-11T13:35:41.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:41 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-47", "mode": "writeback"}]: dispatch 2024-06-11T13:35:41.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:41 smithi057 ceph-mon[40588]: from='client.36383 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi057-77362-62"}]: dispatch 2024-06-11T13:35:41.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:41 smithi057 ceph-mon[40588]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:35:41.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:41 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-47", "mode": "writeback"}]': finished 2024-06-11T13:35:41.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:41 smithi057 ceph-mon[40588]: from='client.36383 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi057-77362-62"}]': finished 2024-06-11T13:35:41.798 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:35:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:35:41.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:41 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-47"}]': finished 2024-06-11T13:35:41.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:41 smithi107 ceph-mon[33178]: osdmap e342: 8 total, 8 up, 8 in 2024-06-11T13:35:41.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:41 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-47", "mode": "writeback"}]: dispatch 2024-06-11T13:35:41.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:41 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/251888670' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi057-77362-62"}]: dispatch 2024-06-11T13:35:41.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:41 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-47", "mode": "writeback"}]: dispatch 2024-06-11T13:35:41.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:41 smithi107 ceph-mon[33178]: from='client.36383 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi057-77362-62"}]: dispatch 2024-06-11T13:35:41.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:41 smithi107 ceph-mon[33178]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:35:41.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:41 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-47", "mode": "writeback"}]': finished 2024-06-11T13:35:41.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:41 smithi107 ceph-mon[33178]: from='client.36383 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi057-77362-62"}]': finished 2024-06-11T13:35:42.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:42 smithi057 ceph-mon[29805]: pgmap v482: 324 pgs: 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-06-11T13:35:42.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:42 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:35:42.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:42 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:35:42.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:42 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/251888670' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi057-77362-62"}]: dispatch 2024-06-11T13:35:42.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:42 smithi057 ceph-mon[29805]: osdmap e343: 8 total, 8 up, 8 in 2024-06-11T13:35:42.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:42 smithi057 ceph-mon[29805]: from='client.36383 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi057-77362-62"}]: dispatch 2024-06-11T13:35:42.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:42 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:35:42.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:42 smithi057 ceph-mon[29805]: from='client.36383 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi057-77362-62"}]': finished 2024-06-11T13:35:42.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:42 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "hit_set_count","val": "2"}]': finished 2024-06-11T13:35:42.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:42 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:35:42.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:42 smithi057 ceph-mon[29805]: osdmap e344: 8 total, 8 up, 8 in 2024-06-11T13:35:42.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:42 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:35:42.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:42 smithi057 ceph-mon[40588]: pgmap v482: 324 pgs: 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-06-11T13:35:42.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:42 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:35:42.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:42 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:35:42.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:42 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/251888670' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi057-77362-62"}]: dispatch 2024-06-11T13:35:42.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:42 smithi057 ceph-mon[40588]: osdmap e343: 8 total, 8 up, 8 in 2024-06-11T13:35:42.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:42 smithi057 ceph-mon[40588]: from='client.36383 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi057-77362-62"}]: dispatch 2024-06-11T13:35:42.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:42 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:35:42.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:42 smithi057 ceph-mon[40588]: from='client.36383 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi057-77362-62"}]': finished 2024-06-11T13:35:42.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:42 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "hit_set_count","val": "2"}]': finished 2024-06-11T13:35:42.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:42 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:35:42.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:42 smithi057 ceph-mon[40588]: osdmap e344: 8 total, 8 up, 8 in 2024-06-11T13:35:42.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:42 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:35:42.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:42 smithi107 ceph-mon[33178]: pgmap v482: 324 pgs: 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-06-11T13:35:42.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:42 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:35:42.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:42 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:35:42.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:42 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/251888670' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi057-77362-62"}]: dispatch 2024-06-11T13:35:42.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:42 smithi107 ceph-mon[33178]: osdmap e343: 8 total, 8 up, 8 in 2024-06-11T13:35:42.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:42 smithi107 ceph-mon[33178]: from='client.36383 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi057-77362-62"}]: dispatch 2024-06-11T13:35:42.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:42 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:35:42.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:42 smithi107 ceph-mon[33178]: from='client.36383 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi057-77362-62"}]': finished 2024-06-11T13:35:42.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:42 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "hit_set_count","val": "2"}]': finished 2024-06-11T13:35:42.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:42 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:35:42.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:42 smithi107 ceph-mon[33178]: osdmap e344: 8 total, 8 up, 8 in 2024-06-11T13:35:42.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:42 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:35:43.732 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:43 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/4215559182' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi057-77362-63"}]: dispatch 2024-06-11T13:35:43.733 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:43 smithi057 ceph-mon[29805]: from='client.36389 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi057-77362-63"}]: dispatch 2024-06-11T13:35:43.733 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:43 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/4215559182' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi057-77362-63"}]: dispatch 2024-06-11T13:35:43.733 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:43 smithi057 ceph-mon[29805]: from='client.36389 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi057-77362-63"}]: dispatch 2024-06-11T13:35:43.733 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:43 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/4215559182' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi057-77362-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:35:43.733 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:43 smithi057 ceph-mon[29805]: from='client.36389 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi057-77362-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:35:43.733 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:43 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "hit_set_period","val": "600"}]': finished 2024-06-11T13:35:43.733 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:43 smithi057 ceph-mon[29805]: from='client.36389 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi057-77362-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:35:43.733 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:43 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-11T13:35:43.733 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:43 smithi057 ceph-mon[29805]: osdmap e345: 8 total, 8 up, 8 in 2024-06-11T13:35:43.734 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:43 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-11T13:35:43.734 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:43 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/4215559182' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi057-77362-63"}]: dispatch 2024-06-11T13:35:43.734 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:43 smithi057 ceph-mon[40588]: from='client.36389 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi057-77362-63"}]: dispatch 2024-06-11T13:35:43.734 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:43 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/4215559182' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi057-77362-63"}]: dispatch 2024-06-11T13:35:43.734 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:43 smithi057 ceph-mon[40588]: from='client.36389 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi057-77362-63"}]: dispatch 2024-06-11T13:35:43.734 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:43 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/4215559182' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi057-77362-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:35:43.734 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:43 smithi057 ceph-mon[40588]: from='client.36389 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi057-77362-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:35:43.734 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:43 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "hit_set_period","val": "600"}]': finished 2024-06-11T13:35:43.734 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:43 smithi057 ceph-mon[40588]: from='client.36389 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi057-77362-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:35:43.734 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:43 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-11T13:35:43.735 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:43 smithi057 ceph-mon[40588]: osdmap e345: 8 total, 8 up, 8 in 2024-06-11T13:35:43.735 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:43 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-11T13:35:43.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:43 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/4215559182' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi057-77362-63"}]: dispatch 2024-06-11T13:35:43.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:43 smithi107 ceph-mon[33178]: from='client.36389 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi057-77362-63"}]: dispatch 2024-06-11T13:35:43.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:43 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/4215559182' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi057-77362-63"}]: dispatch 2024-06-11T13:35:43.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:43 smithi107 ceph-mon[33178]: from='client.36389 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi057-77362-63"}]: dispatch 2024-06-11T13:35:43.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:43 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/4215559182' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi057-77362-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:35:43.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:43 smithi107 ceph-mon[33178]: from='client.36389 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi057-77362-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:35:43.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:43 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "hit_set_period","val": "600"}]': finished 2024-06-11T13:35:43.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:43 smithi107 ceph-mon[33178]: from='client.36389 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi057-77362-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:35:43.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:43 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-11T13:35:43.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:43 smithi107 ceph-mon[33178]: osdmap e345: 8 total, 8 up, 8 in 2024-06-11T13:35:43.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:43 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-11T13:35:44.034 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:35:43 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:35:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:35:45.284 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:44 smithi057 ceph-mon[40588]: pgmap v485: 324 pgs: 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-06-11T13:35:45.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:44 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/4215559182' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi057-77362-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi057-77362-63"}]: dispatch 2024-06-11T13:35:45.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:44 smithi057 ceph-mon[40588]: from='client.36389 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi057-77362-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi057-77362-63"}]: dispatch 2024-06-11T13:35:45.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:44 smithi057 ceph-mon[40588]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:35:45.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:44 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "hit_set_type","val": "bloom"}]': finished 2024-06-11T13:35:45.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:44 smithi057 ceph-mon[29805]: pgmap v485: 324 pgs: 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-06-11T13:35:45.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:44 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/4215559182' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi057-77362-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi057-77362-63"}]: dispatch 2024-06-11T13:35:45.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:44 smithi057 ceph-mon[29805]: from='client.36389 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi057-77362-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi057-77362-63"}]: dispatch 2024-06-11T13:35:45.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:44 smithi057 ceph-mon[29805]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:35:45.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:44 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "hit_set_type","val": "bloom"}]': finished 2024-06-11T13:35:45.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:44 smithi107 ceph-mon[33178]: pgmap v485: 324 pgs: 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-06-11T13:35:45.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:44 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/4215559182' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi057-77362-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi057-77362-63"}]: dispatch 2024-06-11T13:35:45.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:44 smithi107 ceph-mon[33178]: from='client.36389 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi057-77362-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi057-77362-63"}]: dispatch 2024-06-11T13:35:45.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:44 smithi107 ceph-mon[33178]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:35:45.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:44 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "hit_set_type","val": "bloom"}]': finished 2024-06-11T13:35:46.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:45 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-11T13:35:46.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:45 smithi057 ceph-mon[29805]: osdmap e346: 8 total, 8 up, 8 in 2024-06-11T13:35:46.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:45 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-11T13:35:46.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:45 smithi057 ceph-mon[29805]: pgmap v488: 324 pgs: 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:35:46.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:45 smithi057 ceph-mon[29805]: from='client.36389 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi057-77362-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi057-77362-63"}]': finished 2024-06-11T13:35:46.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:45 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-06-11T13:35:46.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:45 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-06-11T13:35:46.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:45 smithi057 ceph-mon[29805]: osdmap e347: 8 total, 8 up, 8 in 2024-06-11T13:35:46.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:45 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-06-11T13:35:46.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:45 smithi057 ceph-mon[29805]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:35:46.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:45 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-11T13:35:46.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:45 smithi057 ceph-mon[40588]: osdmap e346: 8 total, 8 up, 8 in 2024-06-11T13:35:46.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:45 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-11T13:35:46.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:45 smithi057 ceph-mon[40588]: pgmap v488: 324 pgs: 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:35:46.288 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:45 smithi057 ceph-mon[40588]: from='client.36389 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi057-77362-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi057-77362-63"}]': finished 2024-06-11T13:35:46.288 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:45 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-06-11T13:35:46.288 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:45 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-06-11T13:35:46.288 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:45 smithi057 ceph-mon[40588]: osdmap e347: 8 total, 8 up, 8 in 2024-06-11T13:35:46.288 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:45 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-06-11T13:35:46.288 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:45 smithi057 ceph-mon[40588]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:35:46.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:45 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-11T13:35:46.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:45 smithi107 ceph-mon[33178]: osdmap e346: 8 total, 8 up, 8 in 2024-06-11T13:35:46.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:45 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-11T13:35:46.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:45 smithi107 ceph-mon[33178]: pgmap v488: 324 pgs: 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:35:46.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:45 smithi107 ceph-mon[33178]: from='client.36389 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi057-77362-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi057-77362-63"}]': finished 2024-06-11T13:35:46.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:45 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-06-11T13:35:46.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:45 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-06-11T13:35:46.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:45 smithi107 ceph-mon[33178]: osdmap e347: 8 total, 8 up, 8 in 2024-06-11T13:35:46.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:45 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-06-11T13:35:46.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:45 smithi107 ceph-mon[33178]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:35:47.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:47 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-06-11T13:35:47.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:47 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-06-11T13:35:47.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:47 smithi057 ceph-mon[29805]: osdmap e348: 8 total, 8 up, 8 in 2024-06-11T13:35:47.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:47 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-06-11T13:35:47.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:47 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-06-11T13:35:47.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:47 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-06-11T13:35:47.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:47 smithi057 ceph-mon[40588]: osdmap e348: 8 total, 8 up, 8 in 2024-06-11T13:35:47.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:47 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-06-11T13:35:47.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:47 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-06-11T13:35:47.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:47 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-06-11T13:35:47.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:47 smithi107 ceph-mon[33178]: osdmap e348: 8 total, 8 up, 8 in 2024-06-11T13:35:47.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:47 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-06-11T13:35:48.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:48 smithi057 ceph-mon[29805]: pgmap v491: 332 pgs: 8 creating+peering, 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:35:48.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:48 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-06-11T13:35:48.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:48 smithi057 ceph-mon[29805]: osdmap e349: 8 total, 8 up, 8 in 2024-06-11T13:35:48.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:48 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/4215559182' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi057-77362-63"}]: dispatch 2024-06-11T13:35:48.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:48 smithi057 ceph-mon[29805]: from='client.36389 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi057-77362-63"}]: dispatch 2024-06-11T13:35:48.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:48 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:35:48.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:48 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:35:48.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:48 smithi057 ceph-mon[40588]: pgmap v491: 332 pgs: 8 creating+peering, 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:35:48.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:48 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-06-11T13:35:48.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:48 smithi057 ceph-mon[40588]: osdmap e349: 8 total, 8 up, 8 in 2024-06-11T13:35:48.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:48 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/4215559182' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi057-77362-63"}]: dispatch 2024-06-11T13:35:48.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:48 smithi057 ceph-mon[40588]: from='client.36389 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi057-77362-63"}]: dispatch 2024-06-11T13:35:48.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:48 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:35:48.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:48 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:35:48.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:48 smithi107 ceph-mon[33178]: pgmap v491: 332 pgs: 8 creating+peering, 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:35:48.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:48 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-06-11T13:35:48.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:48 smithi107 ceph-mon[33178]: osdmap e349: 8 total, 8 up, 8 in 2024-06-11T13:35:48.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:48 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/4215559182' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi057-77362-63"}]: dispatch 2024-06-11T13:35:48.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:48 smithi107 ceph-mon[33178]: from='client.36389 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi057-77362-63"}]: dispatch 2024-06-11T13:35:48.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:48 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:35:48.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:48 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:35:49.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:49 smithi057 ceph-mon[29805]: from='client.36389 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi057-77362-63"}]': finished 2024-06-11T13:35:49.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:49 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:35:49.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:49 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-47"}]: dispatch 2024-06-11T13:35:49.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:49 smithi057 ceph-mon[29805]: osdmap e350: 8 total, 8 up, 8 in 2024-06-11T13:35:49.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:49 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-47"}]: dispatch 2024-06-11T13:35:49.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:49 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/4215559182' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi057-77362-63"}]: dispatch 2024-06-11T13:35:49.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:49 smithi057 ceph-mon[29805]: from='client.36389 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi057-77362-63"}]: dispatch 2024-06-11T13:35:49.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:49 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-47"}]': finished 2024-06-11T13:35:49.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:49 smithi057 ceph-mon[29805]: from='client.36389 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi057-77362-63"}]': finished 2024-06-11T13:35:49.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:49 smithi057 ceph-mon[29805]: osdmap e351: 8 total, 8 up, 8 in 2024-06-11T13:35:49.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:49 smithi057 ceph-mon[40588]: from='client.36389 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi057-77362-63"}]': finished 2024-06-11T13:35:49.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:49 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:35:49.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:49 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-47"}]: dispatch 2024-06-11T13:35:49.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:49 smithi057 ceph-mon[40588]: osdmap e350: 8 total, 8 up, 8 in 2024-06-11T13:35:49.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:49 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-47"}]: dispatch 2024-06-11T13:35:49.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:49 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/4215559182' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi057-77362-63"}]: dispatch 2024-06-11T13:35:49.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:49 smithi057 ceph-mon[40588]: from='client.36389 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi057-77362-63"}]: dispatch 2024-06-11T13:35:49.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:49 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-47"}]': finished 2024-06-11T13:35:49.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:49 smithi057 ceph-mon[40588]: from='client.36389 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi057-77362-63"}]': finished 2024-06-11T13:35:49.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:49 smithi057 ceph-mon[40588]: osdmap e351: 8 total, 8 up, 8 in 2024-06-11T13:35:49.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:49 smithi107 ceph-mon[33178]: from='client.36389 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi057-77362-63"}]': finished 2024-06-11T13:35:49.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:49 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:35:49.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:49 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-47"}]: dispatch 2024-06-11T13:35:49.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:49 smithi107 ceph-mon[33178]: osdmap e350: 8 total, 8 up, 8 in 2024-06-11T13:35:49.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:49 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-47"}]: dispatch 2024-06-11T13:35:49.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:49 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/4215559182' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi057-77362-63"}]: dispatch 2024-06-11T13:35:49.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:49 smithi107 ceph-mon[33178]: from='client.36389 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi057-77362-63"}]: dispatch 2024-06-11T13:35:49.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:49 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-47"}]': finished 2024-06-11T13:35:49.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:49 smithi107 ceph-mon[33178]: from='client.36389 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi057-77362-63"}]': finished 2024-06-11T13:35:49.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:49 smithi107 ceph-mon[33178]: osdmap e351: 8 total, 8 up, 8 in 2024-06-11T13:35:50.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:50 smithi057 ceph-mon[40588]: pgmap v494: 324 pgs: 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:35:50.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:50 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/4091564414' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi057-77362-64"}]: dispatch 2024-06-11T13:35:50.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:50 smithi057 ceph-mon[40588]: from='client.26389 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi057-77362-64"}]: dispatch 2024-06-11T13:35:50.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:50 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/4091564414' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi057-77362-64"}]: dispatch 2024-06-11T13:35:50.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:50 smithi057 ceph-mon[40588]: from='client.26389 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi057-77362-64"}]: dispatch 2024-06-11T13:35:50.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:50 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/4091564414' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi057-77362-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:35:50.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:50 smithi057 ceph-mon[40588]: from='client.26389 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi057-77362-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:35:50.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:50 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:35:50.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:50 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:35:50.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:50 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-47"}]: dispatch 2024-06-11T13:35:50.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:50 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-47"}]: dispatch 2024-06-11T13:35:50.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:50 smithi057 ceph-mon[29805]: pgmap v494: 324 pgs: 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:35:50.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:50 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/4091564414' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi057-77362-64"}]: dispatch 2024-06-11T13:35:50.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:50 smithi057 ceph-mon[29805]: from='client.26389 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi057-77362-64"}]: dispatch 2024-06-11T13:35:50.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:50 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/4091564414' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi057-77362-64"}]: dispatch 2024-06-11T13:35:50.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:50 smithi057 ceph-mon[29805]: from='client.26389 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi057-77362-64"}]: dispatch 2024-06-11T13:35:50.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:50 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/4091564414' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi057-77362-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:35:50.788 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:50 smithi057 ceph-mon[29805]: from='client.26389 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi057-77362-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:35:50.788 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:50 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:35:50.788 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:50 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:35:50.788 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:50 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-47"}]: dispatch 2024-06-11T13:35:50.788 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:50 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-47"}]: dispatch 2024-06-11T13:35:50.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:50 smithi107 ceph-mon[33178]: pgmap v494: 324 pgs: 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:35:50.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:50 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/4091564414' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi057-77362-64"}]: dispatch 2024-06-11T13:35:50.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:50 smithi107 ceph-mon[33178]: from='client.26389 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi057-77362-64"}]: dispatch 2024-06-11T13:35:50.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:50 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/4091564414' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi057-77362-64"}]: dispatch 2024-06-11T13:35:50.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:50 smithi107 ceph-mon[33178]: from='client.26389 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi057-77362-64"}]: dispatch 2024-06-11T13:35:50.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:50 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/4091564414' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi057-77362-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:35:50.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:50 smithi107 ceph-mon[33178]: from='client.26389 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi057-77362-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:35:50.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:50 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:35:50.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:50 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:35:50.800 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:50 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-47"}]: dispatch 2024-06-11T13:35:50.800 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:50 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-47"}]: dispatch 2024-06-11T13:35:51.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:51 smithi057 ceph-mon[29805]: from='client.26389 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi057-77362-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:35:51.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:51 smithi057 ceph-mon[29805]: osdmap e352: 8 total, 8 up, 8 in 2024-06-11T13:35:51.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:51 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/4091564414' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi057-77362-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi057-77362-64"}]: dispatch 2024-06-11T13:35:51.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:51 smithi057 ceph-mon[29805]: from='client.26389 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi057-77362-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi057-77362-64"}]: dispatch 2024-06-11T13:35:51.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:51 smithi057 ceph-mon[40588]: from='client.26389 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi057-77362-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:35:51.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:51 smithi057 ceph-mon[40588]: osdmap e352: 8 total, 8 up, 8 in 2024-06-11T13:35:51.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:51 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/4091564414' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi057-77362-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi057-77362-64"}]: dispatch 2024-06-11T13:35:51.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:51 smithi057 ceph-mon[40588]: from='client.26389 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi057-77362-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi057-77362-64"}]: dispatch 2024-06-11T13:35:51.797 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:35:51 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:35:51.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:51 smithi107 ceph-mon[33178]: from='client.26389 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi057-77362-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:35:51.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:51 smithi107 ceph-mon[33178]: osdmap e352: 8 total, 8 up, 8 in 2024-06-11T13:35:51.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:51 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/4091564414' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi057-77362-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi057-77362-64"}]: dispatch 2024-06-11T13:35:51.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:51 smithi107 ceph-mon[33178]: from='client.26389 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi057-77362-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi057-77362-64"}]: dispatch 2024-06-11T13:35:52.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:52 smithi057 ceph-mon[29805]: pgmap v497: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-06-11T13:35:52.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:52 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:35:52.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:52 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:35:52.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:52 smithi057 ceph-mon[29805]: osdmap e353: 8 total, 8 up, 8 in 2024-06-11T13:35:52.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:52 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:35:52.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:52 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:35:52.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:52 smithi057 ceph-mon[40588]: pgmap v497: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-06-11T13:35:52.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:52 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:35:52.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:52 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:35:52.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:52 smithi057 ceph-mon[40588]: osdmap e353: 8 total, 8 up, 8 in 2024-06-11T13:35:52.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:52 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:35:52.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:52 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:35:52.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:52 smithi107 ceph-mon[33178]: pgmap v497: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-06-11T13:35:52.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:52 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:35:52.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:52 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:35:52.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:52 smithi107 ceph-mon[33178]: osdmap e353: 8 total, 8 up, 8 in 2024-06-11T13:35:52.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:52 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:35:52.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:52 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:35:53.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:53 smithi107 ceph-mon[33178]: from='client.26389 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi057-77362-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi057-77362-64"}]': finished 2024-06-11T13:35:53.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:53 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:35:53.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:53 smithi107 ceph-mon[33178]: osdmap e354: 8 total, 8 up, 8 in 2024-06-11T13:35:53.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:53 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:35:53.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:53 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:35:54.034 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:35:53 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:35:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:35:54.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:53 smithi057 ceph-mon[29805]: from='client.26389 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi057-77362-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi057-77362-64"}]': finished 2024-06-11T13:35:54.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:53 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:35:54.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:53 smithi057 ceph-mon[29805]: osdmap e354: 8 total, 8 up, 8 in 2024-06-11T13:35:54.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:53 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:35:54.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:53 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:35:54.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:53 smithi057 ceph-mon[40588]: from='client.26389 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi057-77362-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi057-77362-64"}]': finished 2024-06-11T13:35:54.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:53 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:35:54.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:53 smithi057 ceph-mon[40588]: osdmap e354: 8 total, 8 up, 8 in 2024-06-11T13:35:54.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:53 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:35:54.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:53 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:35:54.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:54 smithi107 ceph-mon[33178]: pgmap v500: 332 pgs: 40 unknown, 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-06-11T13:35:54.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:54 smithi107 ceph-mon[33178]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:35:54.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:54 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-49", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:35:54.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:54 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-49"}]: dispatch 2024-06-11T13:35:54.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:54 smithi107 ceph-mon[33178]: osdmap e355: 8 total, 8 up, 8 in 2024-06-11T13:35:54.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:54 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-49"}]: dispatch 2024-06-11T13:35:55.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:54 smithi057 ceph-mon[29805]: pgmap v500: 332 pgs: 40 unknown, 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-06-11T13:35:55.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:54 smithi057 ceph-mon[29805]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:35:55.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:54 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-49", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:35:55.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:54 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-49"}]: dispatch 2024-06-11T13:35:55.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:54 smithi057 ceph-mon[29805]: osdmap e355: 8 total, 8 up, 8 in 2024-06-11T13:35:55.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:54 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-49"}]: dispatch 2024-06-11T13:35:55.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:54 smithi057 ceph-mon[40588]: pgmap v500: 332 pgs: 40 unknown, 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-06-11T13:35:55.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:54 smithi057 ceph-mon[40588]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:35:55.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:54 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-49", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:35:55.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:54 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-49"}]: dispatch 2024-06-11T13:35:55.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:54 smithi057 ceph-mon[40588]: osdmap e355: 8 total, 8 up, 8 in 2024-06-11T13:35:55.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:54 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-49"}]: dispatch 2024-06-11T13:35:55.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:55 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-49"}]': finished 2024-06-11T13:35:55.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:55 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-49", "mode": "readproxy"}]: dispatch 2024-06-11T13:35:55.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:55 smithi107 ceph-mon[33178]: osdmap e356: 8 total, 8 up, 8 in 2024-06-11T13:35:55.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:55 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/4091564414' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi057-77362-64"}]: dispatch 2024-06-11T13:35:55.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:55 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-49", "mode": "readproxy"}]: dispatch 2024-06-11T13:35:55.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:55 smithi107 ceph-mon[33178]: from='client.26389 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi057-77362-64"}]: dispatch 2024-06-11T13:35:55.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:55 smithi107 ceph-mon[33178]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:35:55.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:55 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-49", "mode": "readproxy"}]': finished 2024-06-11T13:35:55.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:55 smithi107 ceph-mon[33178]: from='client.26389 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi057-77362-64"}]': finished 2024-06-11T13:35:55.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:55 smithi107 ceph-mon[33178]: osdmap e357: 8 total, 8 up, 8 in 2024-06-11T13:35:56.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:55 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-49"}]': finished 2024-06-11T13:35:56.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:55 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-49", "mode": "readproxy"}]: dispatch 2024-06-11T13:35:56.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:55 smithi057 ceph-mon[29805]: osdmap e356: 8 total, 8 up, 8 in 2024-06-11T13:35:56.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:55 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/4091564414' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi057-77362-64"}]: dispatch 2024-06-11T13:35:56.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:55 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-49", "mode": "readproxy"}]: dispatch 2024-06-11T13:35:56.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:55 smithi057 ceph-mon[29805]: from='client.26389 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi057-77362-64"}]: dispatch 2024-06-11T13:35:56.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:55 smithi057 ceph-mon[29805]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:35:56.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:55 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-49", "mode": "readproxy"}]': finished 2024-06-11T13:35:56.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:55 smithi057 ceph-mon[29805]: from='client.26389 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi057-77362-64"}]': finished 2024-06-11T13:35:56.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:55 smithi057 ceph-mon[29805]: osdmap e357: 8 total, 8 up, 8 in 2024-06-11T13:35:56.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:55 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-49"}]': finished 2024-06-11T13:35:56.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:55 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-49", "mode": "readproxy"}]: dispatch 2024-06-11T13:35:56.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:55 smithi057 ceph-mon[40588]: osdmap e356: 8 total, 8 up, 8 in 2024-06-11T13:35:56.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:55 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/4091564414' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi057-77362-64"}]: dispatch 2024-06-11T13:35:56.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:55 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-49", "mode": "readproxy"}]: dispatch 2024-06-11T13:35:56.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:55 smithi057 ceph-mon[40588]: from='client.26389 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi057-77362-64"}]: dispatch 2024-06-11T13:35:56.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:55 smithi057 ceph-mon[40588]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:35:56.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:55 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-49", "mode": "readproxy"}]': finished 2024-06-11T13:35:56.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:55 smithi057 ceph-mon[40588]: from='client.26389 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi057-77362-64"}]': finished 2024-06-11T13:35:56.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:55 smithi057 ceph-mon[40588]: osdmap e357: 8 total, 8 up, 8 in 2024-06-11T13:35:57.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:56 smithi057 ceph-mon[29805]: pgmap v503: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:35:57.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:56 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/4091564414' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi057-77362-64"}]: dispatch 2024-06-11T13:35:57.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:56 smithi057 ceph-mon[29805]: from='client.26389 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi057-77362-64"}]: dispatch 2024-06-11T13:35:57.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:56 smithi057 ceph-mon[29805]: from='client.26389 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi057-77362-64"}]': finished 2024-06-11T13:35:57.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:56 smithi057 ceph-mon[40588]: pgmap v503: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:35:57.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:56 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/4091564414' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi057-77362-64"}]: dispatch 2024-06-11T13:35:57.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:56 smithi057 ceph-mon[40588]: from='client.26389 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi057-77362-64"}]: dispatch 2024-06-11T13:35:57.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:56 smithi057 ceph-mon[40588]: from='client.26389 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi057-77362-64"}]': finished 2024-06-11T13:35:57.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:56 smithi107 ceph-mon[33178]: pgmap v503: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:35:57.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:56 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/4091564414' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi057-77362-64"}]: dispatch 2024-06-11T13:35:57.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:56 smithi107 ceph-mon[33178]: from='client.26389 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi057-77362-64"}]: dispatch 2024-06-11T13:35:57.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:56 smithi107 ceph-mon[33178]: from='client.26389 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi057-77362-64"}]': finished 2024-06-11T13:35:58.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:57 smithi057 ceph-mon[29805]: osdmap e358: 8 total, 8 up, 8 in 2024-06-11T13:35:58.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:57 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2666170183' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi057-77362-65"}]: dispatch 2024-06-11T13:35:58.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:57 smithi057 ceph-mon[29805]: from='client.36401 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi057-77362-65"}]: dispatch 2024-06-11T13:35:58.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:57 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2666170183' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi057-77362-65"}]: dispatch 2024-06-11T13:35:58.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:57 smithi057 ceph-mon[29805]: from='client.36401 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi057-77362-65"}]: dispatch 2024-06-11T13:35:58.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:57 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2666170183' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi057-77362-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:35:58.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:57 smithi057 ceph-mon[29805]: from='client.36401 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi057-77362-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:35:58.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:57 smithi057 ceph-mon[40588]: osdmap e358: 8 total, 8 up, 8 in 2024-06-11T13:35:58.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:57 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2666170183' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi057-77362-65"}]: dispatch 2024-06-11T13:35:58.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:57 smithi057 ceph-mon[40588]: from='client.36401 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi057-77362-65"}]: dispatch 2024-06-11T13:35:58.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:57 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2666170183' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi057-77362-65"}]: dispatch 2024-06-11T13:35:58.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:57 smithi057 ceph-mon[40588]: from='client.36401 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi057-77362-65"}]: dispatch 2024-06-11T13:35:58.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:57 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2666170183' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi057-77362-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:35:58.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:57 smithi057 ceph-mon[40588]: from='client.36401 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi057-77362-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:35:58.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:57 smithi107 ceph-mon[33178]: osdmap e358: 8 total, 8 up, 8 in 2024-06-11T13:35:58.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:57 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2666170183' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi057-77362-65"}]: dispatch 2024-06-11T13:35:58.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:57 smithi107 ceph-mon[33178]: from='client.36401 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi057-77362-65"}]: dispatch 2024-06-11T13:35:58.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:57 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2666170183' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi057-77362-65"}]: dispatch 2024-06-11T13:35:58.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:57 smithi107 ceph-mon[33178]: from='client.36401 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi057-77362-65"}]: dispatch 2024-06-11T13:35:58.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:57 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2666170183' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi057-77362-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:35:58.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:57 smithi107 ceph-mon[33178]: from='client.36401 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi057-77362-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:35:59.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:59 smithi057 ceph-mon[29805]: pgmap v506: 324 pgs: 324 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:35:59.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:59 smithi057 ceph-mon[29805]: from='client.36401 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi057-77362-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:35:59.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:59 smithi057 ceph-mon[29805]: osdmap e359: 8 total, 8 up, 8 in 2024-06-11T13:35:59.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:59 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2666170183' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi057-77362-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi057-77362-65"}]: dispatch 2024-06-11T13:35:59.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:35:59 smithi057 ceph-mon[29805]: from='client.36401 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi057-77362-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi057-77362-65"}]: dispatch 2024-06-11T13:35:59.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:59 smithi057 ceph-mon[40588]: pgmap v506: 324 pgs: 324 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:35:59.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:59 smithi057 ceph-mon[40588]: from='client.36401 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi057-77362-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:35:59.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:59 smithi057 ceph-mon[40588]: osdmap e359: 8 total, 8 up, 8 in 2024-06-11T13:35:59.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:59 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2666170183' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi057-77362-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi057-77362-65"}]: dispatch 2024-06-11T13:35:59.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:35:59 smithi057 ceph-mon[40588]: from='client.36401 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi057-77362-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi057-77362-65"}]: dispatch 2024-06-11T13:35:59.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:59 smithi107 ceph-mon[33178]: pgmap v506: 324 pgs: 324 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:35:59.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:59 smithi107 ceph-mon[33178]: from='client.36401 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi057-77362-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:35:59.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:59 smithi107 ceph-mon[33178]: osdmap e359: 8 total, 8 up, 8 in 2024-06-11T13:35:59.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:59 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2666170183' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi057-77362-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi057-77362-65"}]: dispatch 2024-06-11T13:35:59.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:35:59 smithi107 ceph-mon[33178]: from='client.36401 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi057-77362-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi057-77362-65"}]: dispatch 2024-06-11T13:35:59.772 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP started 0x564078a7b420 2024-06-11T13:35:59.773 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7b530 2024-06-11T13:35:59.773 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7b660 2024-06-11T13:35:59.773 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7c040 2024-06-11T13:35:59.773 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7ba40 2024-06-11T13:35:59.773 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7bb90 2024-06-11T13:35:59.773 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7bcc0 2024-06-11T13:35:59.773 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7c190 2024-06-11T13:35:59.773 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7c660 2024-06-11T13:35:59.773 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7c7b0 2024-06-11T13:35:59.773 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a786c0 2024-06-11T13:35:59.773 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a787d0 2024-06-11T13:35:59.774 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a78900 2024-06-11T13:35:59.774 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7c900 2024-06-11T13:35:59.774 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7cdd0 2024-06-11T13:35:59.774 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7cee0 2024-06-11T13:35:59.774 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7c2c0 2024-06-11T13:35:59.774 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7c3f0 2024-06-11T13:35:59.774 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7c520 2024-06-11T13:35:59.774 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7d010 2024-06-11T13:35:59.774 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7d4e0 2024-06-11T13:35:59.774 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7d610 2024-06-11T13:35:59.774 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7ca30 2024-06-11T13:35:59.774 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7cb60 2024-06-11T13:35:59.775 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7ccb0 2024-06-11T13:35:59.775 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7d760 2024-06-11T13:35:59.775 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7dc30 2024-06-11T13:35:59.775 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7dd80 2024-06-11T13:35:59.775 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7e360 2024-06-11T13:35:59.775 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7d890 2024-06-11T13:35:59.775 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7d9c0 2024-06-11T13:35:59.775 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7daf0 2024-06-11T13:35:59.775 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7e490 2024-06-11T13:35:59.775 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7deb0 2024-06-11T13:35:59.775 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7dfe0 2024-06-11T13:35:59.775 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7e130 2024-06-11T13:35:59.775 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7e5c0 2024-06-11T13:35:59.775 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7e710 2024-06-11T13:35:59.776 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7ed10 2024-06-11T13:35:59.776 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7e860 2024-06-11T13:35:59.776 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7e9b0 2024-06-11T13:35:59.776 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7eac0 2024-06-11T13:35:59.776 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7ebf0 2024-06-11T13:35:59.776 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7f2d0 2024-06-11T13:35:59.776 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7ee20 2024-06-11T13:35:59.776 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7ef90 2024-06-11T13:35:59.776 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7f0a0 2024-06-11T13:35:59.776 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7f420 2024-06-11T13:35:59.776 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7f8f0 2024-06-11T13:35:59.776 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7fa00 2024-06-11T13:35:59.776 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7e240 2024-06-11T13:35:59.776 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7fb50 2024-06-11T13:35:59.776 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7fca0 2024-06-11T13:35:59.777 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7f550 2024-06-11T13:35:59.777 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7f680 2024-06-11T13:35:59.777 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7f7d0 2024-06-11T13:35:59.777 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7fdd0 2024-06-11T13:35:59.777 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a802a0 2024-06-11T13:35:59.777 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a803d0 2024-06-11T13:35:59.777 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7d140 2024-06-11T13:35:59.777 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7d250 2024-06-11T13:35:59.777 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7d3a0 2024-06-11T13:35:59.777 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a80500 2024-06-11T13:35:59.777 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a809d0 2024-06-11T13:35:59.777 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a80b00 2024-06-11T13:35:59.777 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a7fee0 2024-06-11T13:35:59.777 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a80010 2024-06-11T13:35:59.778 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a80140 2024-06-11T13:35:59.778 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a80c50 2024-06-11T13:35:59.778 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a80da0 2024-06-11T13:35:59.778 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a813a0 2024-06-11T13:35:59.778 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a80ef0 2024-06-11T13:35:59.778 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a81040 2024-06-11T13:35:59.778 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a81150 2024-06-11T13:35:59.778 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a81280 2024-06-11T13:35:59.778 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a81960 2024-06-11T13:35:59.778 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a814b0 2024-06-11T13:35:59.778 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a81600 2024-06-11T13:35:59.778 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a81730 2024-06-11T13:35:59.779 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a81a90 2024-06-11T13:35:59.779 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a81f60 2024-06-11T13:35:59.779 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a82070 2024-06-11T13:36:00.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:00 smithi057 ceph-mon[29805]: osdmap e360: 8 total, 8 up, 8 in 2024-06-11T13:36:00.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:00 smithi057 ceph-mon[29805]: pgmap v509: 324 pgs: 324 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:36:00.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:00 smithi057 ceph-mon[29805]: from='client.36401 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi057-77362-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi057-77362-65"}]': finished 2024-06-11T13:36:00.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:00 smithi057 ceph-mon[29805]: osdmap e361: 8 total, 8 up, 8 in 2024-06-11T13:36:00.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:00 smithi057 ceph-mon[40588]: osdmap e360: 8 total, 8 up, 8 in 2024-06-11T13:36:00.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:00 smithi057 ceph-mon[40588]: pgmap v509: 324 pgs: 324 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:36:00.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:00 smithi057 ceph-mon[40588]: from='client.36401 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi057-77362-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi057-77362-65"}]': finished 2024-06-11T13:36:00.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:00 smithi057 ceph-mon[40588]: osdmap e361: 8 total, 8 up, 8 in 2024-06-11T13:36:00.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:00 smithi107 ceph-mon[33178]: osdmap e360: 8 total, 8 up, 8 in 2024-06-11T13:36:00.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:00 smithi107 ceph-mon[33178]: pgmap v509: 324 pgs: 324 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:36:00.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:00 smithi107 ceph-mon[33178]: from='client.36401 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi057-77362-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi057-77362-65"}]': finished 2024-06-11T13:36:00.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:00 smithi107 ceph-mon[33178]: osdmap e361: 8 total, 8 up, 8 in 2024-06-11T13:36:01.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:01 smithi107 ceph-mon[33178]: osdmap e362: 8 total, 8 up, 8 in 2024-06-11T13:36:01.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:01 smithi107 ceph-mon[33178]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:36:01.798 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:36:01 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:36:02.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:01 smithi057 ceph-mon[29805]: osdmap e362: 8 total, 8 up, 8 in 2024-06-11T13:36:02.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:01 smithi057 ceph-mon[29805]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:36:02.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:01 smithi057 ceph-mon[40588]: osdmap e362: 8 total, 8 up, 8 in 2024-06-11T13:36:02.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:01 smithi057 ceph-mon[40588]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:36:03.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:02 smithi057 ceph-mon[29805]: pgmap v512: 332 pgs: 332 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:36:03.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:02 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:36:03.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:02 smithi057 ceph-mon[29805]: osdmap e363: 8 total, 8 up, 8 in 2024-06-11T13:36:03.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:02 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2666170183' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi057-77362-65"}]: dispatch 2024-06-11T13:36:03.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:02 smithi057 ceph-mon[29805]: from='client.36401 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi057-77362-65"}]: dispatch 2024-06-11T13:36:03.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:02 smithi057 ceph-mon[40588]: pgmap v512: 332 pgs: 332 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:36:03.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:02 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:36:03.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:02 smithi057 ceph-mon[40588]: osdmap e363: 8 total, 8 up, 8 in 2024-06-11T13:36:03.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:02 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2666170183' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi057-77362-65"}]: dispatch 2024-06-11T13:36:03.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:02 smithi057 ceph-mon[40588]: from='client.36401 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi057-77362-65"}]: dispatch 2024-06-11T13:36:03.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:02 smithi107 ceph-mon[33178]: pgmap v512: 332 pgs: 332 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:36:03.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:02 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:36:03.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:02 smithi107 ceph-mon[33178]: osdmap e363: 8 total, 8 up, 8 in 2024-06-11T13:36:03.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:02 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2666170183' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi057-77362-65"}]: dispatch 2024-06-11T13:36:03.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:02 smithi107 ceph-mon[33178]: from='client.36401 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi057-77362-65"}]: dispatch 2024-06-11T13:36:04.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:03 smithi057 ceph-mon[29805]: from='client.36401 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi057-77362-65"}]': finished 2024-06-11T13:36:04.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:03 smithi057 ceph-mon[29805]: osdmap e364: 8 total, 8 up, 8 in 2024-06-11T13:36:04.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:03 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2666170183' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi057-77362-65"}]: dispatch 2024-06-11T13:36:04.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:03 smithi057 ceph-mon[29805]: from='client.36401 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi057-77362-65"}]: dispatch 2024-06-11T13:36:04.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:03 smithi057 ceph-mon[29805]: from='client.36401 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi057-77362-65"}]': finished 2024-06-11T13:36:04.035 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:36:03 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:36:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:36:04.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:03 smithi057 ceph-mon[40588]: from='client.36401 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi057-77362-65"}]': finished 2024-06-11T13:36:04.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:03 smithi057 ceph-mon[40588]: osdmap e364: 8 total, 8 up, 8 in 2024-06-11T13:36:04.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:03 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2666170183' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi057-77362-65"}]: dispatch 2024-06-11T13:36:04.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:03 smithi057 ceph-mon[40588]: from='client.36401 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi057-77362-65"}]: dispatch 2024-06-11T13:36:04.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:03 smithi057 ceph-mon[40588]: from='client.36401 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi057-77362-65"}]': finished 2024-06-11T13:36:04.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:03 smithi107 ceph-mon[33178]: from='client.36401 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi057-77362-65"}]': finished 2024-06-11T13:36:04.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:03 smithi107 ceph-mon[33178]: osdmap e364: 8 total, 8 up, 8 in 2024-06-11T13:36:04.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:03 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2666170183' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi057-77362-65"}]: dispatch 2024-06-11T13:36:04.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:03 smithi107 ceph-mon[33178]: from='client.36401 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi057-77362-65"}]: dispatch 2024-06-11T13:36:04.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:03 smithi107 ceph-mon[33178]: from='client.36401 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi057-77362-65"}]': finished 2024-06-11T13:36:05.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:04 smithi057 ceph-mon[29805]: pgmap v515: 324 pgs: 324 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:36:05.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:04 smithi057 ceph-mon[29805]: osdmap e365: 8 total, 8 up, 8 in 2024-06-11T13:36:05.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:04 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/101618264' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi057-77362-66"}]: dispatch 2024-06-11T13:36:05.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:04 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/101618264' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi057-77362-66"}]: dispatch 2024-06-11T13:36:05.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:04 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/101618264' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi057-77362-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:36:05.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:04 smithi057 ceph-mon[40588]: pgmap v515: 324 pgs: 324 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:36:05.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:04 smithi057 ceph-mon[40588]: osdmap e365: 8 total, 8 up, 8 in 2024-06-11T13:36:05.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:04 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/101618264' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi057-77362-66"}]: dispatch 2024-06-11T13:36:05.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:04 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/101618264' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi057-77362-66"}]: dispatch 2024-06-11T13:36:05.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:04 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/101618264' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi057-77362-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:36:05.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:04 smithi107 ceph-mon[33178]: pgmap v515: 324 pgs: 324 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:36:05.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:04 smithi107 ceph-mon[33178]: osdmap e365: 8 total, 8 up, 8 in 2024-06-11T13:36:05.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:04 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/101618264' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi057-77362-66"}]: dispatch 2024-06-11T13:36:05.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:04 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/101618264' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi057-77362-66"}]: dispatch 2024-06-11T13:36:05.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:04 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/101618264' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi057-77362-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:36:06.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:05 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/101618264' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi057-77362-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:36:06.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:05 smithi057 ceph-mon[29805]: osdmap e366: 8 total, 8 up, 8 in 2024-06-11T13:36:06.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:05 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/101618264' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi057-77362-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi057-77362-66"}]: dispatch 2024-06-11T13:36:06.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:05 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/101618264' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi057-77362-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:36:06.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:05 smithi057 ceph-mon[40588]: osdmap e366: 8 total, 8 up, 8 in 2024-06-11T13:36:06.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:05 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/101618264' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi057-77362-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi057-77362-66"}]: dispatch 2024-06-11T13:36:06.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:05 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/101618264' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi057-77362-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:36:06.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:05 smithi107 ceph-mon[33178]: osdmap e366: 8 total, 8 up, 8 in 2024-06-11T13:36:06.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:05 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/101618264' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi057-77362-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi057-77362-66"}]: dispatch 2024-06-11T13:36:07.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:06 smithi057 ceph-mon[29805]: pgmap v518: 324 pgs: 324 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:36:07.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:06 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:07.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:06 smithi057 ceph-mon[29805]: osdmap e367: 8 total, 8 up, 8 in 2024-06-11T13:36:07.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:06 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:07.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:06 smithi057 ceph-mon[40588]: pgmap v518: 324 pgs: 324 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:36:07.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:06 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:07.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:06 smithi057 ceph-mon[40588]: osdmap e367: 8 total, 8 up, 8 in 2024-06-11T13:36:07.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:06 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:07.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:06 smithi107 ceph-mon[33178]: pgmap v518: 324 pgs: 324 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:36:07.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:06 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:07.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:06 smithi107 ceph-mon[33178]: osdmap e367: 8 total, 8 up, 8 in 2024-06-11T13:36:07.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:06 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:08.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:07 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/101618264' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi057-77362-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi057-77362-66"}]': finished 2024-06-11T13:36:08.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:07 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:36:08.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:07 smithi057 ceph-mon[40588]: osdmap e368: 8 total, 8 up, 8 in 2024-06-11T13:36:08.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:07 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-49"}]: dispatch 2024-06-11T13:36:08.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:07 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-49"}]: dispatch 2024-06-11T13:36:08.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:07 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:36:08.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:07 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:36:08.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:07 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/101618264' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi057-77362-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi057-77362-66"}]': finished 2024-06-11T13:36:08.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:07 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:36:08.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:07 smithi057 ceph-mon[29805]: osdmap e368: 8 total, 8 up, 8 in 2024-06-11T13:36:08.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:07 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-49"}]: dispatch 2024-06-11T13:36:08.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:07 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-49"}]: dispatch 2024-06-11T13:36:08.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:07 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:36:08.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:07 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:36:08.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:07 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/101618264' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi057-77362-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi057-77362-66"}]': finished 2024-06-11T13:36:08.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:07 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:36:08.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:07 smithi107 ceph-mon[33178]: osdmap e368: 8 total, 8 up, 8 in 2024-06-11T13:36:08.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:07 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-49"}]: dispatch 2024-06-11T13:36:08.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:07 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-49"}]: dispatch 2024-06-11T13:36:08.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:07 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:36:08.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:07 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:36:08.707 INFO:tasks.workunit.client.0.smithi057.stdout: api_m::163:head 2024-06-11T13:36:08.707 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:5d165639:::164:head 2024-06-11T13:36:08.707 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:f43765fc:::165:head 2024-06-11T13:36:08.707 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:b4c720e9:::166:head 2024-06-11T13:36:08.707 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:e694b040:::167:head 2024-06-11T13:36:08.708 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:afa38db2:::168:head 2024-06-11T13:36:08.708 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:77ba9f53:::169:head 2024-06-11T13:36:08.708 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:87495034:::170:head 2024-06-11T13:36:08.708 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:7c96bf0e:::171:head 2024-06-11T13:36:08.708 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:dbe346cc:::172:head 2024-06-11T13:36:08.708 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:e943ec24:::173:head 2024-06-11T13:36:08.708 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:f97a9c0c:::174:head 2024-06-11T13:36:08.708 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:6f26e74d:::175:head 2024-06-11T13:36:08.708 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:4f95e106:::176:head 2024-06-11T13:36:08.709 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:0e6f2f8f:::177:head 2024-06-11T13:36:08.709 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:05db05f1:::178:head 2024-06-11T13:36:08.709 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:38a78d66:::179:head 2024-06-11T13:36:08.709 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:d095610b:::180:head 2024-06-11T13:36:08.709 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:a1a9d709:::181:head 2024-06-11T13:36:08.709 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:1e5d39db:::182:head 2024-06-11T13:36:08.709 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:f7df4fb9:::183:head 2024-06-11T13:36:08.709 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:03a7f161:::184:head 2024-06-11T13:36:08.709 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:ba70721e:::185:head 2024-06-11T13:36:08.709 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:28e5662d:::186:head 2024-06-11T13:36:08.710 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:973d52de:::187:head 2024-06-11T13:36:08.710 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:4303eb1c:::188:head 2024-06-11T13:36:08.710 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:b990b48e:::189:head 2024-06-11T13:36:08.710 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:29b8165b:::190:head 2024-06-11T13:36:08.710 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:3547f197:::191:head 2024-06-11T13:36:08.710 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:7e260936:::192:head 2024-06-11T13:36:08.710 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:1abec7b1:::193:head 2024-06-11T13:36:08.710 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:10fdda93:::194:head 2024-06-11T13:36:08.710 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:15817eea:::195:head 2024-06-11T13:36:08.710 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:770bab57:::196:head 2024-06-11T13:36:08.710 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:ed9e13e7:::197:head 2024-06-11T13:36:08.711 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:71471a8f:::198:head 2024-06-11T13:36:08.711 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: checking for 277:10fb1d02:::199:head 2024-06-11T13:36:08.711 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetWrite (8070 ms) 2024-06-11T13:36:08.711 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetTrim 2024-06-11T13:36:08.711 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: got ls 1718112924,0 2024-06-11T13:36:08.711 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: first is 1718112924 2024-06-11T13:36:08.711 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: got ls 1718112924,0 2024-06-11T13:36:08.711 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: got ls 1718112924,0 2024-06-11T13:36:08.711 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: got ls 1718112924,1718112926,1718112927,0 2024-06-11T13:36:08.712 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: got ls 1718112924,1718112926,1718112927,0 2024-06-11T13:36:08.712 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: got ls 1718112924,1718112926,1718112927,0 2024-06-11T13:36:08.712 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: got ls 1718112924,1718112926,1718112927,1718112929,1718112930,0 2024-06-11T13:36:08.712 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: got ls 1718112924,1718112926,1718112927,1718112929,1718112930,0 2024-06-11T13:36:08.712 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: got ls 1718112924,1718112926,1718112927,1718112929,1718112930,0 2024-06-11T13:36:08.712 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: got ls 1718112924,1718112926,1718112927,1718112929,1718112930,1718112932,1718112933,0 2024-06-11T13:36:08.712 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: got ls 1718112924,1718112926,1718112927,1718112929,1718112930,1718112932,1718112933,0 2024-06-11T13:36:08.712 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: got ls 1718112924,1718112926,1718112927,1718112929,1718112930,1718112932,1718112933,0 2024-06-11T13:36:08.712 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: got ls 1718112927,1718112929,1718112930,1718112932,1718112933,1718112935,1718112936,0 2024-06-11T13:36:08.712 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: first now 1718112927, trimmed 2024-06-11T13:36:08.712 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetTrim (20221 ms) 2024-06-11T13:36:08.712 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2024-06-11T13:36:08.713 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: foo0 2024-06-11T13:36:08.713 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-06-11T13:36:08.713 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteOn2ndRead (14136 ms) 2024-06-11T13:36:08.713 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ProxyRead 2024-06-11T13:36:09.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:08 smithi057 ceph-mon[40588]: pgmap v521: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:36:09.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:08 smithi057 ceph-mon[40588]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:36:09.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:08 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-49"}]': finished 2024-06-11T13:36:09.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:08 smithi057 ceph-mon[40588]: osdmap e369: 8 total, 8 up, 8 in 2024-06-11T13:36:09.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:08 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:09.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:08 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:09.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:08 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-49"}]: dispatch 2024-06-11T13:36:09.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:08 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-49"}]: dispatch 2024-06-11T13:36:09.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:08 smithi057 ceph-mon[40588]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:36:09.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:08 smithi057 ceph-mon[29805]: pgmap v521: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:36:09.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:08 smithi057 ceph-mon[29805]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:36:09.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:08 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-49"}]': finished 2024-06-11T13:36:09.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:08 smithi057 ceph-mon[29805]: osdmap e369: 8 total, 8 up, 8 in 2024-06-11T13:36:09.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:08 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:09.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:08 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:09.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:08 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-49"}]: dispatch 2024-06-11T13:36:09.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:08 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-49"}]: dispatch 2024-06-11T13:36:09.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:08 smithi057 ceph-mon[29805]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:36:09.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:08 smithi107 ceph-mon[33178]: pgmap v521: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:36:09.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:08 smithi107 ceph-mon[33178]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:36:09.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:08 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-49"}]': finished 2024-06-11T13:36:09.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:08 smithi107 ceph-mon[33178]: osdmap e369: 8 total, 8 up, 8 in 2024-06-11T13:36:09.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:08 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:09.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:08 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:09.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:08 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-49"}]: dispatch 2024-06-11T13:36:09.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:08 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-49"}]: dispatch 2024-06-11T13:36:09.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:08 smithi107 ceph-mon[33178]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:36:10.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:09 smithi057 ceph-mon[40588]: osdmap e370: 8 total, 8 up, 8 in 2024-06-11T13:36:10.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:09 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/101618264' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi057-77362-66"}]: dispatch 2024-06-11T13:36:10.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:09 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/101618264' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi057-77362-66"}]': finished 2024-06-11T13:36:10.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:09 smithi057 ceph-mon[40588]: osdmap e371: 8 total, 8 up, 8 in 2024-06-11T13:36:10.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:09 smithi057 ceph-mon[29805]: osdmap e370: 8 total, 8 up, 8 in 2024-06-11T13:36:10.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:09 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/101618264' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi057-77362-66"}]: dispatch 2024-06-11T13:36:10.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:09 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/101618264' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi057-77362-66"}]': finished 2024-06-11T13:36:10.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:09 smithi057 ceph-mon[29805]: osdmap e371: 8 total, 8 up, 8 in 2024-06-11T13:36:10.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:09 smithi107 ceph-mon[33178]: osdmap e370: 8 total, 8 up, 8 in 2024-06-11T13:36:10.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:09 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/101618264' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi057-77362-66"}]: dispatch 2024-06-11T13:36:10.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:09 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/101618264' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi057-77362-66"}]': finished 2024-06-11T13:36:10.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:09 smithi107 ceph-mon[33178]: osdmap e371: 8 total, 8 up, 8 in 2024-06-11T13:36:10.723 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ 2 2024-06-11T13:36:10.723 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP2 (3072 ms) 2024-06-11T13:36:10.723 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPP 2024-06-11T13:36:10.723 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPP (7066 ms) 2024-06-11T13:36:10.723 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPPNS 2024-06-11T13:36:10.723 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPPNS (7051 ms) 2024-06-11T13:36:10.723 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.StatRemovePP 2024-06-11T13:36:10.723 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAioEC.StatRemovePP (7050 ms) 2024-06-11T13:36:10.723 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ExecuteClassPP 2024-06-11T13:36:10.723 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ExecuteClassPP (7085 ms) 2024-06-11T13:36:10.723 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.OmapPP 2024-06-11T13:36:10.724 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAioEC.OmapPP (7069 ms) 2024-06-11T13:36:10.724 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.MultiWritePP 2024-06-11T13:36:10.724 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ OK ] LibRadosAioEC.MultiWritePP (7102 ms) 2024-06-11T13:36:10.724 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC (139964 ms total) 2024-06-11T13:36:10.724 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: 2024-06-11T13:36:10.724 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [----------] Global test environment tear-down 2024-06-11T13:36:10.724 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [==========] 57 tests from 4 test suites ran. (327159 ms total) 2024-06-11T13:36:10.724 INFO:tasks.workunit.client.0.smithi057.stdout: api_aio_pp: [ PASSED ] 57 tests. 2024-06-11T13:36:11.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:10 smithi057 ceph-mon[40588]: pgmap v524: 292 pgs: 292 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:36:11.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:10 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/101618264' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi057-77362-66"}]: dispatch 2024-06-11T13:36:11.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:10 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:36:11.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:10 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:36:11.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:10 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/101618264' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi057-77362-66"}]': finished 2024-06-11T13:36:11.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:10 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:36:11.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:10 smithi057 ceph-mon[40588]: osdmap e372: 8 total, 8 up, 8 in 2024-06-11T13:36:11.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:10 smithi057 ceph-mon[29805]: pgmap v524: 292 pgs: 292 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:36:11.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:10 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/101618264' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi057-77362-66"}]: dispatch 2024-06-11T13:36:11.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:10 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:36:11.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:10 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:36:11.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:10 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/101618264' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi057-77362-66"}]': finished 2024-06-11T13:36:11.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:10 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:36:11.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:10 smithi057 ceph-mon[29805]: osdmap e372: 8 total, 8 up, 8 in 2024-06-11T13:36:11.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:10 smithi107 ceph-mon[33178]: pgmap v524: 292 pgs: 292 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:36:11.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:10 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/101618264' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi057-77362-66"}]: dispatch 2024-06-11T13:36:11.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:10 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:36:11.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:10 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:36:11.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:10 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/101618264' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi057-77362-66"}]': finished 2024-06-11T13:36:11.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:10 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:36:11.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:10 smithi107 ceph-mon[33178]: osdmap e372: 8 total, 8 up, 8 in 2024-06-11T13:36:11.753 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:36:11 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:36:12.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:11 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:36:12.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:11 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:36:12.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:11 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:36:12.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:11 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:36:12.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:11 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:36:12.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:11 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:36:13.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:12 smithi057 ceph-mon[40588]: pgmap v527: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-06-11T13:36:13.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:12 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:36:13.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:12 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-51", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:36:13.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:12 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-51"}]: dispatch 2024-06-11T13:36:13.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:12 smithi057 ceph-mon[40588]: osdmap e373: 8 total, 8 up, 8 in 2024-06-11T13:36:13.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:12 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-51"}]: dispatch 2024-06-11T13:36:13.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:12 smithi057 ceph-mon[29805]: pgmap v527: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-06-11T13:36:13.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:12 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:36:13.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:12 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-51", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:36:13.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:12 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-51"}]: dispatch 2024-06-11T13:36:13.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:12 smithi057 ceph-mon[29805]: osdmap e373: 8 total, 8 up, 8 in 2024-06-11T13:36:13.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:12 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-51"}]: dispatch 2024-06-11T13:36:13.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:12 smithi107 ceph-mon[33178]: pgmap v527: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-06-11T13:36:13.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:12 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:36:13.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:12 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-51", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:36:13.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:12 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-51"}]: dispatch 2024-06-11T13:36:13.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:12 smithi107 ceph-mon[33178]: osdmap e373: 8 total, 8 up, 8 in 2024-06-11T13:36:13.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:12 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-51"}]: dispatch 2024-06-11T13:36:14.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:13 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-51"}]': finished 2024-06-11T13:36:14.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:13 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-51", "mode": "writeback"}]: dispatch 2024-06-11T13:36:14.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:13 smithi057 ceph-mon[29805]: osdmap e374: 8 total, 8 up, 8 in 2024-06-11T13:36:14.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:13 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-51", "mode": "writeback"}]: dispatch 2024-06-11T13:36:14.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:13 smithi057 ceph-mon[29805]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:36:14.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:13 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-51", "mode": "writeback"}]': finished 2024-06-11T13:36:14.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:13 smithi057 ceph-mon[29805]: osdmap e375: 8 total, 8 up, 8 in 2024-06-11T13:36:14.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:36:13 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:36:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:36:14.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:13 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-51"}]': finished 2024-06-11T13:36:14.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:13 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-51", "mode": "writeback"}]: dispatch 2024-06-11T13:36:14.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:13 smithi057 ceph-mon[40588]: osdmap e374: 8 total, 8 up, 8 in 2024-06-11T13:36:14.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:13 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-51", "mode": "writeback"}]: dispatch 2024-06-11T13:36:14.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:13 smithi057 ceph-mon[40588]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:36:14.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:13 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-51", "mode": "writeback"}]': finished 2024-06-11T13:36:14.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:13 smithi057 ceph-mon[40588]: osdmap e375: 8 total, 8 up, 8 in 2024-06-11T13:36:14.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:13 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-51"}]': finished 2024-06-11T13:36:14.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:13 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-51", "mode": "writeback"}]: dispatch 2024-06-11T13:36:14.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:13 smithi107 ceph-mon[33178]: osdmap e374: 8 total, 8 up, 8 in 2024-06-11T13:36:14.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:13 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-51", "mode": "writeback"}]: dispatch 2024-06-11T13:36:14.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:13 smithi107 ceph-mon[33178]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:36:14.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:13 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-51", "mode": "writeback"}]': finished 2024-06-11T13:36:14.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:13 smithi107 ceph-mon[33178]: osdmap e375: 8 total, 8 up, 8 in 2024-06-11T13:36:15.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:14 smithi057 ceph-mon[29805]: pgmap v530: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-06-11T13:36:15.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:14 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-51","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:36:15.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:14 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-51","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:36:15.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:14 smithi057 ceph-mon[40588]: pgmap v530: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-06-11T13:36:15.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:14 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-51","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:36:15.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:14 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-51","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:36:15.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:14 smithi107 ceph-mon[33178]: pgmap v530: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-06-11T13:36:15.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:14 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-51","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:36:15.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:14 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-51","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:36:16.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:15 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-51","var": "hit_set_count","val": "2"}]': finished 2024-06-11T13:36:16.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:15 smithi057 ceph-mon[29805]: osdmap e376: 8 total, 8 up, 8 in 2024-06-11T13:36:16.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:15 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-51","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:36:16.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:15 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-51","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:36:16.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:15 smithi057 ceph-mon[29805]: pgmap v533: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:36:16.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:15 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-51","var": "hit_set_period","val": "600"}]': finished 2024-06-11T13:36:16.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:15 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-51","var": "hit_set_count","val": "2"}]': finished 2024-06-11T13:36:16.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:15 smithi057 ceph-mon[40588]: osdmap e376: 8 total, 8 up, 8 in 2024-06-11T13:36:16.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:15 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-51","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:36:16.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:15 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-51","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:36:16.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:15 smithi057 ceph-mon[40588]: pgmap v533: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:36:16.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:15 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-51","var": "hit_set_period","val": "600"}]': finished 2024-06-11T13:36:16.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:15 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-51","var": "hit_set_count","val": "2"}]': finished 2024-06-11T13:36:16.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:15 smithi107 ceph-mon[33178]: osdmap e376: 8 total, 8 up, 8 in 2024-06-11T13:36:16.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:15 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-51","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:36:16.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:15 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-51","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:36:16.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:15 smithi107 ceph-mon[33178]: pgmap v533: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:36:16.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:15 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-51","var": "hit_set_period","val": "600"}]': finished 2024-06-11T13:36:17.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:16 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-11T13:36:17.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:16 smithi057 ceph-mon[29805]: osdmap e377: 8 total, 8 up, 8 in 2024-06-11T13:36:17.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:16 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-11T13:36:17.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:16 smithi057 ceph-mon[29805]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:36:17.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:16 smithi057 ceph-mon[29805]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:36:17.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:16 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-51","var": "hit_set_type","val": "bloom"}]': finished 2024-06-11T13:36:17.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:16 smithi057 ceph-mon[29805]: osdmap e378: 8 total, 8 up, 8 in 2024-06-11T13:36:17.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:16 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-11T13:36:17.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:16 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-11T13:36:17.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:16 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-11T13:36:17.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:16 smithi057 ceph-mon[40588]: osdmap e377: 8 total, 8 up, 8 in 2024-06-11T13:36:17.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:16 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-11T13:36:17.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:16 smithi057 ceph-mon[40588]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:36:17.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:16 smithi057 ceph-mon[40588]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:36:17.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:16 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-51","var": "hit_set_type","val": "bloom"}]': finished 2024-06-11T13:36:17.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:16 smithi057 ceph-mon[40588]: osdmap e378: 8 total, 8 up, 8 in 2024-06-11T13:36:17.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:16 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-11T13:36:17.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:16 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-11T13:36:17.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:16 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-11T13:36:17.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:16 smithi107 ceph-mon[33178]: osdmap e377: 8 total, 8 up, 8 in 2024-06-11T13:36:17.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:16 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-11T13:36:17.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:16 smithi107 ceph-mon[33178]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:36:17.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:16 smithi107 ceph-mon[33178]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:36:17.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:16 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-51","var": "hit_set_type","val": "bloom"}]': finished 2024-06-11T13:36:17.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:16 smithi107 ceph-mon[33178]: osdmap e378: 8 total, 8 up, 8 in 2024-06-11T13:36:17.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:16 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-11T13:36:17.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:16 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-11T13:36:18.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:17 smithi057 ceph-mon[29805]: pgmap v536: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-06-11T13:36:18.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:17 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-06-11T13:36:18.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:17 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-51","var": "target_max_objects","val": "1"}]: dispatch 2024-06-11T13:36:18.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:17 smithi057 ceph-mon[29805]: osdmap e379: 8 total, 8 up, 8 in 2024-06-11T13:36:18.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:17 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-51","var": "target_max_objects","val": "1"}]: dispatch 2024-06-11T13:36:18.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:17 smithi057 ceph-mon[40588]: pgmap v536: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-06-11T13:36:18.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:17 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-06-11T13:36:18.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:17 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-51","var": "target_max_objects","val": "1"}]: dispatch 2024-06-11T13:36:18.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:17 smithi057 ceph-mon[40588]: osdmap e379: 8 total, 8 up, 8 in 2024-06-11T13:36:18.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:17 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-51","var": "target_max_objects","val": "1"}]: dispatch 2024-06-11T13:36:18.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:17 smithi107 ceph-mon[33178]: pgmap v536: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-06-11T13:36:18.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:17 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-06-11T13:36:18.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:17 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-51","var": "target_max_objects","val": "1"}]: dispatch 2024-06-11T13:36:18.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:17 smithi107 ceph-mon[33178]: osdmap e379: 8 total, 8 up, 8 in 2024-06-11T13:36:18.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:17 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-51","var": "target_max_objects","val": "1"}]: dispatch 2024-06-11T13:36:20.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:19 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-51","var": "target_max_objects","val": "1"}]': finished 2024-06-11T13:36:20.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:19 smithi057 ceph-mon[29805]: osdmap e380: 8 total, 8 up, 8 in 2024-06-11T13:36:20.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:19 smithi057 ceph-mon[29805]: pgmap v539: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-06-11T13:36:20.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:19 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-51","var": "target_max_objects","val": "1"}]': finished 2024-06-11T13:36:20.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:19 smithi057 ceph-mon[40588]: osdmap e380: 8 total, 8 up, 8 in 2024-06-11T13:36:20.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:19 smithi057 ceph-mon[40588]: pgmap v539: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-06-11T13:36:20.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:19 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-51","var": "target_max_objects","val": "1"}]': finished 2024-06-11T13:36:20.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:19 smithi107 ceph-mon[33178]: osdmap e380: 8 total, 8 up, 8 in 2024-06-11T13:36:20.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:19 smithi107 ceph-mon[33178]: pgmap v539: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-06-11T13:36:21.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:20 smithi057 ceph-mon[29805]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-06-11T13:36:21.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:20 smithi057 ceph-mon[40588]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-06-11T13:36:21.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:20 smithi107 ceph-mon[33178]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-06-11T13:36:21.798 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:36:21 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:36:22.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:21 smithi057 ceph-mon[29805]: pgmap v540: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-06-11T13:36:22.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:21 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:36:22.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:21 smithi057 ceph-mon[40588]: pgmap v540: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-06-11T13:36:22.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:21 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:36:22.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:21 smithi107 ceph-mon[33178]: pgmap v540: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-06-11T13:36:22.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:21 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:36:23.186 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:22 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:36:23.186 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:22 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:36:23.187 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:22 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:36:23.187 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:22 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:36:23.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:22 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:36:23.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:22 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:36:24.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:23 smithi057 ceph-mon[29805]: pgmap v541: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 788 B/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:36:24.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:23 smithi057 ceph-mon[40588]: pgmap v541: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 788 B/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:36:24.035 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:36:23 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:36:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:36:24.297 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:23 smithi107 ceph-mon[33178]: pgmap v541: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 788 B/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:36:26.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:26 smithi057 ceph-mon[29805]: pgmap v542: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 0 op/s 2024-06-11T13:36:26.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:26 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:36:26.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:26 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:36:26.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:26 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:36:26.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:26 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:36:26.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:26 smithi057 ceph-mon[40588]: pgmap v542: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 0 op/s 2024-06-11T13:36:26.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:26 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:36:26.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:26 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:36:26.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:26 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:36:26.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:26 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:36:26.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:26 smithi107 ceph-mon[33178]: pgmap v542: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 0 op/s 2024-06-11T13:36:26.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:26 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:36:26.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:26 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:36:26.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:26 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:36:26.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:26 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:36:28.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:28 smithi057 ceph-mon[29805]: pgmap v543: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:36:28.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:28 smithi057 ceph-mon[40588]: pgmap v543: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:36:28.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:28 smithi107 ceph-mon[33178]: pgmap v543: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:36:29.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:29 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:29.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:29 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:29.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:29 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:29.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:29 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:29.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:29 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:29.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:29 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:30.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:30 smithi057 ceph-mon[29805]: pgmap v544: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 976 B/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:36:30.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:30 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:36:30.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:30 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-51"}]: dispatch 2024-06-11T13:36:30.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:30 smithi057 ceph-mon[29805]: osdmap e381: 8 total, 8 up, 8 in 2024-06-11T13:36:30.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:30 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-51"}]: dispatch 2024-06-11T13:36:30.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:30 smithi057 ceph-mon[40588]: pgmap v544: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 976 B/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:36:30.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:30 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:36:30.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:30 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-51"}]: dispatch 2024-06-11T13:36:30.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:30 smithi057 ceph-mon[40588]: osdmap e381: 8 total, 8 up, 8 in 2024-06-11T13:36:30.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:30 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-51"}]: dispatch 2024-06-11T13:36:30.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:30 smithi107 ceph-mon[33178]: pgmap v544: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 976 B/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:36:30.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:30 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:36:30.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:30 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-51"}]: dispatch 2024-06-11T13:36:30.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:30 smithi107 ceph-mon[33178]: osdmap e381: 8 total, 8 up, 8 in 2024-06-11T13:36:30.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:30 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-51"}]: dispatch 2024-06-11T13:36:31.699 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:36:31 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:36:31.700 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:31 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-51"}]': finished 2024-06-11T13:36:31.700 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:31 smithi107 ceph-mon[33178]: osdmap e382: 8 total, 8 up, 8 in 2024-06-11T13:36:31.700 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:31 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:31.700 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:31 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:31.700 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:31 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-51"}]: dispatch 2024-06-11T13:36:31.700 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:31 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-51"}]: dispatch 2024-06-11T13:36:31.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:31 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-51"}]': finished 2024-06-11T13:36:31.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:31 smithi057 ceph-mon[29805]: osdmap e382: 8 total, 8 up, 8 in 2024-06-11T13:36:31.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:31 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:31.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:31 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:31.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:31 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-51"}]: dispatch 2024-06-11T13:36:31.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:31 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-51"}]: dispatch 2024-06-11T13:36:31.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:31 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-51"}]': finished 2024-06-11T13:36:31.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:31 smithi057 ceph-mon[40588]: osdmap e382: 8 total, 8 up, 8 in 2024-06-11T13:36:31.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:31 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:31.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:31 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:31.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:31 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-51"}]: dispatch 2024-06-11T13:36:31.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:31 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-51"}]: dispatch 2024-06-11T13:36:32.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:32 smithi057 ceph-mon[29805]: pgmap v547: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:36:32.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:32 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:36:32.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:32 smithi057 ceph-mon[29805]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-06-11T13:36:32.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:32 smithi057 ceph-mon[29805]: osdmap e383: 8 total, 8 up, 8 in 2024-06-11T13:36:32.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:32 smithi057 ceph-mon[40588]: pgmap v547: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:36:32.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:32 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:36:32.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:32 smithi057 ceph-mon[40588]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-06-11T13:36:32.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:32 smithi057 ceph-mon[40588]: osdmap e383: 8 total, 8 up, 8 in 2024-06-11T13:36:32.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:32 smithi107 ceph-mon[33178]: pgmap v547: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:36:32.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:32 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:36:32.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:32 smithi107 ceph-mon[33178]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-06-11T13:36:32.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:32 smithi107 ceph-mon[33178]: osdmap e383: 8 total, 8 up, 8 in 2024-06-11T13:36:33.733 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:33 smithi057 ceph-mon[40588]: osdmap e384: 8 total, 8 up, 8 in 2024-06-11T13:36:33.733 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:33 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:36:33.733 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:33 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:36:33.733 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:33 smithi057 ceph-mon[29805]: osdmap e384: 8 total, 8 up, 8 in 2024-06-11T13:36:33.734 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:33 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:36:33.734 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:33 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:36:33.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:33 smithi107 ceph-mon[33178]: osdmap e384: 8 total, 8 up, 8 in 2024-06-11T13:36:33.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:33 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:36:33.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:33 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:36:34.034 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:36:33 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:36:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:36:35.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:34 smithi057 ceph-mon[29805]: pgmap v550: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-06-11T13:36:35.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:34 smithi057 ceph-mon[29805]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:36:35.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:34 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:36:35.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:34 smithi057 ceph-mon[29805]: osdmap e385: 8 total, 8 up, 8 in 2024-06-11T13:36:35.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:34 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:35.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:34 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:35.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:34 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-53"}]: dispatch 2024-06-11T13:36:35.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:34 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-53"}]: dispatch 2024-06-11T13:36:35.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:34 smithi057 ceph-mon[40588]: pgmap v550: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-06-11T13:36:35.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:34 smithi057 ceph-mon[40588]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:36:35.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:34 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:36:35.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:34 smithi057 ceph-mon[40588]: osdmap e385: 8 total, 8 up, 8 in 2024-06-11T13:36:35.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:34 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:35.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:34 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:35.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:34 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-53"}]: dispatch 2024-06-11T13:36:35.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:34 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-53"}]: dispatch 2024-06-11T13:36:35.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:34 smithi107 ceph-mon[33178]: pgmap v550: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-06-11T13:36:35.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:34 smithi107 ceph-mon[33178]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:36:35.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:34 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:36:35.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:34 smithi107 ceph-mon[33178]: osdmap e385: 8 total, 8 up, 8 in 2024-06-11T13:36:35.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:34 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:35.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:34 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:35.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:34 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-53"}]: dispatch 2024-06-11T13:36:35.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:34 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-53"}]: dispatch 2024-06-11T13:36:36.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:35 smithi057 ceph-mon[29805]: osdmap e386: 8 total, 8 up, 8 in 2024-06-11T13:36:36.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:35 smithi057 ceph-mon[29805]: pgmap v553: 292 pgs: 292 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:36:36.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:35 smithi057 ceph-mon[29805]: osdmap e387: 8 total, 8 up, 8 in 2024-06-11T13:36:36.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:35 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:36:36.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:35 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:36:36.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:35 smithi057 ceph-mon[40588]: osdmap e386: 8 total, 8 up, 8 in 2024-06-11T13:36:36.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:35 smithi057 ceph-mon[40588]: pgmap v553: 292 pgs: 292 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:36:36.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:35 smithi057 ceph-mon[40588]: osdmap e387: 8 total, 8 up, 8 in 2024-06-11T13:36:36.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:35 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:36:36.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:35 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:36:36.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:35 smithi107 ceph-mon[33178]: osdmap e386: 8 total, 8 up, 8 in 2024-06-11T13:36:36.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:35 smithi107 ceph-mon[33178]: pgmap v553: 292 pgs: 292 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:36:36.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:35 smithi107 ceph-mon[33178]: osdmap e387: 8 total, 8 up, 8 in 2024-06-11T13:36:36.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:35 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:36:36.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:35 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:36:37.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:37 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:36:37.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:37 smithi057 ceph-mon[29805]: osdmap e388: 8 total, 8 up, 8 in 2024-06-11T13:36:37.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:37 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:36:37.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:37 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:37.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:37 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:37.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:37 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-55"}]: dispatch 2024-06-11T13:36:37.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:37 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-55"}]: dispatch 2024-06-11T13:36:37.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:37 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:36:37.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:37 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:36:37.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:37 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:36:37.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:37 smithi057 ceph-mon[40588]: osdmap e388: 8 total, 8 up, 8 in 2024-06-11T13:36:37.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:37 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:36:37.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:37 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:37.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:37 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:37.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:37 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-55"}]: dispatch 2024-06-11T13:36:37.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:37 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-55"}]: dispatch 2024-06-11T13:36:37.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:37 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:36:37.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:37 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:36:37.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:37 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:36:37.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:37 smithi107 ceph-mon[33178]: osdmap e388: 8 total, 8 up, 8 in 2024-06-11T13:36:37.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:37 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:36:37.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:37 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:37.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:37 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:37.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:37 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-55"}]: dispatch 2024-06-11T13:36:37.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:37 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-55"}]: dispatch 2024-06-11T13:36:37.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:37 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:36:37.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:37 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:36:38.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:38 smithi057 ceph-mon[29805]: pgmap v556: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-11T13:36:38.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:38 smithi057 ceph-mon[29805]: osdmap e389: 8 total, 8 up, 8 in 2024-06-11T13:36:38.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:38 smithi057 ceph-mon[40588]: pgmap v556: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-11T13:36:38.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:38 smithi057 ceph-mon[40588]: osdmap e389: 8 total, 8 up, 8 in 2024-06-11T13:36:38.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:38 smithi107 ceph-mon[33178]: pgmap v556: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-11T13:36:38.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:38 smithi107 ceph-mon[33178]: osdmap e389: 8 total, 8 up, 8 in 2024-06-11T13:36:39.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:39 smithi107 ceph-mon[33178]: osdmap e390: 8 total, 8 up, 8 in 2024-06-11T13:36:39.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:39 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:36:39.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:39 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:36:40.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:39 smithi057 ceph-mon[40588]: osdmap e390: 8 total, 8 up, 8 in 2024-06-11T13:36:40.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:39 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:36:40.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:39 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:36:40.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:39 smithi057 ceph-mon[29805]: osdmap e390: 8 total, 8 up, 8 in 2024-06-11T13:36:40.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:39 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:36:40.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:39 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:36:40.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:40 smithi107 ceph-mon[33178]: pgmap v559: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-11T13:36:40.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:40 smithi107 ceph-mon[33178]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:36:40.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:40 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:36:40.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:40 smithi107 ceph-mon[33178]: osdmap e391: 8 total, 8 up, 8 in 2024-06-11T13:36:40.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:40 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:40.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:40 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:40.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:40 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-57"}]: dispatch 2024-06-11T13:36:40.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:40 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-57"}]: dispatch 2024-06-11T13:36:41.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:40 smithi057 ceph-mon[29805]: pgmap v559: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-11T13:36:41.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:40 smithi057 ceph-mon[29805]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:36:41.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:40 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:36:41.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:40 smithi057 ceph-mon[29805]: osdmap e391: 8 total, 8 up, 8 in 2024-06-11T13:36:41.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:40 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:41.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:40 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:41.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:40 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-57"}]: dispatch 2024-06-11T13:36:41.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:40 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-57"}]: dispatch 2024-06-11T13:36:41.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:40 smithi057 ceph-mon[40588]: pgmap v559: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-11T13:36:41.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:40 smithi057 ceph-mon[40588]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:36:41.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:40 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:36:41.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:40 smithi057 ceph-mon[40588]: osdmap e391: 8 total, 8 up, 8 in 2024-06-11T13:36:41.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:40 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:41.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:40 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:41.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:40 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-57"}]: dispatch 2024-06-11T13:36:41.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:40 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-57"}]: dispatch 2024-06-11T13:36:41.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:41 smithi107 ceph-mon[33178]: osdmap e392: 8 total, 8 up, 8 in 2024-06-11T13:36:41.798 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:36:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:36:42.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:41 smithi057 ceph-mon[29805]: osdmap e392: 8 total, 8 up, 8 in 2024-06-11T13:36:42.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:41 smithi057 ceph-mon[40588]: osdmap e392: 8 total, 8 up, 8 in 2024-06-11T13:36:43.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:42 smithi057 ceph-mon[40588]: pgmap v562: 292 pgs: 292 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-06-11T13:36:43.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:42 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:36:43.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:42 smithi057 ceph-mon[40588]: osdmap e393: 8 total, 8 up, 8 in 2024-06-11T13:36:43.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:42 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:36:43.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:42 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:36:43.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:42 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:36:43.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:42 smithi057 ceph-mon[40588]: osdmap e394: 8 total, 8 up, 8 in 2024-06-11T13:36:43.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:42 smithi057 ceph-mon[29805]: pgmap v562: 292 pgs: 292 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-06-11T13:36:43.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:42 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:36:43.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:42 smithi057 ceph-mon[29805]: osdmap e393: 8 total, 8 up, 8 in 2024-06-11T13:36:43.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:42 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:36:43.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:42 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:36:43.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:42 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:36:43.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:42 smithi057 ceph-mon[29805]: osdmap e394: 8 total, 8 up, 8 in 2024-06-11T13:36:43.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:42 smithi107 ceph-mon[33178]: pgmap v562: 292 pgs: 292 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-06-11T13:36:43.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:42 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:36:43.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:42 smithi107 ceph-mon[33178]: osdmap e393: 8 total, 8 up, 8 in 2024-06-11T13:36:43.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:42 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:36:43.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:42 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:36:43.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:42 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:36:43.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:42 smithi107 ceph-mon[33178]: osdmap e394: 8 total, 8 up, 8 in 2024-06-11T13:36:44.038 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:36:43 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:36:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:36:44.039 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:43 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:36:44.039 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:43 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:44.039 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:43 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:44.039 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:43 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-59"}]: dispatch 2024-06-11T13:36:44.039 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:43 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-59"}]: dispatch 2024-06-11T13:36:44.040 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:43 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:36:44.040 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:43 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:44.040 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:43 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:44.040 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:43 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-59"}]: dispatch 2024-06-11T13:36:44.040 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:43 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-59"}]: dispatch 2024-06-11T13:36:44.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:43 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:36:44.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:43 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:44.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:43 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:44.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:43 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-59"}]: dispatch 2024-06-11T13:36:44.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:43 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-59"}]: dispatch 2024-06-11T13:36:45.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:44 smithi057 ceph-mon[29805]: pgmap v565: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-06-11T13:36:45.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:44 smithi057 ceph-mon[29805]: osdmap e395: 8 total, 8 up, 8 in 2024-06-11T13:36:45.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:44 smithi057 ceph-mon[40588]: pgmap v565: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-06-11T13:36:45.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:44 smithi057 ceph-mon[40588]: osdmap e395: 8 total, 8 up, 8 in 2024-06-11T13:36:45.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:44 smithi107 ceph-mon[33178]: pgmap v565: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-06-11T13:36:45.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:44 smithi107 ceph-mon[33178]: osdmap e395: 8 total, 8 up, 8 in 2024-06-11T13:36:46.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:45 smithi057 ceph-mon[29805]: osdmap e396: 8 total, 8 up, 8 in 2024-06-11T13:36:46.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:45 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:36:46.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:45 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:36:46.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:45 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:36:46.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:45 smithi057 ceph-mon[29805]: osdmap e397: 8 total, 8 up, 8 in 2024-06-11T13:36:46.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:45 smithi057 ceph-mon[40588]: osdmap e396: 8 total, 8 up, 8 in 2024-06-11T13:36:46.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:45 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:36:46.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:45 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:36:46.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:45 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:36:46.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:45 smithi057 ceph-mon[40588]: osdmap e397: 8 total, 8 up, 8 in 2024-06-11T13:36:46.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:45 smithi107 ceph-mon[33178]: osdmap e396: 8 total, 8 up, 8 in 2024-06-11T13:36:46.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:45 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:36:46.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:45 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:36:46.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:45 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:36:46.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:45 smithi107 ceph-mon[33178]: osdmap e397: 8 total, 8 up, 8 in 2024-06-11T13:36:47.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:46 smithi057 ceph-mon[29805]: pgmap v568: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:36:47.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:46 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:36:47.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:46 smithi057 ceph-mon[29805]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:36:47.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:46 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:36:47.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:46 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:36:47.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:46 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-11T13:36:47.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:46 smithi057 ceph-mon[29805]: osdmap e398: 8 total, 8 up, 8 in 2024-06-11T13:36:47.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:46 smithi057 ceph-mon[40588]: pgmap v568: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:36:47.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:46 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:36:47.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:46 smithi057 ceph-mon[40588]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:36:47.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:46 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:36:47.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:46 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:36:47.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:46 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-11T13:36:47.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:46 smithi057 ceph-mon[40588]: osdmap e398: 8 total, 8 up, 8 in 2024-06-11T13:36:47.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:46 smithi107 ceph-mon[33178]: pgmap v568: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:36:47.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:46 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:36:47.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:46 smithi107 ceph-mon[33178]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:36:47.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:46 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:36:47.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:46 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:36:47.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:46 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-11T13:36:47.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:46 smithi107 ceph-mon[33178]: osdmap e398: 8 total, 8 up, 8 in 2024-06-11T13:36:48.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:47 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:48.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:47 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:48.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:47 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-61"}]: dispatch 2024-06-11T13:36:48.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:47 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-61"}]: dispatch 2024-06-11T13:36:48.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:47 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:48.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:47 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:48.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:47 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-61"}]: dispatch 2024-06-11T13:36:48.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:47 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-61"}]: dispatch 2024-06-11T13:36:48.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:47 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:48.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:47 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:36:48.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:47 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-61"}]: dispatch 2024-06-11T13:36:48.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:47 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-61"}]: dispatch 2024-06-11T13:36:49.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:48 smithi057 ceph-mon[29805]: pgmap v571: 324 pgs: 324 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-06-11T13:36:49.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:48 smithi057 ceph-mon[29805]: osdmap e399: 8 total, 8 up, 8 in 2024-06-11T13:36:49.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:48 smithi057 ceph-mon[40588]: pgmap v571: 324 pgs: 324 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-06-11T13:36:49.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:48 smithi057 ceph-mon[40588]: osdmap e399: 8 total, 8 up, 8 in 2024-06-11T13:36:49.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:48 smithi107 ceph-mon[33178]: pgmap v571: 324 pgs: 324 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-06-11T13:36:49.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:48 smithi107 ceph-mon[33178]: osdmap e399: 8 total, 8 up, 8 in 2024-06-11T13:36:50.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:49 smithi057 ceph-mon[29805]: osdmap e400: 8 total, 8 up, 8 in 2024-06-11T13:36:50.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:49 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:36:50.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:49 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:36:50.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:49 smithi057 ceph-mon[40588]: osdmap e400: 8 total, 8 up, 8 in 2024-06-11T13:36:50.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:49 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:36:50.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:49 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:36:50.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:49 smithi107 ceph-mon[33178]: osdmap e400: 8 total, 8 up, 8 in 2024-06-11T13:36:50.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:49 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:36:50.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:49 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:36:51.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:50 smithi057 ceph-mon[40588]: pgmap v574: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-06-11T13:36:51.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:50 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:36:51.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:50 smithi057 ceph-mon[40588]: osdmap e401: 8 total, 8 up, 8 in 2024-06-11T13:36:51.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:50 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:36:51.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:50 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:36:51.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:50 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:36:51.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:50 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-11T13:36:51.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:50 smithi057 ceph-mon[40588]: osdmap e402: 8 total, 8 up, 8 in 2024-06-11T13:36:51.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:50 smithi057 ceph-mon[29805]: pgmap v574: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-06-11T13:36:51.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:50 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:36:51.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:50 smithi057 ceph-mon[29805]: osdmap e401: 8 total, 8 up, 8 in 2024-06-11T13:36:51.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:50 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:36:51.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:50 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:36:51.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:50 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:36:51.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:50 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-11T13:36:51.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:50 smithi057 ceph-mon[29805]: osdmap e402: 8 total, 8 up, 8 in 2024-06-11T13:36:51.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:50 smithi107 ceph-mon[33178]: pgmap v574: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-06-11T13:36:51.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:50 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:36:51.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:50 smithi107 ceph-mon[33178]: osdmap e401: 8 total, 8 up, 8 in 2024-06-11T13:36:51.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:50 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:36:51.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:50 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:36:51.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:50 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:36:51.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:50 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-11T13:36:51.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:50 smithi107 ceph-mon[33178]: osdmap e402: 8 total, 8 up, 8 in 2024-06-11T13:36:51.735 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:36:51 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:36:52.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:51 smithi057 ceph-mon[40588]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:36:52.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:51 smithi057 ceph-mon[40588]: osdmap e403: 8 total, 8 up, 8 in 2024-06-11T13:36:52.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:51 smithi057 ceph-mon[29805]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:36:52.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:51 smithi057 ceph-mon[29805]: osdmap e403: 8 total, 8 up, 8 in 2024-06-11T13:36:52.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:51 smithi107 ceph-mon[33178]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:36:52.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:51 smithi107 ceph-mon[33178]: osdmap e403: 8 total, 8 up, 8 in 2024-06-11T13:36:53.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:52 smithi057 ceph-mon[40588]: pgmap v577: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-06-11T13:36:53.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:52 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:36:53.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:52 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:36:53.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:52 smithi057 ceph-mon[40588]: osdmap e404: 8 total, 8 up, 8 in 2024-06-11T13:36:53.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:52 smithi057 ceph-mon[29805]: pgmap v577: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-06-11T13:36:53.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:52 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:36:53.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:52 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:36:53.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:52 smithi057 ceph-mon[29805]: osdmap e404: 8 total, 8 up, 8 in 2024-06-11T13:36:53.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:52 smithi107 ceph-mon[33178]: pgmap v577: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-06-11T13:36:53.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:52 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:36:53.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:52 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:36:53.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:52 smithi107 ceph-mon[33178]: osdmap e404: 8 total, 8 up, 8 in 2024-06-11T13:36:54.034 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:36:53 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:36:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:36:55.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:54 smithi057 ceph-mon[29805]: pgmap v580: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-06-11T13:36:55.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:54 smithi057 ceph-mon[29805]: osdmap e405: 8 total, 8 up, 8 in 2024-06-11T13:36:55.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:54 smithi057 ceph-mon[40588]: pgmap v580: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-06-11T13:36:55.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:54 smithi057 ceph-mon[40588]: osdmap e405: 8 total, 8 up, 8 in 2024-06-11T13:36:55.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:54 smithi107 ceph-mon[33178]: pgmap v580: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-06-11T13:36:55.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:54 smithi107 ceph-mon[33178]: osdmap e405: 8 total, 8 up, 8 in 2024-06-11T13:36:56.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:55 smithi057 ceph-mon[29805]: osdmap e406: 8 total, 8 up, 8 in 2024-06-11T13:36:56.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:55 smithi057 ceph-mon[40588]: osdmap e406: 8 total, 8 up, 8 in 2024-06-11T13:36:56.047 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:55 smithi107 ceph-mon[33178]: osdmap e406: 8 total, 8 up, 8 in 2024-06-11T13:36:57.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:56 smithi057 ceph-mon[29805]: pgmap v583: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:36:57.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:56 smithi057 ceph-mon[40588]: pgmap v583: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:36:57.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:56 smithi107 ceph-mon[33178]: pgmap v583: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:36:59.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:36:58 smithi057 ceph-mon[29805]: pgmap v584: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-06-11T13:36:59.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:36:58 smithi057 ceph-mon[40588]: pgmap v584: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-06-11T13:36:59.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:36:58 smithi107 ceph-mon[33178]: pgmap v584: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-06-11T13:37:01.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:00 smithi057 ceph-mon[29805]: pgmap v585: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 928 B/s wr, 3 op/s 2024-06-11T13:37:01.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:00 smithi057 ceph-mon[40588]: pgmap v585: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 928 B/s wr, 3 op/s 2024-06-11T13:37:01.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:00 smithi107 ceph-mon[33178]: pgmap v585: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 928 B/s wr, 3 op/s 2024-06-11T13:37:01.798 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:37:01 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:37:02.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:02 smithi057 ceph-mon[29805]: osdmap e407: 8 total, 8 up, 8 in 2024-06-11T13:37:02.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:02 smithi057 ceph-mon[29805]: pgmap v587: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 806 B/s wr, 3 op/s 2024-06-11T13:37:02.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:02 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:37:02.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:02 smithi057 ceph-mon[40588]: osdmap e407: 8 total, 8 up, 8 in 2024-06-11T13:37:02.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:02 smithi057 ceph-mon[40588]: pgmap v587: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 806 B/s wr, 3 op/s 2024-06-11T13:37:02.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:02 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:37:02.547 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:02 smithi107 ceph-mon[33178]: osdmap e407: 8 total, 8 up, 8 in 2024-06-11T13:37:02.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:02 smithi107 ceph-mon[33178]: pgmap v587: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 806 B/s wr, 3 op/s 2024-06-11T13:37:02.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:02 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:37:04.034 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:37:03 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:37:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:37:04.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:04 smithi057 ceph-mon[40588]: pgmap v588: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 714 B/s wr, 3 op/s 2024-06-11T13:37:04.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:04 smithi057 ceph-mon[29805]: pgmap v588: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 714 B/s wr, 3 op/s 2024-06-11T13:37:04.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:04 smithi107 ceph-mon[33178]: pgmap v588: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 714 B/s wr, 3 op/s 2024-06-11T13:37:06.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:06 smithi057 ceph-mon[40588]: pgmap v589: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 614 B/s wr, 2 op/s 2024-06-11T13:37:06.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:06 smithi057 ceph-mon[40588]: osdmap e408: 8 total, 8 up, 8 in 2024-06-11T13:37:06.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:06 smithi057 ceph-mon[29805]: pgmap v589: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 614 B/s wr, 2 op/s 2024-06-11T13:37:06.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:06 smithi057 ceph-mon[29805]: osdmap e408: 8 total, 8 up, 8 in 2024-06-11T13:37:06.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:06 smithi107 ceph-mon[33178]: pgmap v589: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 614 B/s wr, 2 op/s 2024-06-11T13:37:06.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:06 smithi107 ceph-mon[33178]: osdmap e408: 8 total, 8 up, 8 in 2024-06-11T13:37:07.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:07 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:37:07.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:07 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:37:07.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:07 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:37:08.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:08 smithi057 ceph-mon[29805]: pgmap v591: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:37:08.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:08 smithi057 ceph-mon[40588]: pgmap v591: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:37:08.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:08 smithi107 ceph-mon[33178]: pgmap v591: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:37:10.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:10 smithi057 ceph-mon[29805]: pgmap v592: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 996 B/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:37:10.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:10 smithi057 ceph-mon[40588]: pgmap v592: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 996 B/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:37:10.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:10 smithi107 ceph-mon[33178]: pgmap v592: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 996 B/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:37:11.798 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:37:11 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:37:12.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:12 smithi057 ceph-mon[40588]: osdmap e409: 8 total, 8 up, 8 in 2024-06-11T13:37:12.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:12 smithi057 ceph-mon[40588]: pgmap v594: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:37:12.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:12 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:37:12.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:12 smithi057 ceph-mon[29805]: osdmap e409: 8 total, 8 up, 8 in 2024-06-11T13:37:12.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:12 smithi057 ceph-mon[29805]: pgmap v594: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:37:12.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:12 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:37:12.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:12 smithi107 ceph-mon[33178]: osdmap e409: 8 total, 8 up, 8 in 2024-06-11T13:37:12.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:12 smithi107 ceph-mon[33178]: pgmap v594: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:37:12.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:12 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:37:14.035 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:37:13 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:37:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:37:14.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:14 smithi057 ceph-mon[29805]: pgmap v595: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:37:14.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:14 smithi057 ceph-mon[40588]: pgmap v595: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:37:14.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:14 smithi107 ceph-mon[33178]: pgmap v595: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:37:16.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:16 smithi057 ceph-mon[29805]: pgmap v596: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:37:16.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:16 smithi057 ceph-mon[40588]: pgmap v596: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:37:16.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:16 smithi107 ceph-mon[33178]: pgmap v596: 324 pgs: 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:37:17.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:17 smithi057 ceph-mon[29805]: osdmap e410: 8 total, 8 up, 8 in 2024-06-11T13:37:17.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:17 smithi057 ceph-mon[40588]: osdmap e410: 8 total, 8 up, 8 in 2024-06-11T13:37:17.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:17 smithi107 ceph-mon[33178]: osdmap e410: 8 total, 8 up, 8 in 2024-06-11T13:37:18.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:18 smithi057 ceph-mon[29805]: pgmap v598: 324 pgs: 324 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-06-11T13:37:18.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:18 smithi057 ceph-mon[40588]: pgmap v598: 324 pgs: 324 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-06-11T13:37:18.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:18 smithi107 ceph-mon[33178]: pgmap v598: 324 pgs: 324 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-06-11T13:37:20.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:20 smithi057 ceph-mon[29805]: pgmap v599: 324 pgs: 324 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 748 B/s rd, 0 op/s 2024-06-11T13:37:20.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:20 smithi057 ceph-mon[40588]: pgmap v599: 324 pgs: 324 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 748 B/s rd, 0 op/s 2024-06-11T13:37:20.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:20 smithi107 ceph-mon[33178]: pgmap v599: 324 pgs: 324 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 748 B/s rd, 0 op/s 2024-06-11T13:37:21.788 INFO:tasks.workunit.client.0.smithi057.stdout:isc: started 0x564078a82670 2024-06-11T13:37:21.788 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a81bc0 2024-06-11T13:37:21.788 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a81d10 2024-06-11T13:37:21.789 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a821c0 2024-06-11T13:37:21.789 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a82310 2024-06-11T13:37:21.789 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a82440 2024-06-11T13:37:21.789 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a827c0 2024-06-11T13:37:21.789 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a82c90 2024-06-11T13:37:21.789 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a82dc0 2024-06-11T13:37:21.789 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a81840 2024-06-11T13:37:21.789 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a82ef0 2024-06-11T13:37:21.789 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a83040 2024-06-11T13:37:21.790 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a82550 2024-06-11T13:37:21.790 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a83170 2024-06-11T13:37:21.790 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a832e0 2024-06-11T13:37:21.790 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a80630 2024-06-11T13:37:21.790 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a80760 2024-06-11T13:37:21.790 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a80890 2024-06-11T13:37:21.790 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a83430 2024-06-11T13:37:21.790 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a83920 2024-06-11T13:37:21.790 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a83a50 2024-06-11T13:37:21.790 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a84050 2024-06-11T13:37:21.790 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a83580 2024-06-11T13:37:21.790 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a836b0 2024-06-11T13:37:21.791 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a837e0 2024-06-11T13:37:21.791 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a841a0 2024-06-11T13:37:21.791 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a83ba0 2024-06-11T13:37:21.791 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a83cf0 2024-06-11T13:37:21.791 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a83e20 2024-06-11T13:37:21.791 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a842d0 2024-06-11T13:37:21.791 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a847f0 2024-06-11T13:37:21.791 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a84920 2024-06-11T13:37:21.791 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a828f0 2024-06-11T13:37:21.791 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a82a20 2024-06-11T13:37:21.791 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a82b50 2024-06-11T13:37:21.792 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a84a50 2024-06-11T13:37:21.792 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a84f20 2024-06-11T13:37:21.792 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a85050 2024-06-11T13:37:21.792 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a85650 2024-06-11T13:37:21.792 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a84b80 2024-06-11T13:37:21.792 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a84cd0 2024-06-11T13:37:21.792 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a84e00 2024-06-11T13:37:21.792 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a85760 2024-06-11T13:37:21.792 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a851a0 2024-06-11T13:37:21.792 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a852d0 2024-06-11T13:37:21.792 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a85400 2024-06-11T13:37:21.792 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a85530 2024-06-11T13:37:21.792 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a85d60 2024-06-11T13:37:21.793 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a858b0 2024-06-11T13:37:21.793 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a859e0 2024-06-11T13:37:21.793 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a85b30 2024-06-11T13:37:21.793 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a85e70 2024-06-11T13:37:21.794 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a86320 2024-06-11T13:37:21.794 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a86470 2024-06-11T13:37:21.794 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a84400 2024-06-11T13:37:21.794 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a84530 2024-06-11T13:37:21.794 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a84660 2024-06-11T13:37:21.794 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a865a0 2024-06-11T13:37:21.794 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a866f0 2024-06-11T13:37:21.794 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a86cf0 2024-06-11T13:37:21.794 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a86840 2024-06-11T13:37:21.795 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a86970 2024-06-11T13:37:21.795 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a86aa0 2024-06-11T13:37:21.795 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a86bd0 2024-06-11T13:37:21.795 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a872f0 2024-06-11T13:37:21.795 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a86e40 2024-06-11T13:37:21.795 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a86f70 2024-06-11T13:37:21.795 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a870a0 2024-06-11T13:37:21.795 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a871d0 2024-06-11T13:37:21.795 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a878d0 2024-06-11T13:37:21.795 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a87420 2024-06-11T13:37:21.795 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a87570 2024-06-11T13:37:21.796 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a876c0 2024-06-11T13:37:21.796 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a87a00 2024-06-11T13:37:21.796 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a87eb0 2024-06-11T13:37:21.796 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a87fe0 2024-06-11T13:37:21.796 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a85f80 2024-06-11T13:37:21.796 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a860b0 2024-06-11T13:37:21.796 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a861e0 2024-06-11T13:37:21.796 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a88110 2024-06-11T13:37:21.796 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a885e0 2024-06-11T13:37:21.796 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a88730 2024-06-11T13:37:21.798 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:37:21 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:37:22.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:22 smithi057 ceph-mon[40588]: pgmap v600: 324 pgs: 324 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-06-11T13:37:22.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:22 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:37:22.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:22 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:37:22.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:22 smithi057 ceph-mon[29805]: pgmap v600: 324 pgs: 324 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-06-11T13:37:22.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:22 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:37:22.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:22 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:37:22.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:22 smithi107 ceph-mon[33178]: pgmap v600: 324 pgs: 324 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-06-11T13:37:22.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:22 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:37:22.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:22 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:37:24.035 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:37:23 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:37:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:37:24.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:24 smithi057 ceph-mon[40588]: pgmap v601: 324 pgs: 324 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-06-11T13:37:24.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:24 smithi057 ceph-mon[29805]: pgmap v601: 324 pgs: 324 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-06-11T13:37:24.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:24 smithi107 ceph-mon[33178]: pgmap v601: 324 pgs: 324 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-06-11T13:37:26.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:26 smithi057 ceph-mon[40588]: pgmap v602: 324 pgs: 324 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-06-11T13:37:26.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:26 smithi057 ceph-mon[40588]: osdmap e411: 8 total, 8 up, 8 in 2024-06-11T13:37:26.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:26 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:37:26.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:26 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:37:26.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:26 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:37:26.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:26 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-63"}]: dispatch 2024-06-11T13:37:26.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:26 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-63"}]: dispatch 2024-06-11T13:37:26.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:26 smithi057 ceph-mon[29805]: pgmap v602: 324 pgs: 324 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-06-11T13:37:26.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:26 smithi057 ceph-mon[29805]: osdmap e411: 8 total, 8 up, 8 in 2024-06-11T13:37:26.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:26 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:37:26.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:26 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:37:26.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:26 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:37:26.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:26 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-63"}]: dispatch 2024-06-11T13:37:26.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:26 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-63"}]: dispatch 2024-06-11T13:37:26.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:26 smithi107 ceph-mon[33178]: pgmap v602: 324 pgs: 324 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-06-11T13:37:26.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:26 smithi107 ceph-mon[33178]: osdmap e411: 8 total, 8 up, 8 in 2024-06-11T13:37:26.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:26 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:37:26.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:26 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:37:26.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:26 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:37:26.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:26 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-63"}]: dispatch 2024-06-11T13:37:26.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:26 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-63"}]: dispatch 2024-06-11T13:37:28.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:27 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:37:28.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:27 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:37:28.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:27 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:37:28.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:27 smithi057 ceph-mon[29805]: osdmap e412: 8 total, 8 up, 8 in 2024-06-11T13:37:28.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:27 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:37:28.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:27 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:37:28.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:27 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:37:28.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:27 smithi057 ceph-mon[40588]: osdmap e412: 8 total, 8 up, 8 in 2024-06-11T13:37:28.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:27 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:37:28.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:27 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:37:28.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:27 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:37:28.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:27 smithi107 ceph-mon[33178]: osdmap e412: 8 total, 8 up, 8 in 2024-06-11T13:37:29.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:28 smithi057 ceph-mon[29805]: pgmap v605: 292 pgs: 292 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:37:29.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:28 smithi057 ceph-mon[29805]: osdmap e413: 8 total, 8 up, 8 in 2024-06-11T13:37:29.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:28 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:37:29.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:28 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:37:29.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:28 smithi057 ceph-mon[40588]: pgmap v605: 292 pgs: 292 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:37:29.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:28 smithi057 ceph-mon[40588]: osdmap e413: 8 total, 8 up, 8 in 2024-06-11T13:37:29.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:28 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:37:29.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:28 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:37:29.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:28 smithi107 ceph-mon[33178]: pgmap v605: 292 pgs: 292 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:37:29.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:28 smithi107 ceph-mon[33178]: osdmap e413: 8 total, 8 up, 8 in 2024-06-11T13:37:29.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:28 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:37:29.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:28 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:37:30.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:30 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:37:30.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:30 smithi057 ceph-mon[40588]: osdmap e414: 8 total, 8 up, 8 in 2024-06-11T13:37:30.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:30 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:37:30.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:30 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:37:30.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:30 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:37:30.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:30 smithi057 ceph-mon[40588]: pgmap v608: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:37:30.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:30 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:37:30.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:30 smithi057 ceph-mon[29805]: osdmap e414: 8 total, 8 up, 8 in 2024-06-11T13:37:30.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:30 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:37:30.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:30 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:37:30.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:30 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:37:30.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:30 smithi057 ceph-mon[29805]: pgmap v608: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:37:30.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:30 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:37:30.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:30 smithi107 ceph-mon[33178]: osdmap e414: 8 total, 8 up, 8 in 2024-06-11T13:37:30.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:30 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:37:30.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:30 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:37:30.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:30 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:37:30.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:30 smithi107 ceph-mon[33178]: pgmap v608: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:37:31.449 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:31 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-11T13:37:31.449 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:31 smithi107 ceph-mon[33178]: osdmap e415: 8 total, 8 up, 8 in 2024-06-11T13:37:31.534 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:31 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-11T13:37:31.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:31 smithi057 ceph-mon[40588]: osdmap e415: 8 total, 8 up, 8 in 2024-06-11T13:37:31.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:31 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-11T13:37:31.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:31 smithi057 ceph-mon[29805]: osdmap e415: 8 total, 8 up, 8 in 2024-06-11T13:37:31.798 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:37:31 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:37:32.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:32 smithi057 ceph-mon[29805]: osdmap e416: 8 total, 8 up, 8 in 2024-06-11T13:37:32.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:32 smithi057 ceph-mon[29805]: pgmap v611: 324 pgs: 324 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-06-11T13:37:32.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:32 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:37:32.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:32 smithi057 ceph-mon[40588]: osdmap e416: 8 total, 8 up, 8 in 2024-06-11T13:37:32.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:32 smithi057 ceph-mon[40588]: pgmap v611: 324 pgs: 324 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-06-11T13:37:32.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:32 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:37:32.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:32 smithi107 ceph-mon[33178]: osdmap e416: 8 total, 8 up, 8 in 2024-06-11T13:37:32.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:32 smithi107 ceph-mon[33178]: pgmap v611: 324 pgs: 324 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-06-11T13:37:32.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:32 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:37:33.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:33 smithi057 ceph-mon[29805]: osdmap e417: 8 total, 8 up, 8 in 2024-06-11T13:37:33.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:33 smithi057 ceph-mon[40588]: osdmap e417: 8 total, 8 up, 8 in 2024-06-11T13:37:33.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:33 smithi107 ceph-mon[33178]: osdmap e417: 8 total, 8 up, 8 in 2024-06-11T13:37:34.034 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:37:33 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:37:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:37:34.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:34 smithi057 ceph-mon[29805]: osdmap e418: 8 total, 8 up, 8 in 2024-06-11T13:37:34.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:34 smithi057 ceph-mon[29805]: pgmap v614: 324 pgs: 324 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-06-11T13:37:34.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:34 smithi057 ceph-mon[40588]: osdmap e418: 8 total, 8 up, 8 in 2024-06-11T13:37:34.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:34 smithi057 ceph-mon[40588]: pgmap v614: 324 pgs: 324 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-06-11T13:37:34.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:34 smithi107 ceph-mon[33178]: osdmap e418: 8 total, 8 up, 8 in 2024-06-11T13:37:34.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:34 smithi107 ceph-mon[33178]: pgmap v614: 324 pgs: 324 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-06-11T13:37:36.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:36 smithi057 ceph-mon[29805]: pgmap v615: 324 pgs: 324 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 987 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-06-11T13:37:36.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:36 smithi057 ceph-mon[40588]: pgmap v615: 324 pgs: 324 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 987 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-06-11T13:37:36.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:36 smithi107 ceph-mon[33178]: pgmap v615: 324 pgs: 324 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 987 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-06-11T13:37:37.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:37 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:37:37.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:37 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:37:37.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:37 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:37:38.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:38 smithi057 ceph-mon[29805]: pgmap v616: 324 pgs: 324 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 2.4 KiB/s rd, 663 B/s wr, 4 op/s 2024-06-11T13:37:38.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:38 smithi057 ceph-mon[40588]: pgmap v616: 324 pgs: 324 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 2.4 KiB/s rd, 663 B/s wr, 4 op/s 2024-06-11T13:37:38.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:38 smithi107 ceph-mon[33178]: pgmap v616: 324 pgs: 324 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 2.4 KiB/s rd, 663 B/s wr, 4 op/s 2024-06-11T13:37:40.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:40 smithi057 ceph-mon[29805]: pgmap v617: 324 pgs: 324 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 511 B/s wr, 3 op/s 2024-06-11T13:37:40.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:40 smithi057 ceph-mon[40588]: pgmap v617: 324 pgs: 324 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 511 B/s wr, 3 op/s 2024-06-11T13:37:40.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:40 smithi107 ceph-mon[33178]: pgmap v617: 324 pgs: 324 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 511 B/s wr, 3 op/s 2024-06-11T13:37:41.798 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:37:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:37:42.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:42 smithi057 ceph-mon[29805]: osdmap e419: 8 total, 8 up, 8 in 2024-06-11T13:37:42.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:42 smithi057 ceph-mon[29805]: pgmap v619: 324 pgs: 324 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 503 B/s wr, 4 op/s 2024-06-11T13:37:42.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:42 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:37:42.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:42 smithi057 ceph-mon[40588]: osdmap e419: 8 total, 8 up, 8 in 2024-06-11T13:37:42.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:42 smithi057 ceph-mon[40588]: pgmap v619: 324 pgs: 324 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 503 B/s wr, 4 op/s 2024-06-11T13:37:42.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:42 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:37:42.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:42 smithi107 ceph-mon[33178]: osdmap e419: 8 total, 8 up, 8 in 2024-06-11T13:37:42.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:42 smithi107 ceph-mon[33178]: pgmap v619: 324 pgs: 324 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 503 B/s wr, 4 op/s 2024-06-11T13:37:42.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:42 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:37:43.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:43 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:37:43.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:43 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:37:43.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:43 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-65"}]: dispatch 2024-06-11T13:37:43.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:43 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-65"}]: dispatch 2024-06-11T13:37:43.733 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:43 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:37:43.733 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:43 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:37:43.734 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:43 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-65"}]: dispatch 2024-06-11T13:37:43.734 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:43 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-65"}]: dispatch 2024-06-11T13:37:43.734 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:43 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:37:43.734 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:43 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:37:43.735 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:43 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-65"}]: dispatch 2024-06-11T13:37:43.735 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:43 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-65"}]: dispatch 2024-06-11T13:37:44.035 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:37:43 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:37:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:37:44.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:44 smithi057 ceph-mon[29805]: osdmap e420: 8 total, 8 up, 8 in 2024-06-11T13:37:44.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:44 smithi057 ceph-mon[29805]: pgmap v621: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-06-11T13:37:44.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:44 smithi057 ceph-mon[40588]: osdmap e420: 8 total, 8 up, 8 in 2024-06-11T13:37:44.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:44 smithi057 ceph-mon[40588]: pgmap v621: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-06-11T13:37:44.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:44 smithi107 ceph-mon[33178]: osdmap e420: 8 total, 8 up, 8 in 2024-06-11T13:37:44.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:44 smithi107 ceph-mon[33178]: pgmap v621: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-06-11T13:37:45.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:45 smithi057 ceph-mon[29805]: osdmap e421: 8 total, 8 up, 8 in 2024-06-11T13:37:45.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:45 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:37:45.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:45 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:37:45.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:45 smithi057 ceph-mon[40588]: osdmap e421: 8 total, 8 up, 8 in 2024-06-11T13:37:45.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:45 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:37:45.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:45 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:37:45.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:45 smithi107 ceph-mon[33178]: osdmap e421: 8 total, 8 up, 8 in 2024-06-11T13:37:45.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:45 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:37:45.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:45 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:37:46.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:46 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:37:46.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:46 smithi057 ceph-mon[29805]: osdmap e422: 8 total, 8 up, 8 in 2024-06-11T13:37:46.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:46 smithi057 ceph-mon[29805]: pgmap v623: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:37:46.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:46 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:37:46.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:46 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:37:46.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:46 smithi057 ceph-mon[40588]: osdmap e422: 8 total, 8 up, 8 in 2024-06-11T13:37:46.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:46 smithi057 ceph-mon[40588]: pgmap v623: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:37:46.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:46 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:37:46.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:46 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:37:46.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:46 smithi107 ceph-mon[33178]: osdmap e422: 8 total, 8 up, 8 in 2024-06-11T13:37:46.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:46 smithi107 ceph-mon[33178]: pgmap v623: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:37:46.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:46 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:37:47.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:47 smithi057 ceph-mon[29805]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:37:47.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:47 smithi057 ceph-mon[29805]: osdmap e423: 8 total, 8 up, 8 in 2024-06-11T13:37:47.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:47 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:37:47.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:47 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:37:47.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:47 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-67"}]: dispatch 2024-06-11T13:37:47.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:47 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-67"}]: dispatch 2024-06-11T13:37:47.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:47 smithi057 ceph-mon[40588]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:37:47.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:47 smithi057 ceph-mon[40588]: osdmap e423: 8 total, 8 up, 8 in 2024-06-11T13:37:47.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:47 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:37:47.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:47 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:37:47.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:47 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-67"}]: dispatch 2024-06-11T13:37:47.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:47 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-67"}]: dispatch 2024-06-11T13:37:47.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:47 smithi107 ceph-mon[33178]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:37:47.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:47 smithi107 ceph-mon[33178]: osdmap e423: 8 total, 8 up, 8 in 2024-06-11T13:37:47.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:47 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:37:47.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:47 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:37:47.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:47 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-67"}]: dispatch 2024-06-11T13:37:47.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:47 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-67"}]: dispatch 2024-06-11T13:37:48.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:48 smithi057 ceph-mon[29805]: pgmap v626: 324 pgs: 324 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-06-11T13:37:48.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:48 smithi057 ceph-mon[29805]: osdmap e424: 8 total, 8 up, 8 in 2024-06-11T13:37:48.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:48 smithi057 ceph-mon[40588]: pgmap v626: 324 pgs: 324 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-06-11T13:37:48.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:48 smithi057 ceph-mon[40588]: osdmap e424: 8 total, 8 up, 8 in 2024-06-11T13:37:48.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:48 smithi107 ceph-mon[33178]: pgmap v626: 324 pgs: 324 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-06-11T13:37:48.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:48 smithi107 ceph-mon[33178]: osdmap e424: 8 total, 8 up, 8 in 2024-06-11T13:37:49.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:49 smithi057 ceph-mon[29805]: osdmap e425: 8 total, 8 up, 8 in 2024-06-11T13:37:49.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:49 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:37:49.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:49 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:37:49.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:49 smithi057 ceph-mon[40588]: osdmap e425: 8 total, 8 up, 8 in 2024-06-11T13:37:49.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:49 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:37:49.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:49 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:37:49.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:49 smithi107 ceph-mon[33178]: osdmap e425: 8 total, 8 up, 8 in 2024-06-11T13:37:49.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:49 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:37:49.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:49 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:37:50.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:50 smithi057 ceph-mon[40588]: pgmap v629: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:37:50.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:50 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:37:50.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:50 smithi057 ceph-mon[40588]: osdmap e426: 8 total, 8 up, 8 in 2024-06-11T13:37:50.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:50 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:37:50.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:50 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:37:50.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:50 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:37:50.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:50 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-69"}]: dispatch 2024-06-11T13:37:50.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:50 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-69"}]: dispatch 2024-06-11T13:37:50.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:50 smithi057 ceph-mon[29805]: pgmap v629: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:37:50.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:50 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:37:50.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:50 smithi057 ceph-mon[29805]: osdmap e426: 8 total, 8 up, 8 in 2024-06-11T13:37:50.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:50 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:37:50.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:50 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:37:50.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:50 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:37:50.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:50 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-69"}]: dispatch 2024-06-11T13:37:50.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:50 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-69"}]: dispatch 2024-06-11T13:37:50.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:50 smithi107 ceph-mon[33178]: pgmap v629: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:37:50.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:50 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:37:50.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:50 smithi107 ceph-mon[33178]: osdmap e426: 8 total, 8 up, 8 in 2024-06-11T13:37:50.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:50 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:37:50.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:50 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:37:50.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:50 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:37:50.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:50 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-69"}]: dispatch 2024-06-11T13:37:50.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:50 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-69"}]: dispatch 2024-06-11T13:37:51.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:51 smithi057 ceph-mon[40588]: osdmap e427: 8 total, 8 up, 8 in 2024-06-11T13:37:51.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:51 smithi057 ceph-mon[29805]: osdmap e427: 8 total, 8 up, 8 in 2024-06-11T13:37:51.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:51 smithi107 ceph-mon[33178]: osdmap e427: 8 total, 8 up, 8 in 2024-06-11T13:37:51.798 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:37:51 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:37:52.784 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:52 smithi057 ceph-mon[40588]: pgmap v632: 292 pgs: 292 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-06-11T13:37:52.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:52 smithi057 ceph-mon[40588]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:37:52.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:52 smithi057 ceph-mon[40588]: osdmap e428: 8 total, 8 up, 8 in 2024-06-11T13:37:52.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:52 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:37:52.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:52 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:37:52.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:52 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:37:52.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:52 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:37:52.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:52 smithi057 ceph-mon[29805]: pgmap v632: 292 pgs: 292 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-06-11T13:37:52.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:52 smithi057 ceph-mon[29805]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:37:52.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:52 smithi057 ceph-mon[29805]: osdmap e428: 8 total, 8 up, 8 in 2024-06-11T13:37:52.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:52 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:37:52.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:52 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:37:52.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:52 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:37:52.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:52 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:37:52.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:52 smithi107 ceph-mon[33178]: pgmap v632: 292 pgs: 292 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-06-11T13:37:52.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:52 smithi107 ceph-mon[33178]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:37:52.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:52 smithi107 ceph-mon[33178]: osdmap e428: 8 total, 8 up, 8 in 2024-06-11T13:37:52.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:52 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:37:52.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:52 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:37:52.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:52 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:37:52.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:52 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:37:53.733 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:53 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:37:53.733 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:53 smithi057 ceph-mon[40588]: osdmap e429: 8 total, 8 up, 8 in 2024-06-11T13:37:53.733 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:53 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:37:53.733 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:53 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:37:53.733 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:53 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:37:53.734 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:53 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:37:53.734 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:53 smithi057 ceph-mon[29805]: osdmap e429: 8 total, 8 up, 8 in 2024-06-11T13:37:53.734 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:53 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:37:53.734 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:53 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:37:53.734 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:53 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:37:53.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:53 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:37:53.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:53 smithi107 ceph-mon[33178]: osdmap e429: 8 total, 8 up, 8 in 2024-06-11T13:37:53.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:53 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:37:53.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:53 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:37:53.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:53 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:37:54.034 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:37:53 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:37:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:37:54.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:54 smithi057 ceph-mon[40588]: pgmap v635: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 2 op/s 2024-06-11T13:37:54.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:54 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-11T13:37:54.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:54 smithi057 ceph-mon[40588]: osdmap e430: 8 total, 8 up, 8 in 2024-06-11T13:37:54.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:54 smithi057 ceph-mon[29805]: pgmap v635: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 2 op/s 2024-06-11T13:37:54.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:54 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-11T13:37:54.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:54 smithi057 ceph-mon[29805]: osdmap e430: 8 total, 8 up, 8 in 2024-06-11T13:37:54.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:54 smithi107 ceph-mon[33178]: pgmap v635: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 2 op/s 2024-06-11T13:37:54.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:54 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-11T13:37:54.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:54 smithi107 ceph-mon[33178]: osdmap e430: 8 total, 8 up, 8 in 2024-06-11T13:37:55.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:55 smithi057 ceph-mon[29805]: osdmap e431: 8 total, 8 up, 8 in 2024-06-11T13:37:55.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:55 smithi057 ceph-mon[40588]: osdmap e431: 8 total, 8 up, 8 in 2024-06-11T13:37:55.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:55 smithi107 ceph-mon[33178]: osdmap e431: 8 total, 8 up, 8 in 2024-06-11T13:37:56.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:56 smithi057 ceph-mon[29805]: pgmap v638: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:37:56.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:56 smithi057 ceph-mon[40588]: pgmap v638: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:37:56.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:56 smithi107 ceph-mon[33178]: pgmap v638: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:37:58.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:37:58 smithi057 ceph-mon[29805]: pgmap v639: 324 pgs: 324 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-06-11T13:37:58.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:37:58 smithi057 ceph-mon[40588]: pgmap v639: 324 pgs: 324 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-06-11T13:37:58.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:37:58 smithi107 ceph-mon[33178]: pgmap v639: 324 pgs: 324 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-06-11T13:38:00.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:00 smithi057 ceph-mon[29805]: pgmap v640: 324 pgs: 324 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 890 B/s rd, 1.9 KiB/s wr, 4 op/s 2024-06-11T13:38:00.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:00 smithi057 ceph-mon[40588]: pgmap v640: 324 pgs: 324 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 890 B/s rd, 1.9 KiB/s wr, 4 op/s 2024-06-11T13:38:00.797 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:00 smithi107 ceph-mon[33178]: pgmap v640: 324 pgs: 324 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 890 B/s rd, 1.9 KiB/s wr, 4 op/s 2024-06-11T13:38:01.798 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:38:01 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:38:02.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:02 smithi057 ceph-mon[29805]: pgmap v641: 324 pgs: 324 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.6 KiB/s wr, 4 op/s 2024-06-11T13:38:02.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:02 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:38:02.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:02 smithi057 ceph-mon[40588]: pgmap v641: 324 pgs: 324 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.6 KiB/s wr, 4 op/s 2024-06-11T13:38:02.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:02 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:38:02.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:02 smithi107 ceph-mon[33178]: pgmap v641: 324 pgs: 324 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.6 KiB/s wr, 4 op/s 2024-06-11T13:38:02.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:02 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:38:04.034 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:38:03 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:38:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:38:04.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:04 smithi107 ceph-mon[33178]: pgmap v642: 324 pgs: 324 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-06-11T13:38:05.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:04 smithi057 ceph-mon[29805]: pgmap v642: 324 pgs: 324 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-06-11T13:38:05.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:04 smithi057 ceph-mon[40588]: pgmap v642: 324 pgs: 324 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-06-11T13:38:06.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:06 smithi107 ceph-mon[33178]: pgmap v643: 324 pgs: 324 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-06-11T13:38:07.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:06 smithi057 ceph-mon[40588]: pgmap v643: 324 pgs: 324 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-06-11T13:38:07.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:06 smithi057 ceph-mon[29805]: pgmap v643: 324 pgs: 324 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-06-11T13:38:08.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:07 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:38:08.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:07 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:38:08.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:07 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:38:09.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:08 smithi057 ceph-mon[29805]: pgmap v644: 324 pgs: 324 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-06-11T13:38:09.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:08 smithi057 ceph-mon[40588]: pgmap v644: 324 pgs: 324 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-06-11T13:38:09.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:08 smithi107 ceph-mon[33178]: pgmap v644: 324 pgs: 324 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-06-11T13:38:11.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:10 smithi057 ceph-mon[29805]: pgmap v645: 324 pgs: 324 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-06-11T13:38:11.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:10 smithi057 ceph-mon[40588]: pgmap v645: 324 pgs: 324 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-06-11T13:38:11.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:10 smithi107 ceph-mon[33178]: pgmap v645: 324 pgs: 324 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-06-11T13:38:11.798 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:38:11 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:38:13.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:12 smithi057 ceph-mon[29805]: pgmap v646: 324 pgs: 324 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-06-11T13:38:13.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:12 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:38:13.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:12 smithi057 ceph-mon[40588]: pgmap v646: 324 pgs: 324 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-06-11T13:38:13.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:12 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:38:13.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:12 smithi107 ceph-mon[33178]: pgmap v646: 324 pgs: 324 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-06-11T13:38:13.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:12 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:38:14.035 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:38:13 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:38:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:38:15.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:15 smithi107 ceph-mon[33178]: pgmap v647: 324 pgs: 324 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-06-11T13:38:15.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:15 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:38:15.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:15 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:38:15.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:15 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-71"}]: dispatch 2024-06-11T13:38:15.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:15 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-71"}]: dispatch 2024-06-11T13:38:15.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:15 smithi057 ceph-mon[29805]: pgmap v647: 324 pgs: 324 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-06-11T13:38:15.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:15 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:38:15.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:15 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:38:15.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:15 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-71"}]: dispatch 2024-06-11T13:38:15.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:15 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-71"}]: dispatch 2024-06-11T13:38:15.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:15 smithi057 ceph-mon[40588]: pgmap v647: 324 pgs: 324 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-06-11T13:38:15.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:15 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:38:15.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:15 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:38:15.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:15 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-71"}]: dispatch 2024-06-11T13:38:15.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:15 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-71"}]: dispatch 2024-06-11T13:38:16.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:16 smithi107 ceph-mon[33178]: osdmap e432: 8 total, 8 up, 8 in 2024-06-11T13:38:16.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:16 smithi107 ceph-mon[33178]: pgmap v649: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-06-11T13:38:16.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:16 smithi107 ceph-mon[33178]: osdmap e433: 8 total, 8 up, 8 in 2024-06-11T13:38:16.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:16 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:38:16.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:16 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:38:16.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:16 smithi057 ceph-mon[29805]: osdmap e432: 8 total, 8 up, 8 in 2024-06-11T13:38:16.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:16 smithi057 ceph-mon[29805]: pgmap v649: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-06-11T13:38:16.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:16 smithi057 ceph-mon[29805]: osdmap e433: 8 total, 8 up, 8 in 2024-06-11T13:38:16.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:16 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:38:16.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:16 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:38:16.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:16 smithi057 ceph-mon[40588]: osdmap e432: 8 total, 8 up, 8 in 2024-06-11T13:38:16.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:16 smithi057 ceph-mon[40588]: pgmap v649: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-06-11T13:38:16.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:16 smithi057 ceph-mon[40588]: osdmap e433: 8 total, 8 up, 8 in 2024-06-11T13:38:16.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:16 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:38:16.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:16 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:38:18.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:17 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:38:18.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:17 smithi057 ceph-mon[29805]: osdmap e434: 8 total, 8 up, 8 in 2024-06-11T13:38:18.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:17 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:38:18.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:17 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:38:18.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:17 smithi057 ceph-mon[40588]: osdmap e434: 8 total, 8 up, 8 in 2024-06-11T13:38:18.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:17 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:38:18.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:17 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:38:18.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:17 smithi107 ceph-mon[33178]: osdmap e434: 8 total, 8 up, 8 in 2024-06-11T13:38:18.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:17 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:38:19.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:18 smithi057 ceph-mon[29805]: pgmap v652: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:38:19.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:18 smithi057 ceph-mon[29805]: osdmap e435: 8 total, 8 up, 8 in 2024-06-11T13:38:19.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:18 smithi057 ceph-mon[40588]: pgmap v652: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:38:19.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:18 smithi057 ceph-mon[40588]: osdmap e435: 8 total, 8 up, 8 in 2024-06-11T13:38:19.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:18 smithi107 ceph-mon[33178]: pgmap v652: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:38:19.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:18 smithi107 ceph-mon[33178]: osdmap e435: 8 total, 8 up, 8 in 2024-06-11T13:38:20.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:19 smithi057 ceph-mon[29805]: osdmap e436: 8 total, 8 up, 8 in 2024-06-11T13:38:20.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:19 smithi057 ceph-mon[40588]: osdmap e436: 8 total, 8 up, 8 in 2024-06-11T13:38:20.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:19 smithi107 ceph-mon[33178]: osdmap e436: 8 total, 8 up, 8 in 2024-06-11T13:38:21.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:20 smithi057 ceph-mon[40588]: pgmap v655: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:38:21.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:20 smithi057 ceph-mon[29805]: pgmap v655: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:38:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:20 smithi107 ceph-mon[33178]: pgmap v655: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:38:21.798 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:38:21 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:38:23.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:22 smithi057 ceph-mon[40588]: pgmap v656: 324 pgs: 324 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s rd, 1.2 KiB/s wr, 81 op/s 2024-06-11T13:38:23.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:22 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:38:23.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:22 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:38:23.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:22 smithi057 ceph-mon[29805]: pgmap v656: 324 pgs: 324 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s rd, 1.2 KiB/s wr, 81 op/s 2024-06-11T13:38:23.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:22 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:38:23.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:22 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:38:23.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:22 smithi107 ceph-mon[33178]: pgmap v656: 324 pgs: 324 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s rd, 1.2 KiB/s wr, 81 op/s 2024-06-11T13:38:23.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:22 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:38:23.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:22 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:38:24.035 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:38:23 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:38:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:38:24.935 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:24 smithi057 ceph-mon[40588]: pgmap v657: 324 pgs: 324 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 40 KiB/s rd, 1.0 KiB/s wr, 68 op/s 2024-06-11T13:38:24.936 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:24 smithi057 ceph-mon[29805]: pgmap v657: 324 pgs: 324 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 40 KiB/s rd, 1.0 KiB/s wr, 68 op/s 2024-06-11T13:38:25.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:24 smithi107 ceph-mon[33178]: pgmap v657: 324 pgs: 324 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 40 KiB/s rd, 1.0 KiB/s wr, 68 op/s 2024-06-11T13:38:27.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:26 smithi057 ceph-mon[29805]: pgmap v658: 324 pgs: 324 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 34 KiB/s rd, 895 B/s wr, 58 op/s 2024-06-11T13:38:27.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:26 smithi057 ceph-mon[40588]: pgmap v658: 324 pgs: 324 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 34 KiB/s rd, 895 B/s wr, 58 op/s 2024-06-11T13:38:27.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:26 smithi107 ceph-mon[33178]: pgmap v658: 324 pgs: 324 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 34 KiB/s rd, 895 B/s wr, 58 op/s 2024-06-11T13:38:28.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:27 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:38:28.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:27 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:38:28.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:27 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:38:28.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:27 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:38:28.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:27 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:38:28.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:27 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:38:28.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:27 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:38:28.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:27 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:38:28.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:27 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:38:28.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:27 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:38:28.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:27 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:38:28.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:27 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:38:29.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:28 smithi057 ceph-mon[29805]: pgmap v659: 324 pgs: 324 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 734 B/s wr, 76 op/s 2024-06-11T13:38:29.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:28 smithi057 ceph-mon[40588]: pgmap v659: 324 pgs: 324 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 734 B/s wr, 76 op/s 2024-06-11T13:38:29.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:28 smithi107 ceph-mon[33178]: pgmap v659: 324 pgs: 324 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 734 B/s wr, 76 op/s 2024-06-11T13:38:30.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:29 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:38:30.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:29 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:38:30.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:29 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-73"}]: dispatch 2024-06-11T13:38:30.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:29 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-73"}]: dispatch 2024-06-11T13:38:30.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:29 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:38:30.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:29 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:38:30.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:29 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-73"}]: dispatch 2024-06-11T13:38:30.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:29 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-73"}]: dispatch 2024-06-11T13:38:30.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:29 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:38:30.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:29 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:38:30.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:29 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-73"}]: dispatch 2024-06-11T13:38:30.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:29 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-73"}]: dispatch 2024-06-11T13:38:31.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:30 smithi057 ceph-mon[40588]: pgmap v660: 324 pgs: 324 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 41 KiB/s rd, 667 B/s wr, 69 op/s 2024-06-11T13:38:31.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:30 smithi057 ceph-mon[40588]: osdmap e437: 8 total, 8 up, 8 in 2024-06-11T13:38:31.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:30 smithi057 ceph-mon[29805]: pgmap v660: 324 pgs: 324 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 41 KiB/s rd, 667 B/s wr, 69 op/s 2024-06-11T13:38:31.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:30 smithi057 ceph-mon[29805]: osdmap e437: 8 total, 8 up, 8 in 2024-06-11T13:38:31.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:30 smithi107 ceph-mon[33178]: pgmap v660: 324 pgs: 324 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 41 KiB/s rd, 667 B/s wr, 69 op/s 2024-06-11T13:38:31.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:30 smithi107 ceph-mon[33178]: osdmap e437: 8 total, 8 up, 8 in 2024-06-11T13:38:31.798 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:38:31 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:38:31.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:31 smithi107 ceph-mon[33178]: osdmap e438: 8 total, 8 up, 8 in 2024-06-11T13:38:31.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:31 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:38:31.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:31 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:38:31.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:31 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:38:31.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:31 smithi107 ceph-mon[33178]: osdmap e439: 8 total, 8 up, 8 in 2024-06-11T13:38:32.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:31 smithi057 ceph-mon[40588]: osdmap e438: 8 total, 8 up, 8 in 2024-06-11T13:38:32.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:31 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:38:32.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:31 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:38:32.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:31 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:38:32.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:31 smithi057 ceph-mon[40588]: osdmap e439: 8 total, 8 up, 8 in 2024-06-11T13:38:32.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:31 smithi057 ceph-mon[29805]: osdmap e438: 8 total, 8 up, 8 in 2024-06-11T13:38:32.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:31 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:38:32.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:31 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:38:32.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:31 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:38:32.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:31 smithi057 ceph-mon[29805]: osdmap e439: 8 total, 8 up, 8 in 2024-06-11T13:38:33.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:32 smithi057 ceph-mon[40588]: pgmap v663: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 21 KiB/s rd, 0 B/s wr, 33 op/s 2024-06-11T13:38:33.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:32 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:38:33.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:32 smithi057 ceph-mon[40588]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:38:33.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:32 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:38:33.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:32 smithi057 ceph-mon[40588]: osdmap e440: 8 total, 8 up, 8 in 2024-06-11T13:38:33.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:32 smithi057 ceph-mon[29805]: pgmap v663: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 21 KiB/s rd, 0 B/s wr, 33 op/s 2024-06-11T13:38:33.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:32 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:38:33.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:32 smithi057 ceph-mon[29805]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:38:33.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:32 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:38:33.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:32 smithi057 ceph-mon[29805]: osdmap e440: 8 total, 8 up, 8 in 2024-06-11T13:38:33.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:32 smithi107 ceph-mon[33178]: pgmap v663: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 21 KiB/s rd, 0 B/s wr, 33 op/s 2024-06-11T13:38:33.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:32 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:38:33.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:32 smithi107 ceph-mon[33178]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:38:33.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:32 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:38:33.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:32 smithi107 ceph-mon[33178]: osdmap e440: 8 total, 8 up, 8 in 2024-06-11T13:38:34.035 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:38:33 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:38:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:38:35.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:34 smithi057 ceph-mon[40588]: pgmap v666: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:38:35.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:34 smithi057 ceph-mon[40588]: osdmap e441: 8 total, 8 up, 8 in 2024-06-11T13:38:35.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:34 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:38:35.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:34 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:38:35.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:34 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-75"}]: dispatch 2024-06-11T13:38:35.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:34 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-75"}]: dispatch 2024-06-11T13:38:35.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:34 smithi057 ceph-mon[29805]: pgmap v666: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:38:35.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:34 smithi057 ceph-mon[29805]: osdmap e441: 8 total, 8 up, 8 in 2024-06-11T13:38:35.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:34 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:38:35.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:34 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:38:35.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:34 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-75"}]: dispatch 2024-06-11T13:38:35.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:34 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-75"}]: dispatch 2024-06-11T13:38:35.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:34 smithi107 ceph-mon[33178]: pgmap v666: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:38:35.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:34 smithi107 ceph-mon[33178]: osdmap e441: 8 total, 8 up, 8 in 2024-06-11T13:38:35.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:34 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:38:35.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:34 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:38:35.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:34 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-75"}]: dispatch 2024-06-11T13:38:35.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:34 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-75"}]: dispatch 2024-06-11T13:38:36.034 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:35 smithi057 ceph-mon[40588]: osdmap e442: 8 total, 8 up, 8 in 2024-06-11T13:38:36.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:35 smithi057 ceph-mon[29805]: osdmap e442: 8 total, 8 up, 8 in 2024-06-11T13:38:36.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:35 smithi107 ceph-mon[33178]: osdmap e442: 8 total, 8 up, 8 in 2024-06-11T13:38:37.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:36 smithi057 ceph-mon[40588]: pgmap v669: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:38:37.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:36 smithi057 ceph-mon[40588]: osdmap e443: 8 total, 8 up, 8 in 2024-06-11T13:38:37.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:36 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:38:37.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:36 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:38:37.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:36 smithi057 ceph-mon[29805]: pgmap v669: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:38:37.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:36 smithi057 ceph-mon[29805]: osdmap e443: 8 total, 8 up, 8 in 2024-06-11T13:38:37.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:36 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:38:37.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:36 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:38:37.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:36 smithi107 ceph-mon[33178]: pgmap v669: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:38:37.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:36 smithi107 ceph-mon[33178]: osdmap e443: 8 total, 8 up, 8 in 2024-06-11T13:38:37.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:36 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:38:37.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:36 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:38:38.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:37 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:38:38.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:37 smithi057 ceph-mon[40588]: osdmap e444: 8 total, 8 up, 8 in 2024-06-11T13:38:38.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:37 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:38:38.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:37 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:38:38.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:37 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:38:38.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:37 smithi057 ceph-mon[29805]: osdmap e444: 8 total, 8 up, 8 in 2024-06-11T13:38:38.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:37 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:38:38.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:37 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:38:38.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:37 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:38:38.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:37 smithi107 ceph-mon[33178]: osdmap e444: 8 total, 8 up, 8 in 2024-06-11T13:38:38.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:37 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:38:38.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:37 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:38:39.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:38 smithi107 ceph-mon[33178]: pgmap v672: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-06-11T13:38:39.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:38 smithi107 ceph-mon[33178]: osdmap e445: 8 total, 8 up, 8 in 2024-06-11T13:38:39.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:38 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:38:39.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:38 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:38:39.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:38 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-77"}]: dispatch 2024-06-11T13:38:39.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:38 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-77"}]: dispatch 2024-06-11T13:38:39.284 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:38 smithi057 ceph-mon[29805]: pgmap v672: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-06-11T13:38:39.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:38 smithi057 ceph-mon[29805]: osdmap e445: 8 total, 8 up, 8 in 2024-06-11T13:38:39.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:38 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:38:39.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:38 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:38:39.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:38 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-77"}]: dispatch 2024-06-11T13:38:39.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:38 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-77"}]: dispatch 2024-06-11T13:38:39.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:38 smithi057 ceph-mon[40588]: pgmap v672: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-06-11T13:38:39.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:38 smithi057 ceph-mon[40588]: osdmap e445: 8 total, 8 up, 8 in 2024-06-11T13:38:39.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:38 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:38:39.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:38 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:38:39.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:38 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-77"}]: dispatch 2024-06-11T13:38:39.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:38 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-77"}]: dispatch 2024-06-11T13:38:40.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:39 smithi057 ceph-mon[29805]: osdmap e446: 8 total, 8 up, 8 in 2024-06-11T13:38:40.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:39 smithi057 ceph-mon[40588]: osdmap e446: 8 total, 8 up, 8 in 2024-06-11T13:38:40.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:39 smithi107 ceph-mon[33178]: osdmap e446: 8 total, 8 up, 8 in 2024-06-11T13:38:41.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:40 smithi057 ceph-mon[29805]: pgmap v675: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-06-11T13:38:41.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:40 smithi057 ceph-mon[29805]: osdmap e447: 8 total, 8 up, 8 in 2024-06-11T13:38:41.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:40 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:38:41.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:40 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:38:41.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:40 smithi057 ceph-mon[40588]: pgmap v675: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-06-11T13:38:41.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:40 smithi057 ceph-mon[40588]: osdmap e447: 8 total, 8 up, 8 in 2024-06-11T13:38:41.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:40 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:38:41.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:40 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:38:41.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:40 smithi107 ceph-mon[33178]: pgmap v675: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-06-11T13:38:41.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:40 smithi107 ceph-mon[33178]: osdmap e447: 8 total, 8 up, 8 in 2024-06-11T13:38:41.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:40 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:38:41.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:40 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:38:41.798 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:38:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:38:42.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:41 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:38:42.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:41 smithi057 ceph-mon[29805]: osdmap e448: 8 total, 8 up, 8 in 2024-06-11T13:38:42.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:41 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:38:42.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:41 smithi057 ceph-mon[29805]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:38:42.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:41 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:38:42.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:41 smithi057 ceph-mon[40588]: osdmap e448: 8 total, 8 up, 8 in 2024-06-11T13:38:42.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:41 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:38:42.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:41 smithi057 ceph-mon[40588]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:38:42.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:41 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:38:42.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:41 smithi107 ceph-mon[33178]: osdmap e448: 8 total, 8 up, 8 in 2024-06-11T13:38:42.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:41 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:38:42.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:41 smithi107 ceph-mon[33178]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:38:43.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:42 smithi057 ceph-mon[29805]: pgmap v678: 324 pgs: 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-06-11T13:38:43.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:42 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:38:43.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:42 smithi057 ceph-mon[29805]: osdmap e449: 8 total, 8 up, 8 in 2024-06-11T13:38:43.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:42 smithi057 ceph-mon[40588]: pgmap v678: 324 pgs: 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-06-11T13:38:43.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:42 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:38:43.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:42 smithi057 ceph-mon[40588]: osdmap e449: 8 total, 8 up, 8 in 2024-06-11T13:38:43.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:42 smithi107 ceph-mon[33178]: pgmap v678: 324 pgs: 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-06-11T13:38:43.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:42 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:38:43.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:42 smithi107 ceph-mon[33178]: osdmap e449: 8 total, 8 up, 8 in 2024-06-11T13:38:43.804 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a87b10 2024-06-11T13:38:43.805 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a87c40 2024-06-11T13:38:43.805 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a87d70 2024-06-11T13:38:43.805 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a88860 2024-06-11T13:38:43.806 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a88d30 2024-06-11T13:38:43.806 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a88e80 2024-06-11T13:38:43.806 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a88240 2024-06-11T13:38:43.807 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a88370 2024-06-11T13:38:43.807 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a884a0 2024-06-11T13:38:43.808 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a88fb0 2024-06-11T13:38:43.808 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a89460 2024-06-11T13:38:43.808 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a895b0 2024-06-11T13:38:43.808 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a88990 2024-06-11T13:38:43.808 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a88ac0 2024-06-11T13:38:43.808 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a88bf0 2024-06-11T13:38:43.809 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a89700 2024-06-11T13:38:43.809 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a89c10 2024-06-11T13:38:43.809 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a89d60 2024-06-11T13:38:43.809 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a890c0 2024-06-11T13:38:43.809 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a891f0 2024-06-11T13:38:43.809 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a89320 2024-06-11T13:38:43.809 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a89e90 2024-06-11T13:38:43.809 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8a340 2024-06-11T13:38:43.809 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8a490 2024-06-11T13:38:43.809 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8aa70 2024-06-11T13:38:43.809 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a89fa0 2024-06-11T13:38:43.809 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8a0d0 2024-06-11T13:38:43.809 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8a200 2024-06-11T13:38:43.810 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8ab80 2024-06-11T13:38:43.810 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8a5c0 2024-06-11T13:38:43.810 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8a6f0 2024-06-11T13:38:43.810 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8a840 2024-06-11T13:38:43.810 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8acb0 2024-06-11T13:38:43.810 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8b180 2024-06-11T13:38:43.810 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8b2b0 2024-06-11T13:38:43.810 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a89850 2024-06-11T13:38:43.810 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a89980 2024-06-11T13:38:43.810 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a89ab0 2024-06-11T13:38:43.810 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8b3e0 2024-06-11T13:38:43.810 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8b900 2024-06-11T13:38:43.811 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8baa0 2024-06-11T13:38:43.811 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8ade0 2024-06-11T13:38:43.811 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8af10 2024-06-11T13:38:43.811 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8b040 2024-06-11T13:38:43.811 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8bbd0 2024-06-11T13:38:43.811 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8c0a0 2024-06-11T13:38:43.811 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8c1f0 2024-06-11T13:38:43.811 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8b510 2024-06-11T13:38:43.811 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8b640 2024-06-11T13:38:43.811 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8b770 2024-06-11T13:38:43.811 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8c320 2024-06-11T13:38:43.811 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8c7f0 2024-06-11T13:38:43.811 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8c940 2024-06-11T13:38:43.811 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8bd00 2024-06-11T13:38:43.812 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8be30 2024-06-11T13:38:43.812 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8bf60 2024-06-11T13:38:43.812 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8ca70 2024-06-11T13:38:43.812 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8cf40 2024-06-11T13:38:43.812 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8d090 2024-06-11T13:38:43.812 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8d670 2024-06-11T13:38:43.812 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8cba0 2024-06-11T13:38:43.812 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8ccd0 2024-06-11T13:38:43.812 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8ce00 2024-06-11T13:38:43.812 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8d7a0 2024-06-11T13:38:43.812 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8d1c0 2024-06-11T13:38:43.812 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8d310 2024-06-11T13:38:43.812 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8d440 2024-06-11T13:38:43.812 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8d8d0 2024-06-11T13:38:43.813 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8dda0 2024-06-11T13:38:43.813 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8ded0 2024-06-11T13:38:43.813 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8c450 2024-06-11T13:38:43.813 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8c580 2024-06-11T13:38:43.813 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8c6b0 2024-06-11T13:38:43.813 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8e000 2024-06-11T13:38:43.813 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8e4d0 2024-06-11T13:38:43.813 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8e600 2024-06-11T13:38:43.813 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8e110 2024-06-11T13:38:43.813 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8e240 2024-06-11T13:38:43.813 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8e390 2024-06-11T13:38:43.813 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8e750 2024-06-11T13:38:43.813 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8ec20 2024-06-11T13:38:43.813 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8ed70 2024-06-11T13:38:44.035 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:38:43 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:38:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:38:44.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:43 smithi057 ceph-mon[40588]: osdmap e450: 8 total, 8 up, 8 in 2024-06-11T13:38:44.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:43 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "304.e"}]: dispatch 2024-06-11T13:38:44.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:43 smithi057 ceph-mon[40588]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "304.e"}]: dispatch 2024-06-11T13:38:44.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:43 smithi057 ceph-mon[40588]: 304.e deep-scrub starts 2024-06-11T13:38:44.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:43 smithi057 ceph-mon[40588]: 304.e deep-scrub ok 2024-06-11T13:38:44.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:43 smithi057 ceph-mon[40588]: pgmap v681: 324 pgs: 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-06-11T13:38:44.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:43 smithi057 ceph-mon[29805]: osdmap e450: 8 total, 8 up, 8 in 2024-06-11T13:38:44.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:43 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "304.e"}]: dispatch 2024-06-11T13:38:44.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:43 smithi057 ceph-mon[29805]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "304.e"}]: dispatch 2024-06-11T13:38:44.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:43 smithi057 ceph-mon[29805]: 304.e deep-scrub starts 2024-06-11T13:38:44.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:43 smithi057 ceph-mon[29805]: 304.e deep-scrub ok 2024-06-11T13:38:44.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:43 smithi057 ceph-mon[29805]: pgmap v681: 324 pgs: 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-06-11T13:38:44.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:43 smithi107 ceph-mon[33178]: osdmap e450: 8 total, 8 up, 8 in 2024-06-11T13:38:44.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:43 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "304.e"}]: dispatch 2024-06-11T13:38:44.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:43 smithi107 ceph-mon[33178]: from='mon.? v1:172.21.15.107:0/2543604917' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "304.e"}]: dispatch 2024-06-11T13:38:44.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:43 smithi107 ceph-mon[33178]: 304.e deep-scrub starts 2024-06-11T13:38:44.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:43 smithi107 ceph-mon[33178]: 304.e deep-scrub ok 2024-06-11T13:38:44.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:43 smithi107 ceph-mon[33178]: pgmap v681: 324 pgs: 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-06-11T13:38:46.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:46 smithi057 ceph-mon[29805]: pgmap v682: 324 pgs: 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 920 B/s wr, 2 op/s 2024-06-11T13:38:46.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:46 smithi057 ceph-mon[40588]: pgmap v682: 324 pgs: 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 920 B/s wr, 2 op/s 2024-06-11T13:38:46.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:46 smithi107 ceph-mon[33178]: pgmap v682: 324 pgs: 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 920 B/s wr, 2 op/s 2024-06-11T13:38:48.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:48 smithi057 ceph-mon[29805]: pgmap v683: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-06-11T13:38:48.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:48 smithi057 ceph-mon[40588]: pgmap v683: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-06-11T13:38:48.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:48 smithi107 ceph-mon[33178]: pgmap v683: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-06-11T13:38:50.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:50 smithi057 ceph-mon[29805]: pgmap v684: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:38:50.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:50 smithi057 ceph-mon[40588]: pgmap v684: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:38:50.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:50 smithi107 ceph-mon[33178]: pgmap v684: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:38:51.798 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:38:51 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:38:52.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:52 smithi057 ceph-mon[29805]: pgmap v685: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 215 B/s wr, 1 op/s 2024-06-11T13:38:52.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:52 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:38:52.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:52 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:38:52.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:52 smithi057 ceph-mon[40588]: pgmap v685: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 215 B/s wr, 1 op/s 2024-06-11T13:38:52.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:52 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:38:52.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:52 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:38:52.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:52 smithi107 ceph-mon[33178]: pgmap v685: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 215 B/s wr, 1 op/s 2024-06-11T13:38:52.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:52 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:38:52.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:52 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:38:54.034 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:38:53 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:38:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:38:54.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:54 smithi057 ceph-mon[29805]: pgmap v686: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 973 B/s rd, 194 B/s wr, 1 op/s 2024-06-11T13:38:54.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:54 smithi057 ceph-mon[40588]: pgmap v686: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 973 B/s rd, 194 B/s wr, 1 op/s 2024-06-11T13:38:54.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:54 smithi107 ceph-mon[33178]: pgmap v686: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 973 B/s rd, 194 B/s wr, 1 op/s 2024-06-11T13:38:56.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:56 smithi057 ceph-mon[29805]: pgmap v687: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 170 B/s wr, 1 op/s 2024-06-11T13:38:56.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:56 smithi057 ceph-mon[40588]: pgmap v687: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 170 B/s wr, 1 op/s 2024-06-11T13:38:56.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:56 smithi107 ceph-mon[33178]: pgmap v687: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 170 B/s wr, 1 op/s 2024-06-11T13:38:58.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:38:58 smithi057 ceph-mon[29805]: pgmap v688: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 170 B/s wr, 1 op/s 2024-06-11T13:38:58.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:38:58 smithi057 ceph-mon[40588]: pgmap v688: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 170 B/s wr, 1 op/s 2024-06-11T13:38:58.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:38:58 smithi107 ceph-mon[33178]: pgmap v688: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 170 B/s wr, 1 op/s 2024-06-11T13:39:00.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:00 smithi057 ceph-mon[29805]: pgmap v689: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:39:00.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:00 smithi057 ceph-mon[40588]: pgmap v689: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:39:00.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:00 smithi107 ceph-mon[33178]: pgmap v689: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:39:01.798 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:39:01 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:39:02.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:02 smithi057 ceph-mon[29805]: pgmap v690: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:39:02.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:02 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:39:02.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:02 smithi057 ceph-mon[40588]: pgmap v690: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:39:02.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:02 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:39:02.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:02 smithi107 ceph-mon[33178]: pgmap v690: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:39:02.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:02 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:39:03.285 INFO:journalctl@ceph.osd.0.smithi057.stdout:Jun 11 13:39:02 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-0[44455]: 2024-06-11T13:39:02.896+0000 7f4ac827b640 -1 snap_mapper.add_oid found existing snaps mapped on 304:738d90f2:test-rados-api-smithi057-77599-80::foo:2, removing 2024-06-11T13:39:03.298 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 11 13:39:02 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-4[37950]: 2024-06-11T13:39:02.896+0000 7f2ff0cc8640 -1 snap_mapper.add_oid found existing snaps mapped on 304:738d90f2:test-rados-api-smithi057-77599-80::foo:2, removing 2024-06-11T13:39:03.298 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 13:39:02 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-5[43766]: 2024-06-11T13:39:02.896+0000 7f6972f11640 -1 snap_mapper.add_oid found existing snaps mapped on 304:738d90f2:test-rados-api-smithi057-77599-80::foo:2, removing 2024-06-11T13:39:03.785 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:39:03 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:39:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:39:03.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:03 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:03.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:03 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:03.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:03 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-79"}]: dispatch 2024-06-11T13:39:03.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:03 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-79"}]: dispatch 2024-06-11T13:39:03.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:03 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:03.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:03 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:03.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:03 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-79"}]: dispatch 2024-06-11T13:39:03.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:03 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-79"}]: dispatch 2024-06-11T13:39:03.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:03 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:03.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:03 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:03.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:03 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-79"}]: dispatch 2024-06-11T13:39:03.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:03 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-79"}]: dispatch 2024-06-11T13:39:04.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:04 smithi057 ceph-mon[40588]: pgmap v691: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:39:04.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:04 smithi057 ceph-mon[40588]: osdmap e451: 8 total, 8 up, 8 in 2024-06-11T13:39:04.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:04 smithi057 ceph-mon[29805]: pgmap v691: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:39:04.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:04 smithi057 ceph-mon[29805]: osdmap e451: 8 total, 8 up, 8 in 2024-06-11T13:39:04.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:04 smithi107 ceph-mon[33178]: pgmap v691: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:39:04.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:04 smithi107 ceph-mon[33178]: osdmap e451: 8 total, 8 up, 8 in 2024-06-11T13:39:05.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:05 smithi107 ceph-mon[33178]: osdmap e452: 8 total, 8 up, 8 in 2024-06-11T13:39:05.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:05 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:39:05.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:05 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:39:06.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:05 smithi057 ceph-mon[40588]: osdmap e452: 8 total, 8 up, 8 in 2024-06-11T13:39:06.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:05 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:39:06.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:05 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:39:06.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:05 smithi057 ceph-mon[29805]: osdmap e452: 8 total, 8 up, 8 in 2024-06-11T13:39:06.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:05 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:39:06.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:05 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:39:07.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:06 smithi057 ceph-mon[29805]: pgmap v694: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-06-11T13:39:07.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:06 smithi057 ceph-mon[29805]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:39:07.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:06 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:39:07.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:06 smithi057 ceph-mon[29805]: osdmap e453: 8 total, 8 up, 8 in 2024-06-11T13:39:07.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:06 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:39:07.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:06 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:39:07.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:06 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:39:07.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:06 smithi057 ceph-mon[40588]: pgmap v694: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-06-11T13:39:07.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:06 smithi057 ceph-mon[40588]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:39:07.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:06 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:39:07.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:06 smithi057 ceph-mon[40588]: osdmap e453: 8 total, 8 up, 8 in 2024-06-11T13:39:07.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:06 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:39:07.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:06 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:39:07.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:06 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:39:07.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:06 smithi107 ceph-mon[33178]: pgmap v694: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-06-11T13:39:07.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:06 smithi107 ceph-mon[33178]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:39:07.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:06 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:39:07.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:06 smithi107 ceph-mon[33178]: osdmap e453: 8 total, 8 up, 8 in 2024-06-11T13:39:07.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:06 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:39:07.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:06 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:39:07.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:06 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:39:08.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:07 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-11T13:39:08.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:07 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_tier","val": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:08.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:07 smithi057 ceph-mon[29805]: osdmap e454: 8 total, 8 up, 8 in 2024-06-11T13:39:08.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:07 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_tier","val": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:08.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:07 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:39:08.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:07 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-11T13:39:08.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:07 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_tier","val": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:08.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:07 smithi057 ceph-mon[40588]: osdmap e454: 8 total, 8 up, 8 in 2024-06-11T13:39:08.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:07 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_tier","val": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:08.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:07 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:39:08.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:07 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-11T13:39:08.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:07 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_tier","val": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:08.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:07 smithi107 ceph-mon[33178]: osdmap e454: 8 total, 8 up, 8 in 2024-06-11T13:39:08.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:07 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_tier","val": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:08.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:07 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:39:09.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:08 smithi057 ceph-mon[29805]: pgmap v697: 324 pgs: 324 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-06-11T13:39:09.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:08 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_tier","val": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:39:09.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:08 smithi057 ceph-mon[29805]: osdmap e455: 8 total, 8 up, 8 in 2024-06-11T13:39:09.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:08 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-11T13:39:09.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:08 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-11T13:39:09.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:08 smithi057 ceph-mon[40588]: pgmap v697: 324 pgs: 324 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-06-11T13:39:09.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:08 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_tier","val": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:39:09.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:08 smithi057 ceph-mon[40588]: osdmap e455: 8 total, 8 up, 8 in 2024-06-11T13:39:09.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:08 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-11T13:39:09.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:08 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-11T13:39:09.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:08 smithi107 ceph-mon[33178]: pgmap v697: 324 pgs: 324 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-06-11T13:39:09.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:08 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_tier","val": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:39:09.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:08 smithi107 ceph-mon[33178]: osdmap e455: 8 total, 8 up, 8 in 2024-06-11T13:39:09.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:08 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-11T13:39:09.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:08 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-11T13:39:10.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:09 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-06-11T13:39:10.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:09 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-11T13:39:10.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:09 smithi057 ceph-mon[29805]: osdmap e456: 8 total, 8 up, 8 in 2024-06-11T13:39:10.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:09 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-11T13:39:10.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:09 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-06-11T13:39:10.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:09 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-11T13:39:10.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:09 smithi057 ceph-mon[40588]: osdmap e456: 8 total, 8 up, 8 in 2024-06-11T13:39:10.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:09 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-11T13:39:10.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:09 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-06-11T13:39:10.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:09 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-11T13:39:10.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:09 smithi107 ceph-mon[33178]: osdmap e456: 8 total, 8 up, 8 in 2024-06-11T13:39:10.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:09 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-11T13:39:11.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:10 smithi057 ceph-mon[29805]: pgmap v700: 324 pgs: 324 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-06-11T13:39:11.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:10 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-06-11T13:39:11.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:10 smithi057 ceph-mon[29805]: osdmap e457: 8 total, 8 up, 8 in 2024-06-11T13:39:11.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:10 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-06-11T13:39:11.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:10 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-06-11T13:39:11.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:10 smithi057 ceph-mon[40588]: pgmap v700: 324 pgs: 324 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-06-11T13:39:11.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:10 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-06-11T13:39:11.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:10 smithi057 ceph-mon[40588]: osdmap e457: 8 total, 8 up, 8 in 2024-06-11T13:39:11.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:10 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-06-11T13:39:11.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:10 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-06-11T13:39:11.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:10 smithi107 ceph-mon[33178]: pgmap v700: 324 pgs: 324 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-06-11T13:39:11.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:10 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-06-11T13:39:11.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:10 smithi107 ceph-mon[33178]: osdmap e457: 8 total, 8 up, 8 in 2024-06-11T13:39:11.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:10 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-06-11T13:39:11.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:10 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-06-11T13:39:11.798 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:39:11 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:39:11.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:11 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-06-11T13:39:11.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:11 smithi107 ceph-mon[33178]: osdmap e458: 8 total, 8 up, 8 in 2024-06-11T13:39:11.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:11 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-06-11T13:39:11.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:11 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-06-11T13:39:11.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:11 smithi107 ceph-mon[33178]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:39:12.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:11 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-06-11T13:39:12.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:11 smithi057 ceph-mon[40588]: osdmap e458: 8 total, 8 up, 8 in 2024-06-11T13:39:12.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:11 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-06-11T13:39:12.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:11 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-06-11T13:39:12.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:11 smithi057 ceph-mon[40588]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:39:12.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:11 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-06-11T13:39:12.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:11 smithi057 ceph-mon[29805]: osdmap e458: 8 total, 8 up, 8 in 2024-06-11T13:39:12.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:11 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-06-11T13:39:12.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:11 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-06-11T13:39:12.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:11 smithi057 ceph-mon[29805]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:39:13.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:12 smithi057 ceph-mon[29805]: pgmap v703: 324 pgs: 324 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 9.2 KiB/s wr, 21 op/s 2024-06-11T13:39:13.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:12 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:39:13.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:12 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-06-11T13:39:13.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:12 smithi057 ceph-mon[29805]: osdmap e459: 8 total, 8 up, 8 in 2024-06-11T13:39:13.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:12 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-11T13:39:13.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:12 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-11T13:39:13.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:12 smithi057 ceph-mon[40588]: pgmap v703: 324 pgs: 324 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 9.2 KiB/s wr, 21 op/s 2024-06-11T13:39:13.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:12 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:39:13.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:12 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-06-11T13:39:13.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:12 smithi057 ceph-mon[40588]: osdmap e459: 8 total, 8 up, 8 in 2024-06-11T13:39:13.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:12 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-11T13:39:13.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:12 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-11T13:39:13.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:12 smithi107 ceph-mon[33178]: pgmap v703: 324 pgs: 324 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 9.2 KiB/s wr, 21 op/s 2024-06-11T13:39:13.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:12 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:39:13.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:12 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-06-11T13:39:13.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:12 smithi107 ceph-mon[33178]: osdmap e459: 8 total, 8 up, 8 in 2024-06-11T13:39:13.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:12 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-11T13:39:13.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:12 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-11T13:39:14.034 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:39:13 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:39:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:39:14.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:13 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-06-11T13:39:14.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:13 smithi057 ceph-mon[29805]: osdmap e460: 8 total, 8 up, 8 in 2024-06-11T13:39:14.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:13 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:14.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:13 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:14.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:13 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-81"}]: dispatch 2024-06-11T13:39:14.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:13 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-81"}]: dispatch 2024-06-11T13:39:14.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:13 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-06-11T13:39:14.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:13 smithi057 ceph-mon[40588]: osdmap e460: 8 total, 8 up, 8 in 2024-06-11T13:39:14.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:13 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:14.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:13 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:14.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:13 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-81"}]: dispatch 2024-06-11T13:39:14.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:13 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-81"}]: dispatch 2024-06-11T13:39:14.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:13 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-06-11T13:39:14.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:13 smithi107 ceph-mon[33178]: osdmap e460: 8 total, 8 up, 8 in 2024-06-11T13:39:14.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:13 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:14.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:13 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:14.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:13 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-81"}]: dispatch 2024-06-11T13:39:14.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:13 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-81"}]: dispatch 2024-06-11T13:39:15.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:14 smithi057 ceph-mon[40588]: pgmap v706: 324 pgs: 324 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 9.2 KiB/s wr, 21 op/s 2024-06-11T13:39:15.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:14 smithi057 ceph-mon[40588]: osdmap e461: 8 total, 8 up, 8 in 2024-06-11T13:39:15.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:14 smithi057 ceph-mon[29805]: pgmap v706: 324 pgs: 324 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 9.2 KiB/s wr, 21 op/s 2024-06-11T13:39:15.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:14 smithi057 ceph-mon[29805]: osdmap e461: 8 total, 8 up, 8 in 2024-06-11T13:39:15.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:14 smithi107 ceph-mon[33178]: pgmap v706: 324 pgs: 324 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 9.2 KiB/s wr, 21 op/s 2024-06-11T13:39:15.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:14 smithi107 ceph-mon[33178]: osdmap e461: 8 total, 8 up, 8 in 2024-06-11T13:39:16.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:15 smithi057 ceph-mon[40588]: osdmap e462: 8 total, 8 up, 8 in 2024-06-11T13:39:16.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:15 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:39:16.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:15 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:39:16.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:15 smithi057 ceph-mon[29805]: osdmap e462: 8 total, 8 up, 8 in 2024-06-11T13:39:16.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:15 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:39:16.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:15 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:39:16.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:15 smithi107 ceph-mon[33178]: osdmap e462: 8 total, 8 up, 8 in 2024-06-11T13:39:16.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:15 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:39:16.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:15 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:39:17.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:16 smithi057 ceph-mon[40588]: pgmap v709: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:39:17.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:16 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:39:17.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:16 smithi057 ceph-mon[40588]: osdmap e463: 8 total, 8 up, 8 in 2024-06-11T13:39:17.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:16 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:39:17.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:16 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:39:17.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:16 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:39:17.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:16 smithi057 ceph-mon[40588]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:39:17.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:16 smithi057 ceph-mon[29805]: pgmap v709: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:39:17.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:16 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:39:17.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:16 smithi057 ceph-mon[29805]: osdmap e463: 8 total, 8 up, 8 in 2024-06-11T13:39:17.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:16 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:39:17.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:16 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:39:17.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:16 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:39:17.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:16 smithi057 ceph-mon[29805]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:39:17.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:16 smithi107 ceph-mon[33178]: pgmap v709: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:39:17.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:16 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:39:17.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:16 smithi107 ceph-mon[33178]: osdmap e463: 8 total, 8 up, 8 in 2024-06-11T13:39:17.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:16 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:39:17.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:16 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:39:17.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:16 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:39:17.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:16 smithi107 ceph-mon[33178]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:39:18.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:17 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-11T13:39:18.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:17 smithi057 ceph-mon[29805]: osdmap e464: 8 total, 8 up, 8 in 2024-06-11T13:39:18.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:17 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-83","var": "dedup_tier","val": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:18.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:17 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-83","var": "dedup_tier","val": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:18.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:17 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-11T13:39:18.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:17 smithi057 ceph-mon[40588]: osdmap e464: 8 total, 8 up, 8 in 2024-06-11T13:39:18.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:17 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-83","var": "dedup_tier","val": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:18.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:17 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-83","var": "dedup_tier","val": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:18.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:17 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-11T13:39:18.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:17 smithi107 ceph-mon[33178]: osdmap e464: 8 total, 8 up, 8 in 2024-06-11T13:39:18.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:17 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-83","var": "dedup_tier","val": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:18.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:17 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-83","var": "dedup_tier","val": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:19.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:18 smithi057 ceph-mon[29805]: pgmap v712: 324 pgs: 324 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 3.0 KiB/s wr, 18 op/s 2024-06-11T13:39:19.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:18 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-83","var": "dedup_tier","val": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:39:19.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:18 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-11T13:39:19.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:18 smithi057 ceph-mon[29805]: osdmap e465: 8 total, 8 up, 8 in 2024-06-11T13:39:19.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:18 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-11T13:39:19.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:18 smithi057 ceph-mon[40588]: pgmap v712: 324 pgs: 324 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 3.0 KiB/s wr, 18 op/s 2024-06-11T13:39:19.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:18 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-83","var": "dedup_tier","val": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:39:19.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:18 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-11T13:39:19.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:18 smithi057 ceph-mon[40588]: osdmap e465: 8 total, 8 up, 8 in 2024-06-11T13:39:19.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:18 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-11T13:39:19.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:18 smithi107 ceph-mon[33178]: pgmap v712: 324 pgs: 324 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 3.0 KiB/s wr, 18 op/s 2024-06-11T13:39:19.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:18 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-83","var": "dedup_tier","val": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:39:19.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:18 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-11T13:39:19.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:18 smithi107 ceph-mon[33178]: osdmap e465: 8 total, 8 up, 8 in 2024-06-11T13:39:19.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:18 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-11T13:39:20.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:19 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-06-11T13:39:20.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:19 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-11T13:39:20.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:19 smithi057 ceph-mon[29805]: osdmap e466: 8 total, 8 up, 8 in 2024-06-11T13:39:20.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:19 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-11T13:39:20.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:19 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-06-11T13:39:20.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:19 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-06-11T13:39:20.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:19 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-11T13:39:20.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:19 smithi057 ceph-mon[40588]: osdmap e466: 8 total, 8 up, 8 in 2024-06-11T13:39:20.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:19 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-11T13:39:20.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:19 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-06-11T13:39:20.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:19 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-06-11T13:39:20.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:19 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-11T13:39:20.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:19 smithi107 ceph-mon[33178]: osdmap e466: 8 total, 8 up, 8 in 2024-06-11T13:39:20.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:19 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-11T13:39:20.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:19 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-06-11T13:39:21.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:20 smithi057 ceph-mon[29805]: pgmap v715: 324 pgs: 324 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 5.2 KiB/s wr, 19 op/s 2024-06-11T13:39:21.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:20 smithi057 ceph-mon[29805]: osdmap e467: 8 total, 8 up, 8 in 2024-06-11T13:39:21.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:20 smithi057 ceph-mon[40588]: pgmap v715: 324 pgs: 324 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 5.2 KiB/s wr, 19 op/s 2024-06-11T13:39:21.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:20 smithi057 ceph-mon[40588]: osdmap e467: 8 total, 8 up, 8 in 2024-06-11T13:39:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:20 smithi107 ceph-mon[33178]: pgmap v715: 324 pgs: 324 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 5.2 KiB/s wr, 19 op/s 2024-06-11T13:39:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:20 smithi107 ceph-mon[33178]: osdmap e467: 8 total, 8 up, 8 in 2024-06-11T13:39:22.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:21 smithi107 ceph-mon[33178]: osdmap e468: 8 total, 8 up, 8 in 2024-06-11T13:39:22.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:21 smithi107 ceph-mon[33178]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:39:22.049 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:39:21 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:39:22.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:21 smithi057 ceph-mon[29805]: osdmap e468: 8 total, 8 up, 8 in 2024-06-11T13:39:22.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:21 smithi057 ceph-mon[29805]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:39:22.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:21 smithi057 ceph-mon[40588]: osdmap e468: 8 total, 8 up, 8 in 2024-06-11T13:39:22.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:21 smithi057 ceph-mon[40588]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:39:23.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:22 smithi107 ceph-mon[33178]: pgmap v718: 324 pgs: 324 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-06-11T13:39:23.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:22 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:39:23.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:22 smithi107 ceph-mon[33178]: osdmap e469: 8 total, 8 up, 8 in 2024-06-11T13:39:23.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:22 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:23.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:22 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:23.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:22 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-83"}]: dispatch 2024-06-11T13:39:23.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:22 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-83"}]: dispatch 2024-06-11T13:39:23.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:22 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:39:23.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:22 smithi057 ceph-mon[29805]: pgmap v718: 324 pgs: 324 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-06-11T13:39:23.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:22 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:39:23.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:22 smithi057 ceph-mon[29805]: osdmap e469: 8 total, 8 up, 8 in 2024-06-11T13:39:23.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:22 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:23.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:22 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:23.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:22 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-83"}]: dispatch 2024-06-11T13:39:23.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:22 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-83"}]: dispatch 2024-06-11T13:39:23.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:22 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:39:23.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:22 smithi057 ceph-mon[40588]: pgmap v718: 324 pgs: 324 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-06-11T13:39:23.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:22 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:39:23.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:22 smithi057 ceph-mon[40588]: osdmap e469: 8 total, 8 up, 8 in 2024-06-11T13:39:23.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:22 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:23.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:22 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:23.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:22 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-83"}]: dispatch 2024-06-11T13:39:23.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:22 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-83"}]: dispatch 2024-06-11T13:39:23.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:22 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:39:24.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:23 smithi057 ceph-mon[29805]: osdmap e470: 8 total, 8 up, 8 in 2024-06-11T13:39:24.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:23 smithi057 ceph-mon[40588]: osdmap e470: 8 total, 8 up, 8 in 2024-06-11T13:39:24.035 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:39:23 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:39:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:39:24.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:23 smithi107 ceph-mon[33178]: osdmap e470: 8 total, 8 up, 8 in 2024-06-11T13:39:25.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:24 smithi057 ceph-mon[40588]: pgmap v721: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-06-11T13:39:25.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:24 smithi057 ceph-mon[40588]: osdmap e471: 8 total, 8 up, 8 in 2024-06-11T13:39:25.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:24 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:39:25.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:24 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:39:25.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:24 smithi057 ceph-mon[29805]: pgmap v721: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-06-11T13:39:25.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:24 smithi057 ceph-mon[29805]: osdmap e471: 8 total, 8 up, 8 in 2024-06-11T13:39:25.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:24 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:39:25.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:24 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:39:25.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:24 smithi107 ceph-mon[33178]: pgmap v721: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-06-11T13:39:25.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:24 smithi107 ceph-mon[33178]: osdmap e471: 8 total, 8 up, 8 in 2024-06-11T13:39:25.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:24 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:39:25.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:24 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:39:26.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:25 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:39:26.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:25 smithi057 ceph-mon[29805]: osdmap e472: 8 total, 8 up, 8 in 2024-06-11T13:39:26.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:25 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:39:26.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:25 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:39:26.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:25 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:39:26.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:25 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:39:26.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:25 smithi057 ceph-mon[40588]: osdmap e472: 8 total, 8 up, 8 in 2024-06-11T13:39:26.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:25 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:39:26.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:25 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:39:26.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:25 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:39:26.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:25 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:39:26.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:25 smithi107 ceph-mon[33178]: osdmap e472: 8 total, 8 up, 8 in 2024-06-11T13:39:26.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:25 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:39:26.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:25 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:39:26.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:25 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:39:27.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:26 smithi057 ceph-mon[29805]: pgmap v724: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:39:27.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:26 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-11T13:39:27.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:26 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-85","var": "dedup_tier","val": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:27.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:26 smithi057 ceph-mon[29805]: osdmap e473: 8 total, 8 up, 8 in 2024-06-11T13:39:27.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:26 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-85","var": "dedup_tier","val": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:27.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:26 smithi057 ceph-mon[40588]: pgmap v724: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:39:27.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:26 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-11T13:39:27.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:26 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-85","var": "dedup_tier","val": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:27.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:26 smithi057 ceph-mon[40588]: osdmap e473: 8 total, 8 up, 8 in 2024-06-11T13:39:27.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:26 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-85","var": "dedup_tier","val": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:27.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:26 smithi107 ceph-mon[33178]: pgmap v724: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:39:27.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:26 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-11T13:39:27.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:26 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-85","var": "dedup_tier","val": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:27.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:26 smithi107 ceph-mon[33178]: osdmap e473: 8 total, 8 up, 8 in 2024-06-11T13:39:27.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:26 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-85","var": "dedup_tier","val": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:28.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-85","var": "dedup_tier","val": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:39:28.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-11T13:39:28.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[29805]: osdmap e474: 8 total, 8 up, 8 in 2024-06-11T13:39:28.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-11T13:39:28.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:39:28.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[29805]: pgmap v727: 324 pgs: 324 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-06-11T13:39:28.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[29805]: Adjusting osd_memory_target on smithi057 to 3249M 2024-06-11T13:39:28.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[29805]: Adjusting osd_memory_target on smithi107 to 3248M 2024-06-11T13:39:28.288 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-85","var": "dedup_tier","val": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:39:28.288 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-11T13:39:28.288 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[40588]: osdmap e474: 8 total, 8 up, 8 in 2024-06-11T13:39:28.288 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-11T13:39:28.288 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:39:28.288 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[40588]: pgmap v727: 324 pgs: 324 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-06-11T13:39:28.288 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.288 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.289 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.289 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.289 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.289 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.289 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.289 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.289 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.289 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.289 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.289 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.289 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.289 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.290 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.290 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.290 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[40588]: Adjusting osd_memory_target on smithi057 to 3249M 2024-06-11T13:39:28.290 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[40588]: Adjusting osd_memory_target on smithi107 to 3248M 2024-06-11T13:39:28.290 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:39:28.290 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:39:28.290 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:39:28.290 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:39:28.290 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:39:28.290 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-06-11T13:39:28.290 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[40588]: osdmap e475: 8 total, 8 up, 8 in 2024-06-11T13:39:28.290 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-11T13:39:28.290 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-11T13:39:28.291 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:39:28.291 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:39:28.291 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:39:28.291 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:39:28.291 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:39:28.291 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-06-11T13:39:28.291 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[29805]: osdmap e475: 8 total, 8 up, 8 in 2024-06-11T13:39:28.291 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-11T13:39:28.291 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:27 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-11T13:39:28.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:27 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-85","var": "dedup_tier","val": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:39:28.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:27 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-11T13:39:28.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:27 smithi107 ceph-mon[33178]: osdmap e474: 8 total, 8 up, 8 in 2024-06-11T13:39:28.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:27 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-11T13:39:28.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:27 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:39:28.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:27 smithi107 ceph-mon[33178]: pgmap v727: 324 pgs: 324 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-06-11T13:39:28.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:27 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:27 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:27 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:27 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:27 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:27 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:27 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:27 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:27 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:27 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:27 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:27 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:27 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:27 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:27 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:27 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-11T13:39:28.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:27 smithi107 ceph-mon[33178]: Adjusting osd_memory_target on smithi057 to 3249M 2024-06-11T13:39:28.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:27 smithi107 ceph-mon[33178]: Adjusting osd_memory_target on smithi107 to 3248M 2024-06-11T13:39:28.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:27 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:39:28.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:27 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:39:28.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:27 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:39:28.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:27 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:39:28.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:27 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:39:28.301 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:27 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-06-11T13:39:28.301 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:27 smithi107 ceph-mon[33178]: osdmap e475: 8 total, 8 up, 8 in 2024-06-11T13:39:28.301 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:27 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-11T13:39:28.301 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:27 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-11T13:39:30.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:29 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-06-11T13:39:30.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:29 smithi057 ceph-mon[29805]: osdmap e476: 8 total, 8 up, 8 in 2024-06-11T13:39:30.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:29 smithi057 ceph-mon[29805]: pgmap v730: 324 pgs: 324 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-06-11T13:39:30.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:29 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-06-11T13:39:30.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:29 smithi057 ceph-mon[40588]: osdmap e476: 8 total, 8 up, 8 in 2024-06-11T13:39:30.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:29 smithi057 ceph-mon[40588]: pgmap v730: 324 pgs: 324 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-06-11T13:39:30.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:29 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-06-11T13:39:30.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:29 smithi107 ceph-mon[33178]: osdmap e476: 8 total, 8 up, 8 in 2024-06-11T13:39:30.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:29 smithi107 ceph-mon[33178]: pgmap v730: 324 pgs: 324 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-06-11T13:39:31.199 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:30 smithi107 ceph-mon[33178]: osdmap e477: 8 total, 8 up, 8 in 2024-06-11T13:39:31.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:30 smithi057 ceph-mon[29805]: osdmap e477: 8 total, 8 up, 8 in 2024-06-11T13:39:31.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:30 smithi057 ceph-mon[40588]: osdmap e477: 8 total, 8 up, 8 in 2024-06-11T13:39:31.893 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:39:31 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:39:32.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:31 smithi057 ceph-mon[40588]: osdmap e478: 8 total, 8 up, 8 in 2024-06-11T13:39:32.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:31 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:32.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:31 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:32.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:31 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-85"}]: dispatch 2024-06-11T13:39:32.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:31 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-85"}]: dispatch 2024-06-11T13:39:32.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:31 smithi057 ceph-mon[40588]: pgmap v733: 324 pgs: 324 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 7.5 KiB/s wr, 12 op/s 2024-06-11T13:39:32.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:31 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:39:32.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:31 smithi057 ceph-mon[29805]: osdmap e478: 8 total, 8 up, 8 in 2024-06-11T13:39:32.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:31 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:32.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:31 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:32.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:31 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-85"}]: dispatch 2024-06-11T13:39:32.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:31 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-85"}]: dispatch 2024-06-11T13:39:32.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:31 smithi057 ceph-mon[29805]: pgmap v733: 324 pgs: 324 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 7.5 KiB/s wr, 12 op/s 2024-06-11T13:39:32.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:31 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:39:32.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:31 smithi107 ceph-mon[33178]: osdmap e478: 8 total, 8 up, 8 in 2024-06-11T13:39:32.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:31 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:32.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:31 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:32.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:31 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-85"}]: dispatch 2024-06-11T13:39:32.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:31 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-85"}]: dispatch 2024-06-11T13:39:32.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:31 smithi107 ceph-mon[33178]: pgmap v733: 324 pgs: 324 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 7.5 KiB/s wr, 12 op/s 2024-06-11T13:39:32.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:31 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:39:33.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:32 smithi057 ceph-mon[29805]: osdmap e479: 8 total, 8 up, 8 in 2024-06-11T13:39:33.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:32 smithi057 ceph-mon[40588]: osdmap e479: 8 total, 8 up, 8 in 2024-06-11T13:39:33.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:32 smithi107 ceph-mon[33178]: osdmap e479: 8 total, 8 up, 8 in 2024-06-11T13:39:33.924 INFO:tasks.workunit.client.0.smithi057.stdout: OK ] LibRadosTwoPoolsPP.ProxyRead (18215 ms) 2024-06-11T13:39:33.924 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.CachePin 2024-06-11T13:39:33.924 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.CachePin (22766 ms) 2024-06-11T13:39:33.924 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.SetRedirectRead 2024-06-11T13:39:33.924 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.SetRedirectRead (3024 ms) 2024-06-11T13:39:33.924 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestPromoteRead 2024-06-11T13:39:33.924 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: require_osd_release = squid 2024-06-11T13:39:33.924 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestPromoteRead (3006 ms) 2024-06-11T13:39:33.925 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRefRead 2024-06-11T13:39:33.925 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRefRead (3051 ms) 2024-06-11T13:39:33.925 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestUnset 2024-06-11T13:39:33.925 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: require_osd_release = squid 2024-06-11T13:39:33.925 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestUnset (3048 ms) 2024-06-11T13:39:33.925 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2024-06-11T13:39:33.925 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: require_osd_release = squid 2024-06-11T13:39:33.925 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestDedupRefRead (4043 ms) 2024-06-11T13:39:33.925 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2024-06-11T13:39:33.925 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: require_osd_release = squid 2024-06-11T13:39:33.925 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount (39498 ms) 2024-06-11T13:39:33.925 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2024-06-11T13:39:33.926 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: require_osd_release = squid 2024-06-11T13:39:33.926 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 (16177 ms) 2024-06-11T13:39:33.926 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2024-06-11T13:39:33.926 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: require_osd_release = squid 2024-06-11T13:39:33.926 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestTestSnapCreate (4059 ms) 2024-06-11T13:39:33.926 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2024-06-11T13:39:33.926 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: require_osd_release = squid 2024-06-11T13:39:33.926 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote (3042 ms) 2024-06-11T13:39:33.926 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2024-06-11T13:39:33.926 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: require_osd_release = squid 2024-06-11T13:39:33.926 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification (24171 ms) 2024-06-11T13:39:33.926 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2024-06-11T13:39:33.926 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: require_osd_release = squid 2024-06-11T13:39:33.927 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapIncCount (15076 ms) 2024-06-11T13:39:33.927 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvict 2024-06-11T13:39:33.927 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: require_osd_release = squid 2024-06-11T13:39:33.927 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvict (5057 ms) 2024-06-11T13:39:33.927 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictPromote 2024-06-11T13:39:33.927 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: require_osd_release = squid 2024-06-11T13:39:33.927 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictPromote (4058 ms) 2024-06-11T13:39:33.927 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2024-06-11T13:39:33.927 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: require_osd_release = squid 2024-06-11T13:39:33.927 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: waiting for scrubs... 2024-06-11T13:39:33.927 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: done waiting 2024-06-11T13:39:33.927 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch (24736 ms) 2024-06-11T13:39:33.927 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.DedupFlushRead 2024-06-11T13:39:33.928 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: require_osd_release = squid 2024-06-11T13:39:33.928 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.DedupFlushRead (10185 ms) 2024-06-11T13:39:33.928 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushSnap 2024-06-11T13:39:33.928 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: require_osd_release = squid 2024-06-11T13:39:33.928 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushSnap (9098 ms) 2024-06-11T13:39:33.928 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2024-06-11T13:39:33.928 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: require_osd_release = squid 2024-06-11T13:39:33.928 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushDupCount (9098 ms) 2024-06-11T13:39:33.928 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2024-06-11T13:39:34.034 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:33 smithi057 ceph-mon[29805]: osdmap e480: 8 total, 8 up, 8 in 2024-06-11T13:39:34.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:33 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:39:34.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:33 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:39:34.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:33 smithi057 ceph-mon[29805]: pgmap v736: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-06-11T13:39:34.035 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:39:33 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:39:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:39:34.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:33 smithi057 ceph-mon[40588]: osdmap e480: 8 total, 8 up, 8 in 2024-06-11T13:39:34.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:33 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:39:34.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:33 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:39:34.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:33 smithi057 ceph-mon[40588]: pgmap v736: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-06-11T13:39:34.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:33 smithi107 ceph-mon[33178]: osdmap e480: 8 total, 8 up, 8 in 2024-06-11T13:39:34.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:33 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:39:34.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:33 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:39:34.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:33 smithi107 ceph-mon[33178]: pgmap v736: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-06-11T13:39:35.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:34 smithi057 ceph-mon[29805]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:39:35.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:34 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:39:35.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:34 smithi057 ceph-mon[29805]: osdmap e481: 8 total, 8 up, 8 in 2024-06-11T13:39:35.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:34 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:39:35.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:34 smithi057 ceph-mon[40588]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:39:35.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:34 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:39:35.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:34 smithi057 ceph-mon[40588]: osdmap e481: 8 total, 8 up, 8 in 2024-06-11T13:39:35.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:34 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:39:35.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:34 smithi107 ceph-mon[33178]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:39:35.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:34 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:39:35.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:34 smithi107 ceph-mon[33178]: osdmap e481: 8 total, 8 up, 8 in 2024-06-11T13:39:35.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:34 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:39:36.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:35 smithi057 ceph-mon[29805]: osdmap e482: 8 total, 8 up, 8 in 2024-06-11T13:39:36.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:35 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:39:36.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:35 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:39:36.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:35 smithi057 ceph-mon[29805]: pgmap v739: 356 pgs: 64 unknown, 292 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:39:36.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:35 smithi057 ceph-mon[40588]: osdmap e482: 8 total, 8 up, 8 in 2024-06-11T13:39:36.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:35 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:39:36.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:35 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:39:36.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:35 smithi057 ceph-mon[40588]: pgmap v739: 356 pgs: 64 unknown, 292 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:39:36.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:35 smithi107 ceph-mon[33178]: osdmap e482: 8 total, 8 up, 8 in 2024-06-11T13:39:36.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:35 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:39:36.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:35 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:39:36.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:35 smithi107 ceph-mon[33178]: pgmap v739: 356 pgs: 64 unknown, 292 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:39:37.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:36 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-11T13:39:37.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:36 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-87","var": "dedup_tier","val": "test-rados-api-smithi057-77599-89-test-flush"}]: dispatch 2024-06-11T13:39:37.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:36 smithi057 ceph-mon[29805]: osdmap e483: 8 total, 8 up, 8 in 2024-06-11T13:39:37.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:36 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-87","var": "dedup_tier","val": "test-rados-api-smithi057-77599-89-test-flush"}]: dispatch 2024-06-11T13:39:37.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:36 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-11T13:39:37.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:36 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-87","var": "dedup_tier","val": "test-rados-api-smithi057-77599-89-test-flush"}]: dispatch 2024-06-11T13:39:37.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:36 smithi057 ceph-mon[40588]: osdmap e483: 8 total, 8 up, 8 in 2024-06-11T13:39:37.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:36 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-87","var": "dedup_tier","val": "test-rados-api-smithi057-77599-89-test-flush"}]: dispatch 2024-06-11T13:39:37.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:36 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-11T13:39:37.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:36 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-87","var": "dedup_tier","val": "test-rados-api-smithi057-77599-89-test-flush"}]: dispatch 2024-06-11T13:39:37.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:36 smithi107 ceph-mon[33178]: osdmap e483: 8 total, 8 up, 8 in 2024-06-11T13:39:37.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:36 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-87","var": "dedup_tier","val": "test-rados-api-smithi057-77599-89-test-flush"}]: dispatch 2024-06-11T13:39:38.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:37 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-87","var": "dedup_tier","val": "test-rados-api-smithi057-77599-89-test-flush"}]': finished 2024-06-11T13:39:38.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:37 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-11T13:39:38.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:37 smithi057 ceph-mon[40588]: osdmap e484: 8 total, 8 up, 8 in 2024-06-11T13:39:38.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:37 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-11T13:39:38.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:37 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:39:38.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:37 smithi057 ceph-mon[40588]: pgmap v742: 356 pgs: 356 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:39:38.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:37 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-06-11T13:39:38.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:37 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-11T13:39:38.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:37 smithi057 ceph-mon[40588]: osdmap e485: 8 total, 8 up, 8 in 2024-06-11T13:39:38.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:37 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-11T13:39:38.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:37 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-87","var": "dedup_tier","val": "test-rados-api-smithi057-77599-89-test-flush"}]': finished 2024-06-11T13:39:38.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:37 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-11T13:39:38.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:37 smithi057 ceph-mon[29805]: osdmap e484: 8 total, 8 up, 8 in 2024-06-11T13:39:38.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:37 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-11T13:39:38.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:37 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:39:38.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:37 smithi057 ceph-mon[29805]: pgmap v742: 356 pgs: 356 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:39:38.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:37 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-06-11T13:39:38.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:37 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-11T13:39:38.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:37 smithi057 ceph-mon[29805]: osdmap e485: 8 total, 8 up, 8 in 2024-06-11T13:39:38.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:37 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-11T13:39:38.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:37 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-87","var": "dedup_tier","val": "test-rados-api-smithi057-77599-89-test-flush"}]': finished 2024-06-11T13:39:38.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:37 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-11T13:39:38.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:37 smithi107 ceph-mon[33178]: osdmap e484: 8 total, 8 up, 8 in 2024-06-11T13:39:38.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:37 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-11T13:39:38.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:37 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:39:38.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:37 smithi107 ceph-mon[33178]: pgmap v742: 356 pgs: 356 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:39:38.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:37 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-06-11T13:39:38.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:37 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-11T13:39:38.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:37 smithi107 ceph-mon[33178]: osdmap e485: 8 total, 8 up, 8 in 2024-06-11T13:39:38.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:37 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-11T13:39:40.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:39 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-06-11T13:39:40.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:39 smithi057 ceph-mon[29805]: osdmap e486: 8 total, 8 up, 8 in 2024-06-11T13:39:40.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:39 smithi057 ceph-mon[29805]: pgmap v745: 356 pgs: 356 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:39:40.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:39 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-06-11T13:39:40.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:39 smithi057 ceph-mon[40588]: osdmap e486: 8 total, 8 up, 8 in 2024-06-11T13:39:40.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:39 smithi057 ceph-mon[40588]: pgmap v745: 356 pgs: 356 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:39:40.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:39 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-06-11T13:39:40.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:39 smithi107 ceph-mon[33178]: osdmap e486: 8 total, 8 up, 8 in 2024-06-11T13:39:40.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:39 smithi107 ceph-mon[33178]: pgmap v745: 356 pgs: 356 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:39:41.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:41 smithi057 ceph-mon[29805]: osdmap e487: 8 total, 8 up, 8 in 2024-06-11T13:39:41.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:41 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:41.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:41 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:41.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:41 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-87"}]: dispatch 2024-06-11T13:39:41.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:41 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-87"}]: dispatch 2024-06-11T13:39:41.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:41 smithi057 ceph-mon[40588]: osdmap e487: 8 total, 8 up, 8 in 2024-06-11T13:39:41.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:41 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:41.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:41 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:41.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:41 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-87"}]: dispatch 2024-06-11T13:39:41.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:41 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-87"}]: dispatch 2024-06-11T13:39:41.798 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:39:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:39:41.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:41 smithi107 ceph-mon[33178]: osdmap e487: 8 total, 8 up, 8 in 2024-06-11T13:39:41.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:41 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:41.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:41 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:41.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:41 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-87"}]: dispatch 2024-06-11T13:39:41.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:41 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-87"}]: dispatch 2024-06-11T13:39:42.715 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:42 smithi107 ceph-mon[33178]: osdmap e488: 8 total, 8 up, 8 in 2024-06-11T13:39:42.716 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:42 smithi107 ceph-mon[33178]: pgmap v748: 292 pgs: 292 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-06-11T13:39:42.716 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:42 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:39:42.716 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:42 smithi107 ceph-mon[33178]: osdmap e489: 8 total, 8 up, 8 in 2024-06-11T13:39:42.716 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:42 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:39:42.716 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:42 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:39:42.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:42 smithi057 ceph-mon[29805]: osdmap e488: 8 total, 8 up, 8 in 2024-06-11T13:39:42.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:42 smithi057 ceph-mon[29805]: pgmap v748: 292 pgs: 292 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-06-11T13:39:42.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:42 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:39:42.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:42 smithi057 ceph-mon[29805]: osdmap e489: 8 total, 8 up, 8 in 2024-06-11T13:39:42.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:42 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:39:42.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:42 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:39:42.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:42 smithi057 ceph-mon[40588]: osdmap e488: 8 total, 8 up, 8 in 2024-06-11T13:39:42.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:42 smithi057 ceph-mon[40588]: pgmap v748: 292 pgs: 292 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-06-11T13:39:42.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:42 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:39:42.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:42 smithi057 ceph-mon[40588]: osdmap e489: 8 total, 8 up, 8 in 2024-06-11T13:39:42.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:42 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:39:42.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:42 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:39:43.048 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:39:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=cleanup t=2024-06-11T13:39:42.718593288Z level=info msg="Completed cleanup jobs" duration=4.488129ms 2024-06-11T13:39:43.733 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:43 smithi057 ceph-mon[40588]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:39:43.733 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:43 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:39:43.733 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:43 smithi057 ceph-mon[40588]: osdmap e490: 8 total, 8 up, 8 in 2024-06-11T13:39:43.733 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:43 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:39:43.733 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:43 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:39:43.733 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:43 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:39:43.733 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:43 smithi057 ceph-mon[29805]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:39:43.734 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:43 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:39:43.734 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:43 smithi057 ceph-mon[29805]: osdmap e490: 8 total, 8 up, 8 in 2024-06-11T13:39:43.734 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:43 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:39:43.734 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:43 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:39:43.734 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:43 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:39:43.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:43 smithi107 ceph-mon[33178]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:39:43.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:43 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:39:43.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:43 smithi107 ceph-mon[33178]: osdmap e490: 8 total, 8 up, 8 in 2024-06-11T13:39:43.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:43 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:39:43.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:43 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:39:43.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:43 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:39:44.035 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:39:43 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:39:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:39:44.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:44 smithi057 ceph-mon[29805]: pgmap v751: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-06-11T13:39:44.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:44 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-11T13:39:44.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:44 smithi057 ceph-mon[29805]: osdmap e491: 8 total, 8 up, 8 in 2024-06-11T13:39:44.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:44 smithi057 ceph-mon[40588]: pgmap v751: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-06-11T13:39:44.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:44 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-11T13:39:44.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:44 smithi057 ceph-mon[40588]: osdmap e491: 8 total, 8 up, 8 in 2024-06-11T13:39:44.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:44 smithi107 ceph-mon[33178]: pgmap v751: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-06-11T13:39:44.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:44 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-11T13:39:44.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:44 smithi107 ceph-mon[33178]: osdmap e491: 8 total, 8 up, 8 in 2024-06-11T13:39:46.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:46 smithi057 ceph-mon[29805]: osdmap e492: 8 total, 8 up, 8 in 2024-06-11T13:39:46.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:46 smithi057 ceph-mon[29805]: pgmap v754: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:39:46.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:46 smithi057 ceph-mon[40588]: osdmap e492: 8 total, 8 up, 8 in 2024-06-11T13:39:46.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:46 smithi057 ceph-mon[40588]: pgmap v754: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:39:46.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:46 smithi107 ceph-mon[33178]: osdmap e492: 8 total, 8 up, 8 in 2024-06-11T13:39:46.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:46 smithi107 ceph-mon[33178]: pgmap v754: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:39:47.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:47 smithi057 ceph-mon[40588]: osdmap e493: 8 total, 8 up, 8 in 2024-06-11T13:39:47.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:47 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:47.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:47 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:47.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:47 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-90"}]: dispatch 2024-06-11T13:39:47.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:47 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-90"}]: dispatch 2024-06-11T13:39:47.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:47 smithi057 ceph-mon[29805]: osdmap e493: 8 total, 8 up, 8 in 2024-06-11T13:39:47.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:47 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:47.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:47 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:47.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:47 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-90"}]: dispatch 2024-06-11T13:39:47.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:47 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-90"}]: dispatch 2024-06-11T13:39:47.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:47 smithi107 ceph-mon[33178]: osdmap e493: 8 total, 8 up, 8 in 2024-06-11T13:39:47.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:47 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:47.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:47 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:47.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:47 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-90"}]: dispatch 2024-06-11T13:39:47.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:47 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-90"}]: dispatch 2024-06-11T13:39:48.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:48 smithi107 ceph-mon[33178]: osdmap e494: 8 total, 8 up, 8 in 2024-06-11T13:39:48.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:48 smithi107 ceph-mon[33178]: pgmap v757: 292 pgs: 292 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-06-11T13:39:48.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:48 smithi057 ceph-mon[29805]: osdmap e494: 8 total, 8 up, 8 in 2024-06-11T13:39:48.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:48 smithi057 ceph-mon[29805]: pgmap v757: 292 pgs: 292 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-06-11T13:39:48.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:48 smithi057 ceph-mon[40588]: osdmap e494: 8 total, 8 up, 8 in 2024-06-11T13:39:48.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:48 smithi057 ceph-mon[40588]: pgmap v757: 292 pgs: 292 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-06-11T13:39:49.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:49 smithi057 ceph-mon[29805]: osdmap e495: 8 total, 8 up, 8 in 2024-06-11T13:39:49.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:49 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:39:49.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:49 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:39:49.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:49 smithi057 ceph-mon[40588]: osdmap e495: 8 total, 8 up, 8 in 2024-06-11T13:39:49.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:49 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:39:49.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:49 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:39:49.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:49 smithi107 ceph-mon[33178]: osdmap e495: 8 total, 8 up, 8 in 2024-06-11T13:39:49.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:49 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:39:49.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:49 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:39:50.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:50 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:39:50.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:50 smithi057 ceph-mon[29805]: osdmap e496: 8 total, 8 up, 8 in 2024-06-11T13:39:50.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:50 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:39:50.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:50 smithi057 ceph-mon[29805]: pgmap v760: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-06-11T13:39:50.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:50 smithi057 ceph-mon[29805]: osdmap e497: 8 total, 8 up, 8 in 2024-06-11T13:39:50.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:50 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:39:50.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:50 smithi057 ceph-mon[40588]: osdmap e496: 8 total, 8 up, 8 in 2024-06-11T13:39:50.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:50 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:39:50.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:50 smithi057 ceph-mon[40588]: pgmap v760: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-06-11T13:39:50.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:50 smithi057 ceph-mon[40588]: osdmap e497: 8 total, 8 up, 8 in 2024-06-11T13:39:50.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:50 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:39:50.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:50 smithi107 ceph-mon[33178]: osdmap e496: 8 total, 8 up, 8 in 2024-06-11T13:39:50.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:50 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:39:50.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:50 smithi107 ceph-mon[33178]: pgmap v760: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-06-11T13:39:50.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:50 smithi107 ceph-mon[33178]: osdmap e497: 8 total, 8 up, 8 in 2024-06-11T13:39:52.048 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:39:51 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:39:52.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:52 smithi057 ceph-mon[40588]: osdmap e498: 8 total, 8 up, 8 in 2024-06-11T13:39:52.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:52 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:52.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:52 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:52.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:52 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-92"}]: dispatch 2024-06-11T13:39:52.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:52 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-92"}]: dispatch 2024-06-11T13:39:52.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:52 smithi057 ceph-mon[40588]: pgmap v763: 324 pgs: 324 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-06-11T13:39:52.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:52 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:39:52.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:52 smithi057 ceph-mon[29805]: osdmap e498: 8 total, 8 up, 8 in 2024-06-11T13:39:52.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:52 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:52.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:52 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:52.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:52 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-92"}]: dispatch 2024-06-11T13:39:52.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:52 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-92"}]: dispatch 2024-06-11T13:39:52.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:52 smithi057 ceph-mon[29805]: pgmap v763: 324 pgs: 324 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-06-11T13:39:52.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:52 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:39:52.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:52 smithi107 ceph-mon[33178]: osdmap e498: 8 total, 8 up, 8 in 2024-06-11T13:39:52.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:52 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:52.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:52 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:39:52.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:52 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-92"}]: dispatch 2024-06-11T13:39:52.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:52 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-92"}]: dispatch 2024-06-11T13:39:52.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:52 smithi107 ceph-mon[33178]: pgmap v763: 324 pgs: 324 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-06-11T13:39:52.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:52 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:39:53.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:53 smithi057 ceph-mon[40588]: osdmap e499: 8 total, 8 up, 8 in 2024-06-11T13:39:53.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:53 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:39:53.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:53 smithi057 ceph-mon[29805]: osdmap e499: 8 total, 8 up, 8 in 2024-06-11T13:39:53.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:53 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:39:53.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:53 smithi107 ceph-mon[33178]: osdmap e499: 8 total, 8 up, 8 in 2024-06-11T13:39:53.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:53 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:39:54.035 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:39:53 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:39:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:39:54.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:54 smithi057 ceph-mon[40588]: osdmap e500: 8 total, 8 up, 8 in 2024-06-11T13:39:54.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:54 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:39:54.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:54 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:39:54.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:54 smithi057 ceph-mon[40588]: pgmap v766: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-11T13:39:54.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:54 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:39:54.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:54 smithi057 ceph-mon[40588]: osdmap e501: 8 total, 8 up, 8 in 2024-06-11T13:39:54.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:54 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:39:54.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:54 smithi057 ceph-mon[29805]: osdmap e500: 8 total, 8 up, 8 in 2024-06-11T13:39:54.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:54 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:39:54.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:54 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:39:54.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:54 smithi057 ceph-mon[29805]: pgmap v766: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-11T13:39:54.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:54 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:39:54.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:54 smithi057 ceph-mon[29805]: osdmap e501: 8 total, 8 up, 8 in 2024-06-11T13:39:54.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:54 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:39:54.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:54 smithi107 ceph-mon[33178]: osdmap e500: 8 total, 8 up, 8 in 2024-06-11T13:39:54.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:54 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:39:54.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:54 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:39:54.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:54 smithi107 ceph-mon[33178]: pgmap v766: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-11T13:39:54.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:54 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:39:54.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:54 smithi107 ceph-mon[33178]: osdmap e501: 8 total, 8 up, 8 in 2024-06-11T13:39:54.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:54 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:39:55.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:55 smithi057 ceph-mon[40588]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:39:55.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:55 smithi057 ceph-mon[40588]: osdmap e502: 8 total, 8 up, 8 in 2024-06-11T13:39:55.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:55 smithi057 ceph-mon[29805]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:39:55.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:55 smithi057 ceph-mon[29805]: osdmap e502: 8 total, 8 up, 8 in 2024-06-11T13:39:55.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:55 smithi107 ceph-mon[33178]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:39:55.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:55 smithi107 ceph-mon[33178]: osdmap e502: 8 total, 8 up, 8 in 2024-06-11T13:39:56.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:56 smithi057 ceph-mon[40588]: pgmap v769: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:39:56.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:56 smithi057 ceph-mon[40588]: osdmap e503: 8 total, 8 up, 8 in 2024-06-11T13:39:56.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:56 smithi057 ceph-mon[29805]: pgmap v769: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:39:56.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:56 smithi057 ceph-mon[29805]: osdmap e503: 8 total, 8 up, 8 in 2024-06-11T13:39:56.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:56 smithi107 ceph-mon[33178]: pgmap v769: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:39:56.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:56 smithi107 ceph-mon[33178]: osdmap e503: 8 total, 8 up, 8 in 2024-06-11T13:39:58.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:39:58 smithi057 ceph-mon[40588]: pgmap v771: 324 pgs: 324 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1.9 KiB/s wr, 3 op/s 2024-06-11T13:39:58.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:39:58 smithi057 ceph-mon[29805]: pgmap v771: 324 pgs: 324 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1.9 KiB/s wr, 3 op/s 2024-06-11T13:39:58.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:39:58 smithi107 ceph-mon[33178]: pgmap v771: 324 pgs: 324 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1.9 KiB/s wr, 3 op/s 2024-06-11T13:40:00.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:00 smithi057 ceph-mon[40588]: pgmap v772: 324 pgs: 324 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.3 KiB/s wr, 2 op/s 2024-06-11T13:40:00.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:00 smithi057 ceph-mon[40588]: Health detail: HEALTH_WARN 4 pool(s) do not have an application enabled 2024-06-11T13:40:00.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:00 smithi057 ceph-mon[40588]: [WRN] POOL_APP_NOT_ENABLED: 4 pool(s) do not have an application enabled 2024-06-11T13:40:00.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:00 smithi057 ceph-mon[40588]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-06-11T13:40:00.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:00 smithi057 ceph-mon[40588]: application not enabled on pool 'test_handler_errorsmithi057-78465-1' 2024-06-11T13:40:00.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:00 smithi057 ceph-mon[40588]: application not enabled on pool 'poolrace.1804289383' 2024-06-11T13:40:00.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:00 smithi057 ceph-mon[40588]: application not enabled on pool 'poolrace2.846930886' 2024-06-11T13:40:00.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:00 smithi057 ceph-mon[40588]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-06-11T13:40:00.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:00 smithi057 ceph-mon[29805]: pgmap v772: 324 pgs: 324 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.3 KiB/s wr, 2 op/s 2024-06-11T13:40:00.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:00 smithi057 ceph-mon[29805]: Health detail: HEALTH_WARN 4 pool(s) do not have an application enabled 2024-06-11T13:40:00.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:00 smithi057 ceph-mon[29805]: [WRN] POOL_APP_NOT_ENABLED: 4 pool(s) do not have an application enabled 2024-06-11T13:40:00.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:00 smithi057 ceph-mon[29805]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-06-11T13:40:00.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:00 smithi057 ceph-mon[29805]: application not enabled on pool 'test_handler_errorsmithi057-78465-1' 2024-06-11T13:40:00.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:00 smithi057 ceph-mon[29805]: application not enabled on pool 'poolrace.1804289383' 2024-06-11T13:40:00.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:00 smithi057 ceph-mon[29805]: application not enabled on pool 'poolrace2.846930886' 2024-06-11T13:40:00.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:00 smithi057 ceph-mon[29805]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-06-11T13:40:00.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:00 smithi107 ceph-mon[33178]: pgmap v772: 324 pgs: 324 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.3 KiB/s wr, 2 op/s 2024-06-11T13:40:00.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:00 smithi107 ceph-mon[33178]: Health detail: HEALTH_WARN 4 pool(s) do not have an application enabled 2024-06-11T13:40:00.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:00 smithi107 ceph-mon[33178]: [WRN] POOL_APP_NOT_ENABLED: 4 pool(s) do not have an application enabled 2024-06-11T13:40:00.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:00 smithi107 ceph-mon[33178]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-06-11T13:40:00.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:00 smithi107 ceph-mon[33178]: application not enabled on pool 'test_handler_errorsmithi057-78465-1' 2024-06-11T13:40:00.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:00 smithi107 ceph-mon[33178]: application not enabled on pool 'poolrace.1804289383' 2024-06-11T13:40:00.800 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:00 smithi107 ceph-mon[33178]: application not enabled on pool 'poolrace2.846930886' 2024-06-11T13:40:00.800 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:00 smithi107 ceph-mon[33178]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-06-11T13:40:01.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:01 smithi057 ceph-mon[40588]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:40:01.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:01 smithi057 ceph-mon[29805]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:40:01.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:01 smithi107 ceph-mon[33178]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:40:01.798 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:40:01 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:40:02.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:02 smithi057 ceph-mon[40588]: pgmap v773: 324 pgs: 324 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-06-11T13:40:02.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:02 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:40:02.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:02 smithi057 ceph-mon[29805]: pgmap v773: 324 pgs: 324 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-06-11T13:40:02.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:02 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:40:02.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:02 smithi107 ceph-mon[33178]: pgmap v773: 324 pgs: 324 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-06-11T13:40:02.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:02 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:40:04.035 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:40:03 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:40:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:40:04.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:04 smithi057 ceph-mon[40588]: pgmap v774: 324 pgs: 324 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-06-11T13:40:04.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:04 smithi057 ceph-mon[29805]: pgmap v774: 324 pgs: 324 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-06-11T13:40:04.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:04 smithi107 ceph-mon[33178]: pgmap v774: 324 pgs: 324 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-06-11T13:40:06.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:06 smithi057 ceph-mon[40588]: pgmap v775: 324 pgs: 324 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 921 B/s wr, 3 op/s 2024-06-11T13:40:06.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:06 smithi057 ceph-mon[29805]: pgmap v775: 324 pgs: 324 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 921 B/s wr, 3 op/s 2024-06-11T13:40:06.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:06 smithi107 ceph-mon[33178]: pgmap v775: 324 pgs: 324 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 921 B/s wr, 3 op/s 2024-06-11T13:40:07.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:07 smithi057 ceph-mon[40588]: osdmap e504: 8 total, 8 up, 8 in 2024-06-11T13:40:07.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:07 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:40:07.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:07 smithi057 ceph-mon[29805]: osdmap e504: 8 total, 8 up, 8 in 2024-06-11T13:40:07.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:07 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:40:07.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:07 smithi107 ceph-mon[33178]: osdmap e504: 8 total, 8 up, 8 in 2024-06-11T13:40:07.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:07 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:40:08.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:08 smithi057 ceph-mon[40588]: pgmap v777: 324 pgs: 324 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 102 B/s wr, 2 op/s 2024-06-11T13:40:08.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:08 smithi057 ceph-mon[29805]: pgmap v777: 324 pgs: 324 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 102 B/s wr, 2 op/s 2024-06-11T13:40:08.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:08 smithi107 ceph-mon[33178]: pgmap v777: 324 pgs: 324 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 102 B/s wr, 2 op/s 2024-06-11T13:40:10.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:10 smithi107 ceph-mon[33178]: pgmap v778: 324 pgs: 324 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 102 B/s wr, 2 op/s 2024-06-11T13:40:11.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:10 smithi057 ceph-mon[40588]: pgmap v778: 324 pgs: 324 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 102 B/s wr, 2 op/s 2024-06-11T13:40:11.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:10 smithi057 ceph-mon[29805]: pgmap v778: 324 pgs: 324 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 102 B/s wr, 2 op/s 2024-06-11T13:40:11.817 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8da00 2024-06-11T13:40:11.817 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8db50 2024-06-11T13:40:11.817 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8dc80 2024-06-11T13:40:11.817 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8ee80 2024-06-11T13:40:11.818 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8f390 2024-06-11T13:40:11.818 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8f4e0 2024-06-11T13:40:11.818 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8e880 2024-06-11T13:40:11.818 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8e9b0 2024-06-11T13:40:11.818 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8eae0 2024-06-11T13:40:11.818 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8f610 2024-06-11T13:40:11.818 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8fae0 2024-06-11T13:40:11.818 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8fc30 2024-06-11T13:40:11.818 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8efd0 2024-06-11T13:40:11.818 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8f100 2024-06-11T13:40:11.818 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8f230 2024-06-11T13:40:11.818 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8fd60 2024-06-11T13:40:11.819 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a90230 2024-06-11T13:40:11.819 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a90380 2024-06-11T13:40:11.819 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8f740 2024-06-11T13:40:11.819 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8f870 2024-06-11T13:40:11.819 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8f9a0 2024-06-11T13:40:11.819 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a904b0 2024-06-11T13:40:11.819 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a90980 2024-06-11T13:40:11.819 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a90ad0 2024-06-11T13:40:11.819 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a910b0 2024-06-11T13:40:11.819 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a905e0 2024-06-11T13:40:11.819 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a90710 2024-06-11T13:40:11.819 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a90840 2024-06-11T13:40:11.820 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a911c0 2024-06-11T13:40:11.820 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a90c00 2024-06-11T13:40:11.820 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a90d50 2024-06-11T13:40:11.820 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a90ea0 2024-06-11T13:40:11.820 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a912f0 2024-06-11T13:40:11.820 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a917c0 2024-06-11T13:40:11.820 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a918d0 2024-06-11T13:40:11.820 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8fe90 2024-06-11T13:40:11.820 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a8ffc0 2024-06-11T13:40:11.820 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a900f0 2024-06-11T13:40:11.820 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a91a20 2024-06-11T13:40:11.820 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a91b50 2024-06-11T13:40:11.820 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a92150 2024-06-11T13:40:11.820 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a91ca0 2024-06-11T13:40:11.821 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a91dd0 2024-06-11T13:40:11.821 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a91f00 2024-06-11T13:40:11.821 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a92030 2024-06-11T13:40:11.821 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a92730 2024-06-11T13:40:11.821 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a92280 2024-06-11T13:40:11.821 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a923b0 2024-06-11T13:40:11.821 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a92500 2024-06-11T13:40:11.821 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a92860 2024-06-11T13:40:11.821 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 0x564078a92d30 2024-06-11T13:40:11.821 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:40:11.821 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:40:11.821 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:40:11.821 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:40:11.821 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:40:11.822 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:40:11.822 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:40:11.822 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:40:11.822 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:40:11.822 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:40:11.822 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:40:11.822 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:40:11.822 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:40:11.822 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:40:11.822 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:40:11.822 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:40:11.822 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:40:11.822 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:40:11.822 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:40:11.823 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:40:11.823 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:40:11.823 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:40:11.823 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:40:11.823 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:40:11.823 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:40:11.823 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:40:11.823 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:40:11.823 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:40:11.823 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:40:11.823 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:40:11.823 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:40:11.823 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:40:11.823 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:40:11.824 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:40:11.824 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:40:11.824 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:40:11.824 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:40:11.824 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:40:12.048 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:40:11 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:40:13.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:12 smithi057 ceph-mon[29805]: pgmap v779: 324 pgs: 324 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:40:13.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:12 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:40:13.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:12 smithi057 ceph-mon[40588]: pgmap v779: 324 pgs: 324 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:40:13.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:12 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:40:13.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:12 smithi107 ceph-mon[33178]: pgmap v779: 324 pgs: 324 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:40:13.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:12 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:40:14.035 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:40:13 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:40:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:40:15.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:14 smithi057 ceph-mon[29805]: pgmap v780: 324 pgs: 324 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:40:15.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:14 smithi057 ceph-mon[40588]: pgmap v780: 324 pgs: 324 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:40:15.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:14 smithi107 ceph-mon[33178]: pgmap v780: 324 pgs: 324 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:40:17.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:16 smithi057 ceph-mon[29805]: pgmap v781: 324 pgs: 324 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:40:17.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:16 smithi057 ceph-mon[29805]: osdmap e505: 8 total, 8 up, 8 in 2024-06-11T13:40:17.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:16 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:40:17.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:16 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:40:17.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:16 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-94"}]: dispatch 2024-06-11T13:40:17.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:16 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-94"}]: dispatch 2024-06-11T13:40:17.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:16 smithi057 ceph-mon[40588]: pgmap v781: 324 pgs: 324 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:40:17.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:16 smithi057 ceph-mon[40588]: osdmap e505: 8 total, 8 up, 8 in 2024-06-11T13:40:17.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:16 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:40:17.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:16 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:40:17.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:16 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-94"}]: dispatch 2024-06-11T13:40:17.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:16 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-94"}]: dispatch 2024-06-11T13:40:17.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:16 smithi107 ceph-mon[33178]: pgmap v781: 324 pgs: 324 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:40:17.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:16 smithi107 ceph-mon[33178]: osdmap e505: 8 total, 8 up, 8 in 2024-06-11T13:40:17.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:16 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:40:17.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:16 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:40:17.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:16 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-94"}]: dispatch 2024-06-11T13:40:17.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:16 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-94"}]: dispatch 2024-06-11T13:40:19.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:18 smithi057 ceph-mon[29805]: pgmap v783: 324 pgs: 324 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:40:19.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:18 smithi057 ceph-mon[29805]: osdmap e506: 8 total, 8 up, 8 in 2024-06-11T13:40:19.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:18 smithi057 ceph-mon[40588]: pgmap v783: 324 pgs: 324 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:40:19.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:18 smithi057 ceph-mon[40588]: osdmap e506: 8 total, 8 up, 8 in 2024-06-11T13:40:19.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:18 smithi107 ceph-mon[33178]: pgmap v783: 324 pgs: 324 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:40:19.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:18 smithi107 ceph-mon[33178]: osdmap e506: 8 total, 8 up, 8 in 2024-06-11T13:40:20.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:19 smithi057 ceph-mon[29805]: osdmap e507: 8 total, 8 up, 8 in 2024-06-11T13:40:20.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:19 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:40:20.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:19 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:40:20.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:19 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:40:20.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:19 smithi057 ceph-mon[29805]: osdmap e508: 8 total, 8 up, 8 in 2024-06-11T13:40:20.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:19 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:40:20.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:19 smithi057 ceph-mon[40588]: osdmap e507: 8 total, 8 up, 8 in 2024-06-11T13:40:20.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:19 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:40:20.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:19 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:40:20.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:19 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:40:20.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:19 smithi057 ceph-mon[40588]: osdmap e508: 8 total, 8 up, 8 in 2024-06-11T13:40:20.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:19 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:40:20.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:19 smithi107 ceph-mon[33178]: osdmap e507: 8 total, 8 up, 8 in 2024-06-11T13:40:20.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:19 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:40:20.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:19 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:40:20.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:19 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:40:20.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:19 smithi107 ceph-mon[33178]: osdmap e508: 8 total, 8 up, 8 in 2024-06-11T13:40:20.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:19 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:40:21.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:20 smithi057 ceph-mon[29805]: pgmap v786: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:40:21.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:20 smithi057 ceph-mon[29805]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:40:21.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:20 smithi057 ceph-mon[29805]: osdmap e509: 8 total, 8 up, 8 in 2024-06-11T13:40:21.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:20 smithi057 ceph-mon[40588]: pgmap v786: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:40:21.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:20 smithi057 ceph-mon[40588]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:40:21.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:20 smithi057 ceph-mon[40588]: osdmap e509: 8 total, 8 up, 8 in 2024-06-11T13:40:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:20 smithi107 ceph-mon[33178]: pgmap v786: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:40:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:20 smithi107 ceph-mon[33178]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:40:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:20 smithi107 ceph-mon[33178]: osdmap e509: 8 total, 8 up, 8 in 2024-06-11T13:40:22.048 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:40:21 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:40:23.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:22 smithi057 ceph-mon[29805]: pgmap v789: 324 pgs: 324 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:40:23.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:22 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:40:23.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:22 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:40:23.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:22 smithi057 ceph-mon[40588]: pgmap v789: 324 pgs: 324 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:40:23.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:22 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:40:23.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:22 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:40:23.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:22 smithi107 ceph-mon[33178]: pgmap v789: 324 pgs: 324 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:40:23.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:22 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:40:23.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:22 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:40:24.034 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:40:23 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:40:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:40:24.936 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:24 smithi057 ceph-mon[40588]: pgmap v790: 324 pgs: 324 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 879 B/s rd, 1.0 KiB/s wr, 2 op/s 2024-06-11T13:40:24.936 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:24 smithi057 ceph-mon[29805]: pgmap v790: 324 pgs: 324 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 879 B/s rd, 1.0 KiB/s wr, 2 op/s 2024-06-11T13:40:25.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:24 smithi107 ceph-mon[33178]: pgmap v790: 324 pgs: 324 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 879 B/s rd, 1.0 KiB/s wr, 2 op/s 2024-06-11T13:40:26.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:26 smithi057 ceph-mon[29805]: pgmap v791: 324 pgs: 324 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 750 B/s rd, 901 B/s wr, 2 op/s 2024-06-11T13:40:26.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:26 smithi057 ceph-mon[29805]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:40:26.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:26 smithi057 ceph-mon[40588]: pgmap v791: 324 pgs: 324 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 750 B/s rd, 901 B/s wr, 2 op/s 2024-06-11T13:40:26.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:26 smithi057 ceph-mon[40588]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:40:27.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:26 smithi107 ceph-mon[33178]: pgmap v791: 324 pgs: 324 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 750 B/s rd, 901 B/s wr, 2 op/s 2024-06-11T13:40:27.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:26 smithi107 ceph-mon[33178]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:40:28.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:28 smithi057 ceph-mon[29805]: pgmap v792: 324 pgs: 324 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-06-11T13:40:28.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:28 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:40:28.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:28 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:40:28.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:28 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:40:28.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:28 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:40:28.671 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:28 smithi057 ceph-mon[40588]: pgmap v792: 324 pgs: 324 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-06-11T13:40:28.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:28 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:40:28.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:28 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:40:28.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:28 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:40:28.672 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:28 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:40:29.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:28 smithi107 ceph-mon[33178]: pgmap v792: 324 pgs: 324 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-06-11T13:40:29.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:28 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:40:29.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:28 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:40:29.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:28 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:40:29.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:28 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:40:31.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:30 smithi057 ceph-mon[40588]: pgmap v793: 324 pgs: 324 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 625 B/s wr, 2 op/s 2024-06-11T13:40:31.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:30 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:40:31.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:30 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:40:31.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:30 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-96"}]: dispatch 2024-06-11T13:40:31.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:30 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-96"}]: dispatch 2024-06-11T13:40:31.035 INFO:journalctl@ceph.osd.2.smithi057.stdout:Jun 11 13:40:30 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-2[56152]: 2024-06-11T13:40:30.617+0000 7fc9506e8640 -1 snap_mapper.add_oid found existing snaps mapped on 111:03a9c41b:test-rados-api-smithi057-77599-97::foo:21, removing 2024-06-11T13:40:31.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:30 smithi057 ceph-mon[29805]: pgmap v793: 324 pgs: 324 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 625 B/s wr, 2 op/s 2024-06-11T13:40:31.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:30 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:40:31.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:30 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:40:31.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:30 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-96"}]: dispatch 2024-06-11T13:40:31.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:30 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-96"}]: dispatch 2024-06-11T13:40:31.048 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 13:40:30 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-5[43766]: 2024-06-11T13:40:30.617+0000 7f697c724640 -1 snap_mapper.add_oid found existing snaps mapped on 111:03a9c41b:test-rados-api-smithi057-77599-97::foo:21, removing 2024-06-11T13:40:31.048 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 13:40:30 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T13:40:30.617+0000 7f319d3d0640 -1 snap_mapper.add_oid found existing snaps mapped on 111:03a9c41b:test-rados-api-smithi057-77599-97::foo:21, removing 2024-06-11T13:40:31.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:30 smithi107 ceph-mon[33178]: pgmap v793: 324 pgs: 324 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 625 B/s wr, 2 op/s 2024-06-11T13:40:31.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:30 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:40:31.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:30 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:40:31.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:30 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-96"}]: dispatch 2024-06-11T13:40:31.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:30 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-96"}]: dispatch 2024-06-11T13:40:31.950 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:31 smithi107 ceph-mon[33178]: osdmap e510: 8 total, 8 up, 8 in 2024-06-11T13:40:31.951 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:40:31 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:40:32.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:31 smithi057 ceph-mon[29805]: osdmap e510: 8 total, 8 up, 8 in 2024-06-11T13:40:32.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:31 smithi057 ceph-mon[40588]: osdmap e510: 8 total, 8 up, 8 in 2024-06-11T13:40:33.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:32 smithi057 ceph-mon[29805]: pgmap v795: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 op/s 2024-06-11T13:40:33.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:32 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:40:33.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:32 smithi057 ceph-mon[29805]: osdmap e511: 8 total, 8 up, 8 in 2024-06-11T13:40:33.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:32 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:40:33.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:32 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:40:33.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:32 smithi057 ceph-mon[40588]: pgmap v795: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 op/s 2024-06-11T13:40:33.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:32 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:40:33.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:32 smithi057 ceph-mon[40588]: osdmap e511: 8 total, 8 up, 8 in 2024-06-11T13:40:33.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:32 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:40:33.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:32 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:40:33.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:32 smithi107 ceph-mon[33178]: pgmap v795: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 op/s 2024-06-11T13:40:33.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:32 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:40:33.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:32 smithi107 ceph-mon[33178]: osdmap e511: 8 total, 8 up, 8 in 2024-06-11T13:40:33.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:32 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:40:33.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:32 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:40:34.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:33 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:40:34.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:33 smithi057 ceph-mon[29805]: osdmap e512: 8 total, 8 up, 8 in 2024-06-11T13:40:34.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:33 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:40:34.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:33 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:40:34.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:40:33 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:40:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:40:34.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:33 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:40:34.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:33 smithi057 ceph-mon[40588]: osdmap e512: 8 total, 8 up, 8 in 2024-06-11T13:40:34.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:33 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:40:34.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:33 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:40:34.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:33 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:40:34.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:33 smithi107 ceph-mon[33178]: osdmap e512: 8 total, 8 up, 8 in 2024-06-11T13:40:34.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:33 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:40:34.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:33 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:40:35.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:34 smithi057 ceph-mon[29805]: pgmap v798: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T13:40:35.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:34 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-98", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:40:35.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:34 smithi057 ceph-mon[29805]: osdmap e513: 8 total, 8 up, 8 in 2024-06-11T13:40:35.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:34 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-98", "mode": "writeback"}]: dispatch 2024-06-11T13:40:35.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:34 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-98", "mode": "writeback"}]: dispatch 2024-06-11T13:40:35.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:34 smithi057 ceph-mon[40588]: pgmap v798: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T13:40:35.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:34 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-98", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:40:35.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:34 smithi057 ceph-mon[40588]: osdmap e513: 8 total, 8 up, 8 in 2024-06-11T13:40:35.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:34 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-98", "mode": "writeback"}]: dispatch 2024-06-11T13:40:35.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:34 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-98", "mode": "writeback"}]: dispatch 2024-06-11T13:40:35.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:34 smithi107 ceph-mon[33178]: pgmap v798: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T13:40:35.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:34 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-98", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:40:35.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:34 smithi107 ceph-mon[33178]: osdmap e513: 8 total, 8 up, 8 in 2024-06-11T13:40:35.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:34 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-98", "mode": "writeback"}]: dispatch 2024-06-11T13:40:35.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:34 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-98", "mode": "writeback"}]: dispatch 2024-06-11T13:40:36.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:35 smithi057 ceph-mon[29805]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:40:36.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:35 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-98", "mode": "writeback"}]': finished 2024-06-11T13:40:36.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:35 smithi057 ceph-mon[29805]: osdmap e514: 8 total, 8 up, 8 in 2024-06-11T13:40:36.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:35 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-98"}]: dispatch 2024-06-11T13:40:36.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:35 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-98"}]: dispatch 2024-06-11T13:40:36.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:35 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-98"}]': finished 2024-06-11T13:40:36.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:35 smithi057 ceph-mon[40588]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:40:36.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:35 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-98", "mode": "writeback"}]': finished 2024-06-11T13:40:36.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:35 smithi057 ceph-mon[40588]: osdmap e514: 8 total, 8 up, 8 in 2024-06-11T13:40:36.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:35 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-98"}]: dispatch 2024-06-11T13:40:36.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:35 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-98"}]: dispatch 2024-06-11T13:40:36.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:35 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-98"}]': finished 2024-06-11T13:40:36.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:35 smithi107 ceph-mon[33178]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:40:36.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:35 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-98", "mode": "writeback"}]': finished 2024-06-11T13:40:36.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:35 smithi107 ceph-mon[33178]: osdmap e514: 8 total, 8 up, 8 in 2024-06-11T13:40:36.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:35 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-98"}]: dispatch 2024-06-11T13:40:36.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:35 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-98"}]: dispatch 2024-06-11T13:40:36.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:35 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-98"}]': finished 2024-06-11T13:40:37.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:36 smithi057 ceph-mon[29805]: pgmap v801: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:40:37.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:36 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-11T13:40:37.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:36 smithi057 ceph-mon[29805]: osdmap e515: 8 total, 8 up, 8 in 2024-06-11T13:40:37.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:36 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-11T13:40:37.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:36 smithi057 ceph-mon[29805]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:40:37.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:36 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-98","var": "hit_set_type","val": "bloom"}]': finished 2024-06-11T13:40:37.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:36 smithi057 ceph-mon[29805]: osdmap e516: 8 total, 8 up, 8 in 2024-06-11T13:40:37.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:36 smithi057 ceph-mon[40588]: pgmap v801: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:40:37.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:36 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-11T13:40:37.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:36 smithi057 ceph-mon[40588]: osdmap e515: 8 total, 8 up, 8 in 2024-06-11T13:40:37.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:36 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-11T13:40:37.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:36 smithi057 ceph-mon[40588]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:40:37.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:36 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-98","var": "hit_set_type","val": "bloom"}]': finished 2024-06-11T13:40:37.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:36 smithi057 ceph-mon[40588]: osdmap e516: 8 total, 8 up, 8 in 2024-06-11T13:40:37.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:36 smithi107 ceph-mon[33178]: pgmap v801: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:40:37.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:36 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-11T13:40:37.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:36 smithi107 ceph-mon[33178]: osdmap e515: 8 total, 8 up, 8 in 2024-06-11T13:40:37.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:36 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-11T13:40:37.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:36 smithi107 ceph-mon[33178]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:40:37.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:36 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-98","var": "hit_set_type","val": "bloom"}]': finished 2024-06-11T13:40:37.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:36 smithi107 ceph-mon[33178]: osdmap e516: 8 total, 8 up, 8 in 2024-06-11T13:40:38.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:37 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-98","var": "hit_set_count","val": "1"}]: dispatch 2024-06-11T13:40:38.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:37 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-98","var": "hit_set_count","val": "1"}]: dispatch 2024-06-11T13:40:38.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:37 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:40:38.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:37 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-98","var": "hit_set_count","val": "1"}]': finished 2024-06-11T13:40:38.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:37 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-98","var": "hit_set_count","val": "1"}]: dispatch 2024-06-11T13:40:38.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:37 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-98","var": "hit_set_count","val": "1"}]: dispatch 2024-06-11T13:40:38.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:37 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:40:38.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:37 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-98","var": "hit_set_count","val": "1"}]': finished 2024-06-11T13:40:38.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:37 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-98","var": "hit_set_count","val": "1"}]: dispatch 2024-06-11T13:40:38.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:37 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-98","var": "hit_set_count","val": "1"}]: dispatch 2024-06-11T13:40:38.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:37 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:40:38.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:37 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-98","var": "hit_set_count","val": "1"}]': finished 2024-06-11T13:40:39.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:38 smithi057 ceph-mon[29805]: pgmap v804: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:40:39.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:38 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-98","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:40:39.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:38 smithi057 ceph-mon[29805]: osdmap e517: 8 total, 8 up, 8 in 2024-06-11T13:40:39.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:38 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-98","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:40:39.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:38 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-98","var": "hit_set_period","val": "600"}]': finished 2024-06-11T13:40:39.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:38 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-98","var": "target_max_objects","val": "250"}]: dispatch 2024-06-11T13:40:39.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:38 smithi057 ceph-mon[29805]: osdmap e518: 8 total, 8 up, 8 in 2024-06-11T13:40:39.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:38 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-98","var": "target_max_objects","val": "250"}]: dispatch 2024-06-11T13:40:39.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:38 smithi057 ceph-mon[40588]: pgmap v804: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:40:39.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:38 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-98","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:40:39.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:38 smithi057 ceph-mon[40588]: osdmap e517: 8 total, 8 up, 8 in 2024-06-11T13:40:39.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:38 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-98","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:40:39.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:38 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-98","var": "hit_set_period","val": "600"}]': finished 2024-06-11T13:40:39.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:38 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-98","var": "target_max_objects","val": "250"}]: dispatch 2024-06-11T13:40:39.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:38 smithi057 ceph-mon[40588]: osdmap e518: 8 total, 8 up, 8 in 2024-06-11T13:40:39.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:38 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-98","var": "target_max_objects","val": "250"}]: dispatch 2024-06-11T13:40:39.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:38 smithi107 ceph-mon[33178]: pgmap v804: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:40:39.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:38 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-98","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:40:39.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:38 smithi107 ceph-mon[33178]: osdmap e517: 8 total, 8 up, 8 in 2024-06-11T13:40:39.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:38 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-98","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:40:39.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:38 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-98","var": "hit_set_period","val": "600"}]': finished 2024-06-11T13:40:39.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:38 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-98","var": "target_max_objects","val": "250"}]: dispatch 2024-06-11T13:40:39.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:38 smithi107 ceph-mon[33178]: osdmap e518: 8 total, 8 up, 8 in 2024-06-11T13:40:39.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:38 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-98","var": "target_max_objects","val": "250"}]: dispatch 2024-06-11T13:40:41.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:40 smithi057 ceph-mon[29805]: pgmap v807: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:40:41.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:40 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-98","var": "target_max_objects","val": "250"}]': finished 2024-06-11T13:40:41.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:40 smithi057 ceph-mon[29805]: osdmap e519: 8 total, 8 up, 8 in 2024-06-11T13:40:41.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:40 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:40:41.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:40 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:40:41.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:40 smithi057 ceph-mon[40588]: pgmap v807: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:40:41.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:40 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-98","var": "target_max_objects","val": "250"}]': finished 2024-06-11T13:40:41.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:40 smithi057 ceph-mon[40588]: osdmap e519: 8 total, 8 up, 8 in 2024-06-11T13:40:41.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:40 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:40:41.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:40 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:40:41.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:40 smithi107 ceph-mon[33178]: pgmap v807: 324 pgs: 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:40:41.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:40 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-98","var": "target_max_objects","val": "250"}]': finished 2024-06-11T13:40:41.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:40 smithi107 ceph-mon[33178]: osdmap e519: 8 total, 8 up, 8 in 2024-06-11T13:40:41.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:40 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:40:41.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:40 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:40:42.048 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:40:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:40:42.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:41 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:40:42.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:41 smithi107 ceph-mon[33178]: osdmap e520: 8 total, 8 up, 8 in 2024-06-11T13:40:42.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:41 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-98"}]: dispatch 2024-06-11T13:40:42.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:41 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-98"}]: dispatch 2024-06-11T13:40:42.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:41 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:40:42.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:41 smithi057 ceph-mon[40588]: osdmap e520: 8 total, 8 up, 8 in 2024-06-11T13:40:42.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:41 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-98"}]: dispatch 2024-06-11T13:40:42.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:41 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-98"}]: dispatch 2024-06-11T13:40:42.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:41 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:40:42.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:41 smithi057 ceph-mon[29805]: osdmap e520: 8 total, 8 up, 8 in 2024-06-11T13:40:42.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:41 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-98"}]: dispatch 2024-06-11T13:40:42.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:41 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-98"}]: dispatch 2024-06-11T13:40:43.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:43 smithi057 ceph-mon[40588]: pgmap v810: 324 pgs: 324 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:40:43.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:43 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:40:43.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:43 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-98"}]': finished 2024-06-11T13:40:43.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:43 smithi057 ceph-mon[40588]: osdmap e521: 8 total, 8 up, 8 in 2024-06-11T13:40:43.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:43 smithi057 ceph-mon[29805]: pgmap v810: 324 pgs: 324 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:40:43.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:43 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:40:43.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:43 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-98"}]': finished 2024-06-11T13:40:43.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:43 smithi057 ceph-mon[29805]: osdmap e521: 8 total, 8 up, 8 in 2024-06-11T13:40:43.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:43 smithi107 ceph-mon[33178]: pgmap v810: 324 pgs: 324 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:40:43.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:43 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:40:43.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:43 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-98"}]': finished 2024-06-11T13:40:43.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:43 smithi107 ceph-mon[33178]: osdmap e521: 8 total, 8 up, 8 in 2024-06-11T13:40:44.035 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:40:43 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:40:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:40:44.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:44 smithi057 ceph-mon[29805]: osdmap e522: 8 total, 8 up, 8 in 2024-06-11T13:40:44.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:44 smithi057 ceph-mon[29805]: pgmap v813: 292 pgs: 292 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:40:44.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:44 smithi057 ceph-mon[29805]: osdmap e523: 8 total, 8 up, 8 in 2024-06-11T13:40:44.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:44 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:40:44.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:44 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:40:44.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:44 smithi057 ceph-mon[40588]: osdmap e522: 8 total, 8 up, 8 in 2024-06-11T13:40:44.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:44 smithi057 ceph-mon[40588]: pgmap v813: 292 pgs: 292 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:40:44.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:44 smithi057 ceph-mon[40588]: osdmap e523: 8 total, 8 up, 8 in 2024-06-11T13:40:44.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:44 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:40:44.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:44 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:40:44.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:44 smithi107 ceph-mon[33178]: osdmap e522: 8 total, 8 up, 8 in 2024-06-11T13:40:44.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:44 smithi107 ceph-mon[33178]: pgmap v813: 292 pgs: 292 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:40:44.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:44 smithi107 ceph-mon[33178]: osdmap e523: 8 total, 8 up, 8 in 2024-06-11T13:40:44.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:44 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:40:44.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:44 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:40:46.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:45 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:40:46.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:45 smithi107 ceph-mon[33178]: osdmap e524: 8 total, 8 up, 8 in 2024-06-11T13:40:46.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:45 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:40:46.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:45 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:40:46.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:45 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:40:46.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:45 smithi057 ceph-mon[29805]: osdmap e524: 8 total, 8 up, 8 in 2024-06-11T13:40:46.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:45 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:40:46.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:45 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:40:46.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:45 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:40:46.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:45 smithi057 ceph-mon[40588]: osdmap e524: 8 total, 8 up, 8 in 2024-06-11T13:40:46.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:45 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:40:46.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:45 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:40:47.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:46 smithi057 ceph-mon[29805]: pgmap v816: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:40:47.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:46 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-100", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:40:47.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:46 smithi057 ceph-mon[29805]: osdmap e525: 8 total, 8 up, 8 in 2024-06-11T13:40:47.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:46 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-100"}]: dispatch 2024-06-11T13:40:47.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:46 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-100"}]: dispatch 2024-06-11T13:40:47.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:46 smithi057 ceph-mon[40588]: pgmap v816: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:40:47.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:46 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-100", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:40:47.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:46 smithi057 ceph-mon[40588]: osdmap e525: 8 total, 8 up, 8 in 2024-06-11T13:40:47.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:46 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-100"}]: dispatch 2024-06-11T13:40:47.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:46 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-100"}]: dispatch 2024-06-11T13:40:47.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:46 smithi107 ceph-mon[33178]: pgmap v816: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:40:47.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:46 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-100", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:40:47.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:46 smithi107 ceph-mon[33178]: osdmap e525: 8 total, 8 up, 8 in 2024-06-11T13:40:47.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:46 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-100"}]: dispatch 2024-06-11T13:40:47.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:46 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-100"}]: dispatch 2024-06-11T13:40:48.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:47 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-100"}]': finished 2024-06-11T13:40:48.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:47 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-100", "mode": "writeback"}]: dispatch 2024-06-11T13:40:48.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:47 smithi057 ceph-mon[40588]: osdmap e526: 8 total, 8 up, 8 in 2024-06-11T13:40:48.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:47 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-100", "mode": "writeback"}]: dispatch 2024-06-11T13:40:48.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:47 smithi057 ceph-mon[40588]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:40:48.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:47 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-100", "mode": "writeback"}]': finished 2024-06-11T13:40:48.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:47 smithi057 ceph-mon[40588]: osdmap e527: 8 total, 8 up, 8 in 2024-06-11T13:40:48.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:47 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-100","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:40:48.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:47 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-100","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:40:48.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:47 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-100"}]': finished 2024-06-11T13:40:48.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:47 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-100", "mode": "writeback"}]: dispatch 2024-06-11T13:40:48.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:47 smithi057 ceph-mon[29805]: osdmap e526: 8 total, 8 up, 8 in 2024-06-11T13:40:48.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:47 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-100", "mode": "writeback"}]: dispatch 2024-06-11T13:40:48.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:47 smithi057 ceph-mon[29805]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:40:48.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:47 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-100", "mode": "writeback"}]': finished 2024-06-11T13:40:48.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:47 smithi057 ceph-mon[29805]: osdmap e527: 8 total, 8 up, 8 in 2024-06-11T13:40:48.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:47 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-100","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:40:48.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:47 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-100","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:40:48.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:47 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-6", "overlaypool": "test-rados-api-smithi057-77599-100"}]': finished 2024-06-11T13:40:48.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:47 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-100", "mode": "writeback"}]: dispatch 2024-06-11T13:40:48.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:47 smithi107 ceph-mon[33178]: osdmap e526: 8 total, 8 up, 8 in 2024-06-11T13:40:48.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:47 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-100", "mode": "writeback"}]: dispatch 2024-06-11T13:40:48.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:47 smithi107 ceph-mon[33178]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:40:48.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:47 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-100", "mode": "writeback"}]': finished 2024-06-11T13:40:48.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:47 smithi107 ceph-mon[33178]: osdmap e527: 8 total, 8 up, 8 in 2024-06-11T13:40:48.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:47 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-100","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:40:48.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:47 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-100","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:40:49.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:48 smithi057 ceph-mon[40588]: pgmap v819: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:40:49.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:48 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-100","var": "hit_set_count","val": "2"}]': finished 2024-06-11T13:40:49.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:48 smithi057 ceph-mon[40588]: osdmap e528: 8 total, 8 up, 8 in 2024-06-11T13:40:49.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:48 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-100","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:40:49.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:48 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-100","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:40:49.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:48 smithi057 ceph-mon[29805]: pgmap v819: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:40:49.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:48 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-100","var": "hit_set_count","val": "2"}]': finished 2024-06-11T13:40:49.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:48 smithi057 ceph-mon[29805]: osdmap e528: 8 total, 8 up, 8 in 2024-06-11T13:40:49.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:48 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-100","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:40:49.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:48 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-100","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:40:49.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:48 smithi107 ceph-mon[33178]: pgmap v819: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:40:49.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:48 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-100","var": "hit_set_count","val": "2"}]': finished 2024-06-11T13:40:49.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:48 smithi107 ceph-mon[33178]: osdmap e528: 8 total, 8 up, 8 in 2024-06-11T13:40:49.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:48 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-100","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:40:49.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:48 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-100","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:40:50.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:49 smithi057 ceph-mon[29805]: pgmap v822: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:40:50.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:49 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-100","var": "hit_set_period","val": "600"}]': finished 2024-06-11T13:40:50.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:49 smithi057 ceph-mon[29805]: osdmap e529: 8 total, 8 up, 8 in 2024-06-11T13:40:50.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:49 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-11T13:40:50.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:49 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-11T13:40:50.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:49 smithi057 ceph-mon[40588]: pgmap v822: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:40:50.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:49 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-100","var": "hit_set_period","val": "600"}]': finished 2024-06-11T13:40:50.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:49 smithi057 ceph-mon[40588]: osdmap e529: 8 total, 8 up, 8 in 2024-06-11T13:40:50.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:49 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-11T13:40:50.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:49 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-11T13:40:50.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:49 smithi107 ceph-mon[33178]: pgmap v822: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:40:50.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:49 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-100","var": "hit_set_period","val": "600"}]': finished 2024-06-11T13:40:50.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:49 smithi107 ceph-mon[33178]: osdmap e529: 8 total, 8 up, 8 in 2024-06-11T13:40:50.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:49 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-11T13:40:50.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:49 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-11T13:40:51.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:50 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "315.7", "id": [7, 0]}]: dispatch 2024-06-11T13:40:51.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:50 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "315.1a", "id": [2, 0]}]: dispatch 2024-06-11T13:40:51.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:50 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "315.7", "id": [7, 0]}]: dispatch 2024-06-11T13:40:51.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:50 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "315.1c", "id": [2, 1]}]: dispatch 2024-06-11T13:40:51.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:50 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "315.1a", "id": [2, 0]}]: dispatch 2024-06-11T13:40:51.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:50 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "315.1c", "id": [2, 1]}]: dispatch 2024-06-11T13:40:51.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:50 smithi057 ceph-mon[29805]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:40:51.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:50 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-06-11T13:40:51.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:50 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "315.7", "id": [7, 0]}]': finished 2024-06-11T13:40:51.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:50 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "315.1a", "id": [2, 0]}]': finished 2024-06-11T13:40:51.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:50 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "315.1c", "id": [2, 1]}]': finished 2024-06-11T13:40:51.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:50 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-06-11T13:40:51.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:50 smithi057 ceph-mon[29805]: osdmap e530: 8 total, 8 up, 8 in 2024-06-11T13:40:51.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:50 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-06-11T13:40:51.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:50 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "315.7", "id": [7, 0]}]: dispatch 2024-06-11T13:40:51.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:50 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "315.1a", "id": [2, 0]}]: dispatch 2024-06-11T13:40:51.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:50 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "315.7", "id": [7, 0]}]: dispatch 2024-06-11T13:40:51.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:50 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "315.1c", "id": [2, 1]}]: dispatch 2024-06-11T13:40:51.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:50 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "315.1a", "id": [2, 0]}]: dispatch 2024-06-11T13:40:51.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:50 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "315.1c", "id": [2, 1]}]: dispatch 2024-06-11T13:40:51.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:50 smithi057 ceph-mon[40588]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:40:51.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:50 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-06-11T13:40:51.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:50 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "315.7", "id": [7, 0]}]': finished 2024-06-11T13:40:51.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:50 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "315.1a", "id": [2, 0]}]': finished 2024-06-11T13:40:51.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:50 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "315.1c", "id": [2, 1]}]': finished 2024-06-11T13:40:51.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:50 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-06-11T13:40:51.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:50 smithi057 ceph-mon[40588]: osdmap e530: 8 total, 8 up, 8 in 2024-06-11T13:40:51.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:50 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-06-11T13:40:51.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:50 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "315.7", "id": [7, 0]}]: dispatch 2024-06-11T13:40:51.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:50 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "315.1a", "id": [2, 0]}]: dispatch 2024-06-11T13:40:51.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:50 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "315.7", "id": [7, 0]}]: dispatch 2024-06-11T13:40:51.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:50 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "315.1c", "id": [2, 1]}]: dispatch 2024-06-11T13:40:51.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:50 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "315.1a", "id": [2, 0]}]: dispatch 2024-06-11T13:40:51.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:50 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "315.1c", "id": [2, 1]}]: dispatch 2024-06-11T13:40:51.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:50 smithi107 ceph-mon[33178]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:40:51.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:50 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-06-11T13:40:51.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:50 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "315.7", "id": [7, 0]}]': finished 2024-06-11T13:40:51.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:50 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "315.1a", "id": [2, 0]}]': finished 2024-06-11T13:40:51.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:50 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "315.1c", "id": [2, 1]}]': finished 2024-06-11T13:40:51.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:50 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-06-11T13:40:51.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:50 smithi107 ceph-mon[33178]: osdmap e530: 8 total, 8 up, 8 in 2024-06-11T13:40:51.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:50 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-06-11T13:40:51.898 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:40:51 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:40:52.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:51 smithi057 ceph-mon[29805]: pgmap v825: 324 pgs: 3 peering, 321 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:40:52.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:51 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:40:52.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:51 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-06-11T13:40:52.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:51 smithi057 ceph-mon[29805]: osdmap e531: 8 total, 8 up, 8 in 2024-06-11T13:40:52.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:51 smithi057 ceph-mon[40588]: pgmap v825: 324 pgs: 3 peering, 321 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:40:52.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:51 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:40:52.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:51 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-06-11T13:40:52.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:51 smithi057 ceph-mon[40588]: osdmap e531: 8 total, 8 up, 8 in 2024-06-11T13:40:52.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:51 smithi107 ceph-mon[33178]: pgmap v825: 324 pgs: 3 peering, 321 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:40:52.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:51 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:40:52.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:51 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-06-11T13:40:52.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:51 smithi107 ceph-mon[33178]: osdmap e531: 8 total, 8 up, 8 in 2024-06-11T13:40:53.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:52 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:40:53.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:52 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:40:53.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:52 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:40:53.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:52 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:40:53.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:52 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:40:53.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:52 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:40:53.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:52 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:40:53.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:52 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:40:53.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:52 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:40:54.035 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:40:53 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:40:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:40:54.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:53 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:40:54.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:53 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-100"}]: dispatch 2024-06-11T13:40:54.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:53 smithi057 ceph-mon[40588]: osdmap e532: 8 total, 8 up, 8 in 2024-06-11T13:40:54.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:53 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-100"}]: dispatch 2024-06-11T13:40:54.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:53 smithi057 ceph-mon[40588]: pgmap v828: 324 pgs: 3 peering, 321 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:40:54.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:53 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:40:54.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:53 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-100"}]: dispatch 2024-06-11T13:40:54.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:53 smithi057 ceph-mon[29805]: osdmap e532: 8 total, 8 up, 8 in 2024-06-11T13:40:54.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:53 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-100"}]: dispatch 2024-06-11T13:40:54.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:53 smithi057 ceph-mon[29805]: pgmap v828: 324 pgs: 3 peering, 321 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:40:54.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:53 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]': finished 2024-06-11T13:40:54.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:53 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-100"}]: dispatch 2024-06-11T13:40:54.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:53 smithi107 ceph-mon[33178]: osdmap e532: 8 total, 8 up, 8 in 2024-06-11T13:40:54.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:53 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-100"}]: dispatch 2024-06-11T13:40:54.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:53 smithi107 ceph-mon[33178]: pgmap v828: 324 pgs: 3 peering, 321 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:40:55.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:54 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-100"}]': finished 2024-06-11T13:40:55.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:54 smithi057 ceph-mon[29805]: osdmap e533: 8 total, 8 up, 8 in 2024-06-11T13:40:55.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:54 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-100"}]': finished 2024-06-11T13:40:55.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:54 smithi057 ceph-mon[40588]: osdmap e533: 8 total, 8 up, 8 in 2024-06-11T13:40:55.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:54 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-100"}]': finished 2024-06-11T13:40:55.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:54 smithi107 ceph-mon[33178]: osdmap e533: 8 total, 8 up, 8 in 2024-06-11T13:40:56.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:55 smithi057 ceph-mon[29805]: osdmap e534: 8 total, 8 up, 8 in 2024-06-11T13:40:56.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:55 smithi057 ceph-mon[29805]: pgmap v831: 292 pgs: 292 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:40:56.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:55 smithi057 ceph-mon[40588]: osdmap e534: 8 total, 8 up, 8 in 2024-06-11T13:40:56.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:55 smithi057 ceph-mon[40588]: pgmap v831: 292 pgs: 292 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:40:56.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:55 smithi107 ceph-mon[33178]: osdmap e534: 8 total, 8 up, 8 in 2024-06-11T13:40:56.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:55 smithi107 ceph-mon[33178]: pgmap v831: 292 pgs: 292 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:40:57.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:56 smithi057 ceph-mon[29805]: osdmap e535: 8 total, 8 up, 8 in 2024-06-11T13:40:57.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:56 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:40:57.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:56 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:40:57.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:56 smithi057 ceph-mon[40588]: osdmap e535: 8 total, 8 up, 8 in 2024-06-11T13:40:57.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:56 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:40:57.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:56 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:40:57.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:56 smithi107 ceph-mon[33178]: osdmap e535: 8 total, 8 up, 8 in 2024-06-11T13:40:57.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:56 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:40:57.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:56 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:40:58.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:57 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:40:58.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:57 smithi057 ceph-mon[29805]: osdmap e536: 8 total, 8 up, 8 in 2024-06-11T13:40:58.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:57 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:40:58.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:57 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:40:58.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:57 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:40:58.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:57 smithi057 ceph-mon[29805]: pgmap v834: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:40:58.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:57 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:40:58.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:57 smithi057 ceph-mon[40588]: osdmap e536: 8 total, 8 up, 8 in 2024-06-11T13:40:58.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:57 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:40:58.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:57 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:40:58.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:57 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:40:58.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:57 smithi057 ceph-mon[40588]: pgmap v834: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:40:58.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:57 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:40:58.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:57 smithi107 ceph-mon[33178]: osdmap e536: 8 total, 8 up, 8 in 2024-06-11T13:40:58.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:57 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-06-11T13:40:58.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:57 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:40:58.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:57 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-06-11T13:40:58.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:57 smithi107 ceph-mon[33178]: pgmap v834: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:40:59.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:58 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-11T13:40:59.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:58 smithi057 ceph-mon[29805]: osdmap e537: 8 total, 8 up, 8 in 2024-06-11T13:40:59.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:58 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-11T13:40:59.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:40:58 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-11T13:40:59.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:58 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-11T13:40:59.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:58 smithi057 ceph-mon[40588]: osdmap e537: 8 total, 8 up, 8 in 2024-06-11T13:40:59.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:58 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-11T13:40:59.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:40:58 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-11T13:40:59.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:58 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-06-11T13:40:59.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:58 smithi107 ceph-mon[33178]: osdmap e537: 8 total, 8 up, 8 in 2024-06-11T13:40:59.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:58 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-11T13:40:59.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:40:58 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-06-11T13:41:00.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:00 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-06-11T13:41:00.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:00 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-11T13:41:00.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:00 smithi057 ceph-mon[29805]: osdmap e538: 8 total, 8 up, 8 in 2024-06-11T13:41:00.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:00 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-11T13:41:00.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:00 smithi057 ceph-mon[29805]: pgmap v837: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:41:00.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:00 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-06-11T13:41:00.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:00 smithi057 ceph-mon[29805]: osdmap e539: 8 total, 8 up, 8 in 2024-06-11T13:41:00.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:00 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-06-11T13:41:00.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:00 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-11T13:41:00.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:00 smithi057 ceph-mon[40588]: osdmap e538: 8 total, 8 up, 8 in 2024-06-11T13:41:00.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:00 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-11T13:41:00.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:00 smithi057 ceph-mon[40588]: pgmap v837: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:41:00.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:00 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-06-11T13:41:00.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:00 smithi057 ceph-mon[40588]: osdmap e539: 8 total, 8 up, 8 in 2024-06-11T13:41:00.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:00 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-06-11T13:41:00.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:00 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-11T13:41:00.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:00 smithi107 ceph-mon[33178]: osdmap e538: 8 total, 8 up, 8 in 2024-06-11T13:41:00.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:00 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-06-11T13:41:00.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:00 smithi107 ceph-mon[33178]: pgmap v837: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:41:00.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:00 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-06-11T13:41:00.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:00 smithi107 ceph-mon[33178]: osdmap e539: 8 total, 8 up, 8 in 2024-06-11T13:41:01.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:01 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:41:01.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:01 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:41:01.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:01 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-102"}]: dispatch 2024-06-11T13:41:01.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:01 smithi057 ceph-mon[29805]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-102"}]: dispatch 2024-06-11T13:41:01.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:01 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:41:01.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:01 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:41:01.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:01 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-102"}]: dispatch 2024-06-11T13:41:01.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:01 smithi057 ceph-mon[40588]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-102"}]: dispatch 2024-06-11T13:41:01.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:01 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:41:01.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:01 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-6"}]: dispatch 2024-06-11T13:41:01.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:01 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/894258792' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-102"}]: dispatch 2024-06-11T13:41:01.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:01 smithi107 ceph-mon[33178]: from='client.25579 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-6", "tierpool": "test-rados-api-smithi057-77599-102"}]: dispatch 2024-06-11T13:41:02.047 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:41:01 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:41:02.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:02 smithi107 ceph-mon[33178]: osdmap e540: 8 total, 8 up, 8 in 2024-06-11T13:41:02.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:02 smithi107 ceph-mon[33178]: pgmap v840: 292 pgs: 292 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-06-11T13:41:02.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:02 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:41:02.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:02 smithi057 ceph-mon[29805]: osdmap e540: 8 total, 8 up, 8 in 2024-06-11T13:41:02.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:02 smithi057 ceph-mon[29805]: pgmap v840: 292 pgs: 292 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-06-11T13:41:02.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:02 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:41:02.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:02 smithi057 ceph-mon[40588]: osdmap e540: 8 total, 8 up, 8 in 2024-06-11T13:41:02.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:02 smithi057 ceph-mon[40588]: pgmap v840: 292 pgs: 292 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-06-11T13:41:02.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:02 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:41:03.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:03 smithi057 ceph-mon[29805]: osdmap e541: 8 total, 8 up, 8 in 2024-06-11T13:41:03.538 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:03 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1434727939' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi057-77599-104"}]: dispatch 2024-06-11T13:41:03.538 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:03 smithi057 ceph-mon[29805]: from='client.36413 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi057-77599-104"}]: dispatch 2024-06-11T13:41:03.538 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:03 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1434727939' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi057-77599-104"}]: dispatch 2024-06-11T13:41:03.538 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:03 smithi057 ceph-mon[29805]: from='client.36413 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi057-77599-104"}]: dispatch 2024-06-11T13:41:03.538 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:03 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1434727939' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi057-77599-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:41:03.538 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:03 smithi057 ceph-mon[29805]: from='client.36413 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi057-77599-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:41:03.538 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:03 smithi057 ceph-mon[40588]: osdmap e541: 8 total, 8 up, 8 in 2024-06-11T13:41:03.538 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:03 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1434727939' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi057-77599-104"}]: dispatch 2024-06-11T13:41:03.538 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:03 smithi057 ceph-mon[40588]: from='client.36413 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi057-77599-104"}]: dispatch 2024-06-11T13:41:03.539 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:03 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1434727939' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi057-77599-104"}]: dispatch 2024-06-11T13:41:03.539 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:03 smithi057 ceph-mon[40588]: from='client.36413 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi057-77599-104"}]: dispatch 2024-06-11T13:41:03.539 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:03 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1434727939' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi057-77599-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:41:03.539 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:03 smithi057 ceph-mon[40588]: from='client.36413 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi057-77599-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:41:03.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:03 smithi107 ceph-mon[33178]: osdmap e541: 8 total, 8 up, 8 in 2024-06-11T13:41:03.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:03 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1434727939' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi057-77599-104"}]: dispatch 2024-06-11T13:41:03.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:03 smithi107 ceph-mon[33178]: from='client.36413 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi057-77599-104"}]: dispatch 2024-06-11T13:41:03.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:03 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1434727939' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi057-77599-104"}]: dispatch 2024-06-11T13:41:03.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:03 smithi107 ceph-mon[33178]: from='client.36413 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi057-77599-104"}]: dispatch 2024-06-11T13:41:03.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:03 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1434727939' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi057-77599-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:41:03.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:03 smithi107 ceph-mon[33178]: from='client.36413 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi057-77599-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:41:04.035 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:41:03 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:41:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:41:04.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:04 smithi057 ceph-mon[40588]: from='client.36413 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi057-77599-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:41:04.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:04 smithi057 ceph-mon[40588]: osdmap e542: 8 total, 8 up, 8 in 2024-06-11T13:41:04.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:04 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1434727939' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi057-77599-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi057-77599-104"}]: dispatch 2024-06-11T13:41:04.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:04 smithi057 ceph-mon[40588]: from='client.36413 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi057-77599-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi057-77599-104"}]: dispatch 2024-06-11T13:41:04.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:04 smithi057 ceph-mon[40588]: pgmap v843: 260 pgs: 260 active+clean; 583 KiB data, 663 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:41:04.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:04 smithi057 ceph-mon[29805]: from='client.36413 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi057-77599-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:41:04.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:04 smithi057 ceph-mon[29805]: osdmap e542: 8 total, 8 up, 8 in 2024-06-11T13:41:04.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:04 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1434727939' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi057-77599-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi057-77599-104"}]: dispatch 2024-06-11T13:41:04.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:04 smithi057 ceph-mon[29805]: from='client.36413 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi057-77599-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi057-77599-104"}]: dispatch 2024-06-11T13:41:04.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:04 smithi057 ceph-mon[29805]: pgmap v843: 260 pgs: 260 active+clean; 583 KiB data, 663 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:41:04.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:04 smithi107 ceph-mon[33178]: from='client.36413 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi057-77599-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:41:04.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:04 smithi107 ceph-mon[33178]: osdmap e542: 8 total, 8 up, 8 in 2024-06-11T13:41:04.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:04 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1434727939' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi057-77599-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi057-77599-104"}]: dispatch 2024-06-11T13:41:04.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:04 smithi107 ceph-mon[33178]: from='client.36413 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi057-77599-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi057-77599-104"}]: dispatch 2024-06-11T13:41:04.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:04 smithi107 ceph-mon[33178]: pgmap v843: 260 pgs: 260 active+clean; 583 KiB data, 663 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:41:05.372 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:05 smithi057 ceph-mon[40588]: osdmap e543: 8 total, 8 up, 8 in 2024-06-11T13:41:05.372 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:05 smithi057 ceph-mon[29805]: osdmap e543: 8 total, 8 up, 8 in 2024-06-11T13:41:05.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:05 smithi107 ceph-mon[33178]: osdmap e543: 8 total, 8 up, 8 in 2024-06-11T13:41:06.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:06 smithi057 ceph-mon[29805]: from='client.36413 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi057-77599-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi057-77599-104"}]': finished 2024-06-11T13:41:06.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:06 smithi057 ceph-mon[29805]: osdmap e544: 8 total, 8 up, 8 in 2024-06-11T13:41:06.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:06 smithi057 ceph-mon[29805]: pgmap v846: 268 pgs: 8 unknown, 260 active+clean; 583 KiB data, 663 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:41:06.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:06 smithi057 ceph-mon[40588]: from='client.36413 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi057-77599-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi057-77599-104"}]': finished 2024-06-11T13:41:06.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:06 smithi057 ceph-mon[40588]: osdmap e544: 8 total, 8 up, 8 in 2024-06-11T13:41:06.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:06 smithi057 ceph-mon[40588]: pgmap v846: 268 pgs: 8 unknown, 260 active+clean; 583 KiB data, 663 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:41:06.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:06 smithi107 ceph-mon[33178]: from='client.36413 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi057-77599-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi057-77599-104"}]': finished 2024-06-11T13:41:06.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:06 smithi107 ceph-mon[33178]: osdmap e544: 8 total, 8 up, 8 in 2024-06-11T13:41:06.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:06 smithi107 ceph-mon[33178]: pgmap v846: 268 pgs: 8 unknown, 260 active+clean; 583 KiB data, 663 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:41:07.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:07 smithi057 ceph-mon[29805]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:41:07.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:07 smithi057 ceph-mon[29805]: osdmap e545: 8 total, 8 up, 8 in 2024-06-11T13:41:07.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:07 smithi057 ceph-mon[40588]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:41:07.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:07 smithi057 ceph-mon[40588]: osdmap e545: 8 total, 8 up, 8 in 2024-06-11T13:41:07.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:07 smithi107 ceph-mon[33178]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:41:07.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:07 smithi107 ceph-mon[33178]: osdmap e545: 8 total, 8 up, 8 in 2024-06-11T13:41:09.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:08 smithi057 ceph-mon[29805]: osdmap e546: 8 total, 8 up, 8 in 2024-06-11T13:41:09.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:08 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3489209746' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:41:09.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:08 smithi057 ceph-mon[29805]: from='client.36419 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:41:09.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:08 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:41:09.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:08 smithi057 ceph-mon[29805]: pgmap v849: 300 pgs: 32 unknown, 8 creating+peering, 260 active+clean; 583 KiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:41:09.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:08 smithi057 ceph-mon[40588]: osdmap e546: 8 total, 8 up, 8 in 2024-06-11T13:41:09.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:08 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3489209746' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:41:09.039 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:08 smithi057 ceph-mon[40588]: from='client.36419 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:41:09.039 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:08 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:41:09.039 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:08 smithi057 ceph-mon[40588]: pgmap v849: 300 pgs: 32 unknown, 8 creating+peering, 260 active+clean; 583 KiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:41:09.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:08 smithi107 ceph-mon[33178]: osdmap e546: 8 total, 8 up, 8 in 2024-06-11T13:41:09.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:08 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3489209746' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:41:09.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:08 smithi107 ceph-mon[33178]: from='client.36419 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:41:09.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:08 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:41:09.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:08 smithi107 ceph-mon[33178]: pgmap v849: 300 pgs: 32 unknown, 8 creating+peering, 260 active+clean; 583 KiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:41:10.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:09 smithi057 ceph-mon[40588]: from='client.36419 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:41:10.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:09 smithi057 ceph-mon[40588]: osdmap e547: 8 total, 8 up, 8 in 2024-06-11T13:41:10.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:09 smithi057 ceph-mon[40588]: osdmap e548: 8 total, 8 up, 8 in 2024-06-11T13:41:10.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:09 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3489209746' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:41:10.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:09 smithi057 ceph-mon[40588]: from='client.36419 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:41:10.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:09 smithi057 ceph-mon[29805]: from='client.36419 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:41:10.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:09 smithi057 ceph-mon[29805]: osdmap e547: 8 total, 8 up, 8 in 2024-06-11T13:41:10.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:09 smithi057 ceph-mon[29805]: osdmap e548: 8 total, 8 up, 8 in 2024-06-11T13:41:10.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:09 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3489209746' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:41:10.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:09 smithi057 ceph-mon[29805]: from='client.36419 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:41:10.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:09 smithi107 ceph-mon[33178]: from='client.36419 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:41:10.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:09 smithi107 ceph-mon[33178]: osdmap e547: 8 total, 8 up, 8 in 2024-06-11T13:41:10.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:09 smithi107 ceph-mon[33178]: osdmap e548: 8 total, 8 up, 8 in 2024-06-11T13:41:10.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:09 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3489209746' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:41:10.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:09 smithi107 ceph-mon[33178]: from='client.36419 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:41:11.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:10 smithi057 ceph-mon[29805]: pgmap v852: 332 pgs: 64 unknown, 8 creating+peering, 260 active+clean; 583 KiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:41:11.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:10 smithi057 ceph-mon[29805]: from='client.36419 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:41:11.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:10 smithi057 ceph-mon[29805]: osdmap e549: 8 total, 8 up, 8 in 2024-06-11T13:41:11.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:10 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3489209746' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-107", "tierpool": "test-rados-api-smithi057-77599-107-cache"}]: dispatch 2024-06-11T13:41:11.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:10 smithi057 ceph-mon[29805]: from='client.36419 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-107", "tierpool": "test-rados-api-smithi057-77599-107-cache"}]: dispatch 2024-06-11T13:41:11.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:10 smithi057 ceph-mon[40588]: pgmap v852: 332 pgs: 64 unknown, 8 creating+peering, 260 active+clean; 583 KiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:41:11.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:10 smithi057 ceph-mon[40588]: from='client.36419 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:41:11.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:10 smithi057 ceph-mon[40588]: osdmap e549: 8 total, 8 up, 8 in 2024-06-11T13:41:11.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:10 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3489209746' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-107", "tierpool": "test-rados-api-smithi057-77599-107-cache"}]: dispatch 2024-06-11T13:41:11.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:10 smithi057 ceph-mon[40588]: from='client.36419 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-107", "tierpool": "test-rados-api-smithi057-77599-107-cache"}]: dispatch 2024-06-11T13:41:11.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:10 smithi107 ceph-mon[33178]: pgmap v852: 332 pgs: 64 unknown, 8 creating+peering, 260 active+clean; 583 KiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:41:11.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:10 smithi107 ceph-mon[33178]: from='client.36419 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:41:11.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:10 smithi107 ceph-mon[33178]: osdmap e549: 8 total, 8 up, 8 in 2024-06-11T13:41:11.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:10 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3489209746' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-107", "tierpool": "test-rados-api-smithi057-77599-107-cache"}]: dispatch 2024-06-11T13:41:11.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:10 smithi107 ceph-mon[33178]: from='client.36419 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-107", "tierpool": "test-rados-api-smithi057-77599-107-cache"}]: dispatch 2024-06-11T13:41:12.048 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:41:11 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:41:12.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:12 smithi057 ceph-mon[29805]: from='client.36419 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-107", "tierpool": "test-rados-api-smithi057-77599-107-cache"}]': finished 2024-06-11T13:41:12.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:12 smithi057 ceph-mon[29805]: osdmap e550: 8 total, 8 up, 8 in 2024-06-11T13:41:12.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:12 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3489209746' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-107", "overlaypool": "test-rados-api-smithi057-77599-107-cache"}]: dispatch 2024-06-11T13:41:12.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:12 smithi057 ceph-mon[29805]: from='client.36419 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-107", "overlaypool": "test-rados-api-smithi057-77599-107-cache"}]: dispatch 2024-06-11T13:41:12.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:12 smithi057 ceph-mon[29805]: pgmap v855: 332 pgs: 332 active+clean; 583 KiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:41:12.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:12 smithi057 ceph-mon[29805]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:41:12.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:12 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:41:12.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:12 smithi057 ceph-mon[40588]: from='client.36419 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-107", "tierpool": "test-rados-api-smithi057-77599-107-cache"}]': finished 2024-06-11T13:41:12.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:12 smithi057 ceph-mon[40588]: osdmap e550: 8 total, 8 up, 8 in 2024-06-11T13:41:12.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:12 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3489209746' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-107", "overlaypool": "test-rados-api-smithi057-77599-107-cache"}]: dispatch 2024-06-11T13:41:12.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:12 smithi057 ceph-mon[40588]: from='client.36419 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-107", "overlaypool": "test-rados-api-smithi057-77599-107-cache"}]: dispatch 2024-06-11T13:41:12.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:12 smithi057 ceph-mon[40588]: pgmap v855: 332 pgs: 332 active+clean; 583 KiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:41:12.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:12 smithi057 ceph-mon[40588]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:41:12.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:12 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:41:12.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:12 smithi107 ceph-mon[33178]: from='client.36419 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-107", "tierpool": "test-rados-api-smithi057-77599-107-cache"}]': finished 2024-06-11T13:41:12.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:12 smithi107 ceph-mon[33178]: osdmap e550: 8 total, 8 up, 8 in 2024-06-11T13:41:12.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:12 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3489209746' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-107", "overlaypool": "test-rados-api-smithi057-77599-107-cache"}]: dispatch 2024-06-11T13:41:12.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:12 smithi107 ceph-mon[33178]: from='client.36419 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-107", "overlaypool": "test-rados-api-smithi057-77599-107-cache"}]: dispatch 2024-06-11T13:41:12.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:12 smithi107 ceph-mon[33178]: pgmap v855: 332 pgs: 332 active+clean; 583 KiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:41:12.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:12 smithi107 ceph-mon[33178]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:41:12.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:12 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:41:13.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:13 smithi057 ceph-mon[40588]: from='client.36419 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-107", "overlaypool": "test-rados-api-smithi057-77599-107-cache"}]': finished 2024-06-11T13:41:13.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:13 smithi057 ceph-mon[40588]: osdmap e551: 8 total, 8 up, 8 in 2024-06-11T13:41:13.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:13 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3489209746' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-107-cache", "mode": "writeback"}]: dispatch 2024-06-11T13:41:13.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:13 smithi057 ceph-mon[40588]: from='client.36419 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-107-cache", "mode": "writeback"}]: dispatch 2024-06-11T13:41:13.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:13 smithi057 ceph-mon[29805]: from='client.36419 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-107", "overlaypool": "test-rados-api-smithi057-77599-107-cache"}]': finished 2024-06-11T13:41:13.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:13 smithi057 ceph-mon[29805]: osdmap e551: 8 total, 8 up, 8 in 2024-06-11T13:41:13.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:13 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3489209746' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-107-cache", "mode": "writeback"}]: dispatch 2024-06-11T13:41:13.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:13 smithi057 ceph-mon[29805]: from='client.36419 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-107-cache", "mode": "writeback"}]: dispatch 2024-06-11T13:41:13.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:13 smithi107 ceph-mon[33178]: from='client.36419 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-107", "overlaypool": "test-rados-api-smithi057-77599-107-cache"}]': finished 2024-06-11T13:41:13.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:13 smithi107 ceph-mon[33178]: osdmap e551: 8 total, 8 up, 8 in 2024-06-11T13:41:13.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:13 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3489209746' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-107-cache", "mode": "writeback"}]: dispatch 2024-06-11T13:41:13.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:13 smithi107 ceph-mon[33178]: from='client.36419 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-107-cache", "mode": "writeback"}]: dispatch 2024-06-11T13:41:14.035 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:41:13 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:41:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:41:14.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:14 smithi057 ceph-mon[40588]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:41:14.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:14 smithi057 ceph-mon[40588]: from='client.36419 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-107-cache", "mode": "writeback"}]': finished 2024-06-11T13:41:14.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:14 smithi057 ceph-mon[40588]: osdmap e552: 8 total, 8 up, 8 in 2024-06-11T13:41:14.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:14 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3489209746' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-107"}]: dispatch 2024-06-11T13:41:14.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:14 smithi057 ceph-mon[40588]: from='client.36419 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-107"}]: dispatch 2024-06-11T13:41:14.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:14 smithi057 ceph-mon[40588]: pgmap v858: 332 pgs: 332 active+clean; 583 KiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:41:14.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:14 smithi057 ceph-mon[29805]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:41:14.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:14 smithi057 ceph-mon[29805]: from='client.36419 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-107-cache", "mode": "writeback"}]': finished 2024-06-11T13:41:14.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:14 smithi057 ceph-mon[29805]: osdmap e552: 8 total, 8 up, 8 in 2024-06-11T13:41:14.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:14 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3489209746' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-107"}]: dispatch 2024-06-11T13:41:14.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:14 smithi057 ceph-mon[29805]: from='client.36419 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-107"}]: dispatch 2024-06-11T13:41:14.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:14 smithi057 ceph-mon[29805]: pgmap v858: 332 pgs: 332 active+clean; 583 KiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:41:14.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:14 smithi107 ceph-mon[33178]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:41:14.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:14 smithi107 ceph-mon[33178]: from='client.36419 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-107-cache", "mode": "writeback"}]': finished 2024-06-11T13:41:14.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:14 smithi107 ceph-mon[33178]: osdmap e552: 8 total, 8 up, 8 in 2024-06-11T13:41:14.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:14 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3489209746' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-107"}]: dispatch 2024-06-11T13:41:14.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:14 smithi107 ceph-mon[33178]: from='client.36419 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-107"}]: dispatch 2024-06-11T13:41:14.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:14 smithi107 ceph-mon[33178]: pgmap v858: 332 pgs: 332 active+clean; 583 KiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:41:15.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:15 smithi057 ceph-mon[29805]: from='client.36419 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-107"}]': finished 2024-06-11T13:41:15.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:15 smithi057 ceph-mon[29805]: osdmap e553: 8 total, 8 up, 8 in 2024-06-11T13:41:15.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:15 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3489209746' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-107", "tierpool": "test-rados-api-smithi057-77599-107-cache"}]: dispatch 2024-06-11T13:41:15.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:15 smithi057 ceph-mon[29805]: from='client.36419 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-107", "tierpool": "test-rados-api-smithi057-77599-107-cache"}]: dispatch 2024-06-11T13:41:15.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:15 smithi057 ceph-mon[40588]: from='client.36419 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-107"}]': finished 2024-06-11T13:41:15.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:15 smithi057 ceph-mon[40588]: osdmap e553: 8 total, 8 up, 8 in 2024-06-11T13:41:15.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:15 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3489209746' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-107", "tierpool": "test-rados-api-smithi057-77599-107-cache"}]: dispatch 2024-06-11T13:41:15.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:15 smithi057 ceph-mon[40588]: from='client.36419 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-107", "tierpool": "test-rados-api-smithi057-77599-107-cache"}]: dispatch 2024-06-11T13:41:15.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:15 smithi107 ceph-mon[33178]: from='client.36419 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-107"}]': finished 2024-06-11T13:41:15.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:15 smithi107 ceph-mon[33178]: osdmap e553: 8 total, 8 up, 8 in 2024-06-11T13:41:15.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:15 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3489209746' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-107", "tierpool": "test-rados-api-smithi057-77599-107-cache"}]: dispatch 2024-06-11T13:41:15.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:15 smithi107 ceph-mon[33178]: from='client.36419 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-107", "tierpool": "test-rados-api-smithi057-77599-107-cache"}]: dispatch 2024-06-11T13:41:16.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:16 smithi057 ceph-mon[29805]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:41:16.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:16 smithi057 ceph-mon[29805]: from='client.36419 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-107", "tierpool": "test-rados-api-smithi057-77599-107-cache"}]': finished 2024-06-11T13:41:16.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:16 smithi057 ceph-mon[29805]: osdmap e554: 8 total, 8 up, 8 in 2024-06-11T13:41:16.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:16 smithi057 ceph-mon[29805]: pgmap v861: 332 pgs: 332 active+clean; 583 KiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:41:16.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:16 smithi057 ceph-mon[40588]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:41:16.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:16 smithi057 ceph-mon[40588]: from='client.36419 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-107", "tierpool": "test-rados-api-smithi057-77599-107-cache"}]': finished 2024-06-11T13:41:16.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:16 smithi057 ceph-mon[40588]: osdmap e554: 8 total, 8 up, 8 in 2024-06-11T13:41:16.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:16 smithi057 ceph-mon[40588]: pgmap v861: 332 pgs: 332 active+clean; 583 KiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:41:16.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:16 smithi107 ceph-mon[33178]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:41:16.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:16 smithi107 ceph-mon[33178]: from='client.36419 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-107", "tierpool": "test-rados-api-smithi057-77599-107-cache"}]': finished 2024-06-11T13:41:16.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:16 smithi107 ceph-mon[33178]: osdmap e554: 8 total, 8 up, 8 in 2024-06-11T13:41:16.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:16 smithi107 ceph-mon[33178]: pgmap v861: 332 pgs: 332 active+clean; 583 KiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:41:17.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:17 smithi057 ceph-mon[29805]: osdmap e555: 8 total, 8 up, 8 in 2024-06-11T13:41:17.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:17 smithi057 ceph-mon[29805]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:41:17.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:17 smithi057 ceph-mon[40588]: osdmap e555: 8 total, 8 up, 8 in 2024-06-11T13:41:17.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:17 smithi057 ceph-mon[40588]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:41:17.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:17 smithi107 ceph-mon[33178]: osdmap e555: 8 total, 8 up, 8 in 2024-06-11T13:41:17.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:17 smithi107 ceph-mon[33178]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:41:18.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:18 smithi057 ceph-mon[29805]: osdmap e556: 8 total, 8 up, 8 in 2024-06-11T13:41:18.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:18 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi057-77599-109"}]: dispatch 2024-06-11T13:41:18.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:18 smithi057 ceph-mon[29805]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi057-77599-109"}]: dispatch 2024-06-11T13:41:18.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:18 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi057-77599-109"}]: dispatch 2024-06-11T13:41:18.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:18 smithi057 ceph-mon[29805]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi057-77599-109"}]: dispatch 2024-06-11T13:41:18.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:18 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi057-77599-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:41:18.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:18 smithi057 ceph-mon[29805]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi057-77599-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:41:18.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:18 smithi057 ceph-mon[29805]: pgmap v864: 268 pgs: 268 active+clean; 583 KiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:41:18.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:18 smithi057 ceph-mon[40588]: osdmap e556: 8 total, 8 up, 8 in 2024-06-11T13:41:18.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:18 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi057-77599-109"}]: dispatch 2024-06-11T13:41:18.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:18 smithi057 ceph-mon[40588]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi057-77599-109"}]: dispatch 2024-06-11T13:41:18.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:18 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi057-77599-109"}]: dispatch 2024-06-11T13:41:18.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:18 smithi057 ceph-mon[40588]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi057-77599-109"}]: dispatch 2024-06-11T13:41:18.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:18 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi057-77599-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:41:18.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:18 smithi057 ceph-mon[40588]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi057-77599-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:41:18.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:18 smithi057 ceph-mon[40588]: pgmap v864: 268 pgs: 268 active+clean; 583 KiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:41:18.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:18 smithi107 ceph-mon[33178]: osdmap e556: 8 total, 8 up, 8 in 2024-06-11T13:41:18.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:18 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi057-77599-109"}]: dispatch 2024-06-11T13:41:18.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:18 smithi107 ceph-mon[33178]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi057-77599-109"}]: dispatch 2024-06-11T13:41:18.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:18 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi057-77599-109"}]: dispatch 2024-06-11T13:41:18.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:18 smithi107 ceph-mon[33178]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi057-77599-109"}]: dispatch 2024-06-11T13:41:18.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:18 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi057-77599-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:41:18.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:18 smithi107 ceph-mon[33178]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi057-77599-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:41:18.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:18 smithi107 ceph-mon[33178]: pgmap v864: 268 pgs: 268 active+clean; 583 KiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:41:20.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:19 smithi057 ceph-mon[40588]: from='client.26419 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi057-77599-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:41:20.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:19 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi057-77599-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi057-77599-109"}]: dispatch 2024-06-11T13:41:20.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:19 smithi057 ceph-mon[40588]: osdmap e557: 8 total, 8 up, 8 in 2024-06-11T13:41:20.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:19 smithi057 ceph-mon[40588]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi057-77599-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi057-77599-109"}]: dispatch 2024-06-11T13:41:20.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:19 smithi057 ceph-mon[29805]: from='client.26419 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi057-77599-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:41:20.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:19 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi057-77599-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi057-77599-109"}]: dispatch 2024-06-11T13:41:20.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:19 smithi057 ceph-mon[29805]: osdmap e557: 8 total, 8 up, 8 in 2024-06-11T13:41:20.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:19 smithi057 ceph-mon[29805]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi057-77599-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi057-77599-109"}]: dispatch 2024-06-11T13:41:20.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:19 smithi107 ceph-mon[33178]: from='client.26419 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi057-77599-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:41:20.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:19 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi057-77599-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi057-77599-109"}]: dispatch 2024-06-11T13:41:20.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:19 smithi107 ceph-mon[33178]: osdmap e557: 8 total, 8 up, 8 in 2024-06-11T13:41:20.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:19 smithi107 ceph-mon[33178]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi057-77599-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi057-77599-109"}]: dispatch 2024-06-11T13:41:21.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:20 smithi057 ceph-mon[29805]: pgmap v867: 268 pgs: 268 active+clean; 583 KiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:41:21.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:20 smithi057 ceph-mon[29805]: osdmap e558: 8 total, 8 up, 8 in 2024-06-11T13:41:21.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:20 smithi057 ceph-mon[29805]: from='client.26419 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi057-77599-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi057-77599-109"}]': finished 2024-06-11T13:41:21.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:20 smithi057 ceph-mon[29805]: osdmap e559: 8 total, 8 up, 8 in 2024-06-11T13:41:21.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:20 smithi057 ceph-mon[40588]: pgmap v867: 268 pgs: 268 active+clean; 583 KiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:41:21.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:20 smithi057 ceph-mon[40588]: osdmap e558: 8 total, 8 up, 8 in 2024-06-11T13:41:21.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:20 smithi057 ceph-mon[40588]: from='client.26419 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi057-77599-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi057-77599-109"}]': finished 2024-06-11T13:41:21.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:20 smithi057 ceph-mon[40588]: osdmap e559: 8 total, 8 up, 8 in 2024-06-11T13:41:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:20 smithi107 ceph-mon[33178]: pgmap v867: 268 pgs: 268 active+clean; 583 KiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:41:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:20 smithi107 ceph-mon[33178]: osdmap e558: 8 total, 8 up, 8 in 2024-06-11T13:41:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:20 smithi107 ceph-mon[33178]: from='client.26419 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi057-77599-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi057-77599-109"}]': finished 2024-06-11T13:41:21.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:20 smithi107 ceph-mon[33178]: osdmap e559: 8 total, 8 up, 8 in 2024-06-11T13:41:22.048 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:41:21 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:41:22.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:22 smithi057 ceph-mon[29805]: osdmap e560: 8 total, 8 up, 8 in 2024-06-11T13:41:22.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:22 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:41:22.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:22 smithi057 ceph-mon[29805]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:41:22.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:22 smithi057 ceph-mon[29805]: pgmap v870: 308 pgs: 32 unknown, 8 creating+peering, 268 active+clean; 583 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:41:22.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:22 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:41:22.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:22 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:41:22.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:22 smithi057 ceph-mon[40588]: osdmap e560: 8 total, 8 up, 8 in 2024-06-11T13:41:22.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:22 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:41:22.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:22 smithi057 ceph-mon[40588]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:41:22.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:22 smithi057 ceph-mon[40588]: pgmap v870: 308 pgs: 32 unknown, 8 creating+peering, 268 active+clean; 583 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:41:22.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:22 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:41:22.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:22 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:41:22.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:22 smithi107 ceph-mon[33178]: osdmap e560: 8 total, 8 up, 8 in 2024-06-11T13:41:22.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:22 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:41:22.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:22 smithi107 ceph-mon[33178]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:41:22.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:22 smithi107 ceph-mon[33178]: pgmap v870: 308 pgs: 32 unknown, 8 creating+peering, 268 active+clean; 583 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:41:22.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:22 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:41:22.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:22 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:41:23.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:23 smithi057 ceph-mon[29805]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:41:23.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:23 smithi057 ceph-mon[29805]: from='client.26419 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:41:23.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:23 smithi057 ceph-mon[29805]: osdmap e561: 8 total, 8 up, 8 in 2024-06-11T13:41:23.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:23 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-109", "tierpool": "test-rados-api-smithi057-77599-109-cache"}]: dispatch 2024-06-11T13:41:23.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:23 smithi057 ceph-mon[29805]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-109", "tierpool": "test-rados-api-smithi057-77599-109-cache"}]: dispatch 2024-06-11T13:41:23.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:23 smithi057 ceph-mon[40588]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:41:23.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:23 smithi057 ceph-mon[40588]: from='client.26419 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:41:23.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:23 smithi057 ceph-mon[40588]: osdmap e561: 8 total, 8 up, 8 in 2024-06-11T13:41:23.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:23 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-109", "tierpool": "test-rados-api-smithi057-77599-109-cache"}]: dispatch 2024-06-11T13:41:23.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:23 smithi057 ceph-mon[40588]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-109", "tierpool": "test-rados-api-smithi057-77599-109-cache"}]: dispatch 2024-06-11T13:41:23.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:23 smithi107 ceph-mon[33178]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:41:23.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:23 smithi107 ceph-mon[33178]: from='client.26419 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:41:23.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:23 smithi107 ceph-mon[33178]: osdmap e561: 8 total, 8 up, 8 in 2024-06-11T13:41:23.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:23 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-109", "tierpool": "test-rados-api-smithi057-77599-109-cache"}]: dispatch 2024-06-11T13:41:23.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:23 smithi107 ceph-mon[33178]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-109", "tierpool": "test-rados-api-smithi057-77599-109-cache"}]: dispatch 2024-06-11T13:41:24.035 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:41:23 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:41:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:41:24.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:24 smithi107 ceph-mon[33178]: from='client.26419 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-109", "tierpool": "test-rados-api-smithi057-77599-109-cache"}]': finished 2024-06-11T13:41:24.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:24 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-109", "overlaypool": "test-rados-api-smithi057-77599-109-cache"}]: dispatch 2024-06-11T13:41:24.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:24 smithi107 ceph-mon[33178]: osdmap e562: 8 total, 8 up, 8 in 2024-06-11T13:41:24.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:24 smithi107 ceph-mon[33178]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-109", "overlaypool": "test-rados-api-smithi057-77599-109-cache"}]: dispatch 2024-06-11T13:41:24.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:24 smithi107 ceph-mon[33178]: pgmap v873: 308 pgs: 32 unknown, 8 creating+peering, 268 active+clean; 583 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:41:24.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:24 smithi107 ceph-mon[33178]: from='client.26419 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-109", "overlaypool": "test-rados-api-smithi057-77599-109-cache"}]': finished 2024-06-11T13:41:24.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:24 smithi107 ceph-mon[33178]: osdmap e563: 8 total, 8 up, 8 in 2024-06-11T13:41:24.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:24 smithi057 ceph-mon[29805]: from='client.26419 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-109", "tierpool": "test-rados-api-smithi057-77599-109-cache"}]': finished 2024-06-11T13:41:24.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:24 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-109", "overlaypool": "test-rados-api-smithi057-77599-109-cache"}]: dispatch 2024-06-11T13:41:24.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:24 smithi057 ceph-mon[29805]: osdmap e562: 8 total, 8 up, 8 in 2024-06-11T13:41:24.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:24 smithi057 ceph-mon[29805]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-109", "overlaypool": "test-rados-api-smithi057-77599-109-cache"}]: dispatch 2024-06-11T13:41:24.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:24 smithi057 ceph-mon[29805]: pgmap v873: 308 pgs: 32 unknown, 8 creating+peering, 268 active+clean; 583 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:41:24.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:24 smithi057 ceph-mon[29805]: from='client.26419 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-109", "overlaypool": "test-rados-api-smithi057-77599-109-cache"}]': finished 2024-06-11T13:41:24.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:24 smithi057 ceph-mon[29805]: osdmap e563: 8 total, 8 up, 8 in 2024-06-11T13:41:24.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:24 smithi057 ceph-mon[40588]: from='client.26419 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-109", "tierpool": "test-rados-api-smithi057-77599-109-cache"}]': finished 2024-06-11T13:41:24.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:24 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-109", "overlaypool": "test-rados-api-smithi057-77599-109-cache"}]: dispatch 2024-06-11T13:41:24.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:24 smithi057 ceph-mon[40588]: osdmap e562: 8 total, 8 up, 8 in 2024-06-11T13:41:24.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:24 smithi057 ceph-mon[40588]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-109", "overlaypool": "test-rados-api-smithi057-77599-109-cache"}]: dispatch 2024-06-11T13:41:24.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:24 smithi057 ceph-mon[40588]: pgmap v873: 308 pgs: 32 unknown, 8 creating+peering, 268 active+clean; 583 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:41:24.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:24 smithi057 ceph-mon[40588]: from='client.26419 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-109", "overlaypool": "test-rados-api-smithi057-77599-109-cache"}]': finished 2024-06-11T13:41:24.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:24 smithi057 ceph-mon[40588]: osdmap e563: 8 total, 8 up, 8 in 2024-06-11T13:41:25.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:25 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-109-cache", "mode": "writeback"}]: dispatch 2024-06-11T13:41:25.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:25 smithi107 ceph-mon[33178]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-109-cache", "mode": "writeback"}]: dispatch 2024-06-11T13:41:25.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:25 smithi107 ceph-mon[33178]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:41:25.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:25 smithi107 ceph-mon[33178]: from='client.26419 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-109-cache", "mode": "writeback"}]': finished 2024-06-11T13:41:25.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:25 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:41:25.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:25 smithi107 ceph-mon[33178]: osdmap e564: 8 total, 8 up, 8 in 2024-06-11T13:41:25.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:25 smithi107 ceph-mon[33178]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:41:25.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:25 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-109-cache", "mode": "writeback"}]: dispatch 2024-06-11T13:41:25.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:25 smithi057 ceph-mon[29805]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-109-cache", "mode": "writeback"}]: dispatch 2024-06-11T13:41:25.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:25 smithi057 ceph-mon[29805]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:41:25.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:25 smithi057 ceph-mon[29805]: from='client.26419 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-109-cache", "mode": "writeback"}]': finished 2024-06-11T13:41:25.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:25 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:41:25.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:25 smithi057 ceph-mon[29805]: osdmap e564: 8 total, 8 up, 8 in 2024-06-11T13:41:25.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:25 smithi057 ceph-mon[29805]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:41:25.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:25 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-109-cache", "mode": "writeback"}]: dispatch 2024-06-11T13:41:25.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:25 smithi057 ceph-mon[40588]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-109-cache", "mode": "writeback"}]: dispatch 2024-06-11T13:41:25.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:25 smithi057 ceph-mon[40588]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:41:25.787 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:25 smithi057 ceph-mon[40588]: from='client.26419 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-109-cache", "mode": "writeback"}]': finished 2024-06-11T13:41:25.787 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:25 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:41:25.787 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:25 smithi057 ceph-mon[40588]: osdmap e564: 8 total, 8 up, 8 in 2024-06-11T13:41:25.787 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:25 smithi057 ceph-mon[40588]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:41:26.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:26 smithi057 ceph-mon[29805]: pgmap v876: 308 pgs: 32 unknown, 8 creating+peering, 268 active+clean; 583 KiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:41:26.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:26 smithi057 ceph-mon[29805]: from='client.26419 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-06-11T13:41:26.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:26 smithi057 ceph-mon[29805]: osdmap e565: 8 total, 8 up, 8 in 2024-06-11T13:41:26.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:26 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:41:26.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:26 smithi057 ceph-mon[29805]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:41:26.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:26 smithi057 ceph-mon[40588]: pgmap v876: 308 pgs: 32 unknown, 8 creating+peering, 268 active+clean; 583 KiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:41:26.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:26 smithi057 ceph-mon[40588]: from='client.26419 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-06-11T13:41:26.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:26 smithi057 ceph-mon[40588]: osdmap e565: 8 total, 8 up, 8 in 2024-06-11T13:41:26.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:26 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:41:26.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:26 smithi057 ceph-mon[40588]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:41:26.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:26 smithi107 ceph-mon[33178]: pgmap v876: 308 pgs: 32 unknown, 8 creating+peering, 268 active+clean; 583 KiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:41:26.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:26 smithi107 ceph-mon[33178]: from='client.26419 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-06-11T13:41:26.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:26 smithi107 ceph-mon[33178]: osdmap e565: 8 total, 8 up, 8 in 2024-06-11T13:41:26.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:26 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:41:26.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:26 smithi107 ceph-mon[33178]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:41:28.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:28 smithi107 ceph-mon[33178]: from='client.26419 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-06-11T13:41:28.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:28 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-11T13:41:28.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:28 smithi107 ceph-mon[33178]: osdmap e566: 8 total, 8 up, 8 in 2024-06-11T13:41:28.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:28 smithi107 ceph-mon[33178]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-11T13:41:28.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:28 smithi107 ceph-mon[33178]: pgmap v879: 308 pgs: 308 active+clean; 583 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:41:28.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:28 smithi057 ceph-mon[29805]: from='client.26419 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-06-11T13:41:28.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:28 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-11T13:41:28.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:28 smithi057 ceph-mon[29805]: osdmap e566: 8 total, 8 up, 8 in 2024-06-11T13:41:28.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:28 smithi057 ceph-mon[29805]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-11T13:41:28.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:28 smithi057 ceph-mon[29805]: pgmap v879: 308 pgs: 308 active+clean; 583 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:41:28.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:28 smithi057 ceph-mon[40588]: from='client.26419 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-06-11T13:41:28.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:28 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-11T13:41:28.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:28 smithi057 ceph-mon[40588]: osdmap e566: 8 total, 8 up, 8 in 2024-06-11T13:41:28.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:28 smithi057 ceph-mon[40588]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-11T13:41:28.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:28 smithi057 ceph-mon[40588]: pgmap v879: 308 pgs: 308 active+clean; 583 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:41:29.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:29 smithi107 ceph-mon[33178]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:41:29.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:29 smithi107 ceph-mon[33178]: from='client.26419 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-06-11T13:41:29.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:29 smithi107 ceph-mon[33178]: osdmap e567: 8 total, 8 up, 8 in 2024-06-11T13:41:29.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:29 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-06-11T13:41:29.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:29 smithi107 ceph-mon[33178]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-06-11T13:41:29.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:29 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:41:29.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:29 smithi057 ceph-mon[29805]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:41:29.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:29 smithi057 ceph-mon[29805]: from='client.26419 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-06-11T13:41:29.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:29 smithi057 ceph-mon[29805]: osdmap e567: 8 total, 8 up, 8 in 2024-06-11T13:41:29.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:29 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-06-11T13:41:29.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:29 smithi057 ceph-mon[29805]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-06-11T13:41:29.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:29 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:41:29.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:29 smithi057 ceph-mon[40588]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:41:29.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:29 smithi057 ceph-mon[40588]: from='client.26419 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-06-11T13:41:29.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:29 smithi057 ceph-mon[40588]: osdmap e567: 8 total, 8 up, 8 in 2024-06-11T13:41:29.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:29 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-06-11T13:41:29.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:29 smithi057 ceph-mon[40588]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-06-11T13:41:29.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:29 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:41:30.722 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:30 smithi107 ceph-mon[33178]: from='client.26419 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-06-11T13:41:30.722 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:30 smithi107 ceph-mon[33178]: osdmap e568: 8 total, 8 up, 8 in 2024-06-11T13:41:30.722 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:30 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-109"}]: dispatch 2024-06-11T13:41:30.722 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:30 smithi107 ceph-mon[33178]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-109"}]: dispatch 2024-06-11T13:41:30.722 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:30 smithi107 ceph-mon[33178]: pgmap v882: 308 pgs: 308 active+clean; 583 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:41:30.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:30 smithi057 ceph-mon[40588]: from='client.26419 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-06-11T13:41:30.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:30 smithi057 ceph-mon[40588]: osdmap e568: 8 total, 8 up, 8 in 2024-06-11T13:41:30.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:30 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-109"}]: dispatch 2024-06-11T13:41:30.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:30 smithi057 ceph-mon[40588]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-109"}]: dispatch 2024-06-11T13:41:30.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:30 smithi057 ceph-mon[40588]: pgmap v882: 308 pgs: 308 active+clean; 583 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:41:30.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:30 smithi057 ceph-mon[29805]: from='client.26419 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-06-11T13:41:30.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:30 smithi057 ceph-mon[29805]: osdmap e568: 8 total, 8 up, 8 in 2024-06-11T13:41:30.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:30 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-109"}]: dispatch 2024-06-11T13:41:30.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:30 smithi057 ceph-mon[29805]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-109"}]: dispatch 2024-06-11T13:41:30.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:30 smithi057 ceph-mon[29805]: pgmap v882: 308 pgs: 308 active+clean; 583 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:41:31.670 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:31 smithi107 ceph-mon[33178]: from='client.26419 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-109"}]': finished 2024-06-11T13:41:31.670 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:31 smithi107 ceph-mon[33178]: osdmap e569: 8 total, 8 up, 8 in 2024-06-11T13:41:31.670 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:31 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-109", "tierpool": "test-rados-api-smithi057-77599-109-cache"}]: dispatch 2024-06-11T13:41:31.670 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:31 smithi107 ceph-mon[33178]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-109", "tierpool": "test-rados-api-smithi057-77599-109-cache"}]: dispatch 2024-06-11T13:41:31.670 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:31 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:41:31.670 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:31 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:41:31.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:31 smithi057 ceph-mon[29805]: from='client.26419 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-109"}]': finished 2024-06-11T13:41:31.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:31 smithi057 ceph-mon[29805]: osdmap e569: 8 total, 8 up, 8 in 2024-06-11T13:41:31.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:31 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-109", "tierpool": "test-rados-api-smithi057-77599-109-cache"}]: dispatch 2024-06-11T13:41:31.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:31 smithi057 ceph-mon[29805]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-109", "tierpool": "test-rados-api-smithi057-77599-109-cache"}]: dispatch 2024-06-11T13:41:31.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:31 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:41:31.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:31 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:41:31.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:31 smithi057 ceph-mon[40588]: from='client.26419 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-109"}]': finished 2024-06-11T13:41:31.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:31 smithi057 ceph-mon[40588]: osdmap e569: 8 total, 8 up, 8 in 2024-06-11T13:41:31.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:31 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-109", "tierpool": "test-rados-api-smithi057-77599-109-cache"}]: dispatch 2024-06-11T13:41:31.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:31 smithi057 ceph-mon[40588]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-109", "tierpool": "test-rados-api-smithi057-77599-109-cache"}]: dispatch 2024-06-11T13:41:31.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:31 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:41:31.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:31 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:41:32.048 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:41:31 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:41:32.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:32 smithi057 ceph-mon[29805]: from='client.26419 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-109", "tierpool": "test-rados-api-smithi057-77599-109-cache"}]': finished 2024-06-11T13:41:32.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:32 smithi057 ceph-mon[29805]: osdmap e570: 8 total, 8 up, 8 in 2024-06-11T13:41:32.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:32 smithi057 ceph-mon[29805]: pgmap v885: 308 pgs: 308 active+clean; 583 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-06-11T13:41:32.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:32 smithi057 ceph-mon[29805]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:41:32.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:32 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:41:32.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:32 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:41:32.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:32 smithi057 ceph-mon[29805]: osdmap e571: 8 total, 8 up, 8 in 2024-06-11T13:41:32.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:32 smithi057 ceph-mon[40588]: from='client.26419 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-109", "tierpool": "test-rados-api-smithi057-77599-109-cache"}]': finished 2024-06-11T13:41:32.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:32 smithi057 ceph-mon[40588]: osdmap e570: 8 total, 8 up, 8 in 2024-06-11T13:41:32.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:32 smithi057 ceph-mon[40588]: pgmap v885: 308 pgs: 308 active+clean; 583 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-06-11T13:41:32.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:32 smithi057 ceph-mon[40588]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:41:32.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:32 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:41:32.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:32 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:41:32.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:32 smithi057 ceph-mon[40588]: osdmap e571: 8 total, 8 up, 8 in 2024-06-11T13:41:32.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:32 smithi107 ceph-mon[33178]: from='client.26419 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-109", "tierpool": "test-rados-api-smithi057-77599-109-cache"}]': finished 2024-06-11T13:41:32.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:32 smithi107 ceph-mon[33178]: osdmap e570: 8 total, 8 up, 8 in 2024-06-11T13:41:32.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:32 smithi107 ceph-mon[33178]: pgmap v885: 308 pgs: 308 active+clean; 583 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-06-11T13:41:32.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:32 smithi107 ceph-mon[33178]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:41:32.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:32 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:41:32.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:32 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:41:32.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:32 smithi107 ceph-mon[33178]: osdmap e571: 8 total, 8 up, 8 in 2024-06-11T13:41:33.734 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:33 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:41:33.734 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:33 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:41:33.734 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:33 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:41:33.734 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:33 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:41:33.734 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:33 smithi057 ceph-mon[40588]: osdmap e572: 8 total, 8 up, 8 in 2024-06-11T13:41:33.735 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:33 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi057-77599-109"}]: dispatch 2024-06-11T13:41:33.735 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:33 smithi057 ceph-mon[40588]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi057-77599-109"}]: dispatch 2024-06-11T13:41:33.735 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:33 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:41:33.735 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:33 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:41:33.735 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:33 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:41:33.735 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:33 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:41:33.735 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:33 smithi057 ceph-mon[29805]: osdmap e572: 8 total, 8 up, 8 in 2024-06-11T13:41:33.735 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:33 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi057-77599-109"}]: dispatch 2024-06-11T13:41:33.735 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:33 smithi057 ceph-mon[29805]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi057-77599-109"}]: dispatch 2024-06-11T13:41:33.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:33 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:41:33.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:33 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:41:33.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:33 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:41:33.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:33 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:41:33.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:33 smithi107 ceph-mon[33178]: osdmap e572: 8 total, 8 up, 8 in 2024-06-11T13:41:33.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:33 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi057-77599-109"}]: dispatch 2024-06-11T13:41:33.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:33 smithi107 ceph-mon[33178]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi057-77599-109"}]: dispatch 2024-06-11T13:41:34.035 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:41:33 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:41:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:41:34.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:34 smithi057 ceph-mon[29805]: pgmap v888: 268 pgs: 268 active+clean; 583 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:41:34.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:34 smithi057 ceph-mon[29805]: from='client.26419 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi057-77599-109"}]': finished 2024-06-11T13:41:34.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:34 smithi057 ceph-mon[29805]: osdmap e573: 8 total, 8 up, 8 in 2024-06-11T13:41:34.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:34 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi057-77599-109"}]: dispatch 2024-06-11T13:41:34.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:34 smithi057 ceph-mon[29805]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi057-77599-109"}]: dispatch 2024-06-11T13:41:34.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:34 smithi057 ceph-mon[40588]: pgmap v888: 268 pgs: 268 active+clean; 583 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:41:34.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:34 smithi057 ceph-mon[40588]: from='client.26419 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi057-77599-109"}]': finished 2024-06-11T13:41:34.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:34 smithi057 ceph-mon[40588]: osdmap e573: 8 total, 8 up, 8 in 2024-06-11T13:41:34.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:34 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi057-77599-109"}]: dispatch 2024-06-11T13:41:34.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:34 smithi057 ceph-mon[40588]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi057-77599-109"}]: dispatch 2024-06-11T13:41:34.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:34 smithi107 ceph-mon[33178]: pgmap v888: 268 pgs: 268 active+clean; 583 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:41:34.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:34 smithi107 ceph-mon[33178]: from='client.26419 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi057-77599-109"}]': finished 2024-06-11T13:41:34.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:34 smithi107 ceph-mon[33178]: osdmap e573: 8 total, 8 up, 8 in 2024-06-11T13:41:34.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:34 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1814531778' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi057-77599-109"}]: dispatch 2024-06-11T13:41:34.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:34 smithi107 ceph-mon[33178]: from='client.26419 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi057-77599-109"}]: dispatch 2024-06-11T13:41:36.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:36 smithi057 ceph-mon[29805]: from='client.26419 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi057-77599-109"}]': finished 2024-06-11T13:41:36.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:36 smithi057 ceph-mon[29805]: osdmap e574: 8 total, 8 up, 8 in 2024-06-11T13:41:36.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:36 smithi057 ceph-mon[29805]: pgmap v891: 268 pgs: 268 active+clean; 583 KiB data, 655 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:41:36.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:36 smithi057 ceph-mon[40588]: from='client.26419 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi057-77599-109"}]': finished 2024-06-11T13:41:36.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:36 smithi057 ceph-mon[40588]: osdmap e574: 8 total, 8 up, 8 in 2024-06-11T13:41:36.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:36 smithi057 ceph-mon[40588]: pgmap v891: 268 pgs: 268 active+clean; 583 KiB data, 655 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:41:36.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:36 smithi107 ceph-mon[33178]: from='client.26419 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi057-77599-109"}]': finished 2024-06-11T13:41:36.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:36 smithi107 ceph-mon[33178]: osdmap e574: 8 total, 8 up, 8 in 2024-06-11T13:41:36.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:36 smithi107 ceph-mon[33178]: pgmap v891: 268 pgs: 268 active+clean; 583 KiB data, 655 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:41:37.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:37 smithi057 ceph-mon[40588]: osdmap e575: 8 total, 8 up, 8 in 2024-06-11T13:41:37.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:37 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1434727939' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi057-77599-104"}]: dispatch 2024-06-11T13:41:37.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:37 smithi057 ceph-mon[40588]: from='client.36413 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi057-77599-104"}]: dispatch 2024-06-11T13:41:37.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:37 smithi057 ceph-mon[40588]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:41:37.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:37 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:41:37.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:37 smithi057 ceph-mon[29805]: osdmap e575: 8 total, 8 up, 8 in 2024-06-11T13:41:37.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:37 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1434727939' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi057-77599-104"}]: dispatch 2024-06-11T13:41:37.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:37 smithi057 ceph-mon[29805]: from='client.36413 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi057-77599-104"}]: dispatch 2024-06-11T13:41:37.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:37 smithi057 ceph-mon[29805]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:41:37.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:37 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:41:37.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:37 smithi107 ceph-mon[33178]: osdmap e575: 8 total, 8 up, 8 in 2024-06-11T13:41:37.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:37 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1434727939' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi057-77599-104"}]: dispatch 2024-06-11T13:41:37.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:37 smithi107 ceph-mon[33178]: from='client.36413 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi057-77599-104"}]: dispatch 2024-06-11T13:41:37.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:37 smithi107 ceph-mon[33178]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:41:37.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:37 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:41:38.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:38 smithi057 ceph-mon[29805]: from='client.36413 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi057-77599-104"}]': finished 2024-06-11T13:41:38.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:38 smithi057 ceph-mon[29805]: osdmap e576: 8 total, 8 up, 8 in 2024-06-11T13:41:38.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:38 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1434727939' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi057-77599-104"}]: dispatch 2024-06-11T13:41:38.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:38 smithi057 ceph-mon[29805]: from='client.36413 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi057-77599-104"}]: dispatch 2024-06-11T13:41:38.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:38 smithi057 ceph-mon[29805]: pgmap v894: 260 pgs: 260 active+clean; 583 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-06-11T13:41:38.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:38 smithi057 ceph-mon[40588]: from='client.36413 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi057-77599-104"}]': finished 2024-06-11T13:41:38.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:38 smithi057 ceph-mon[40588]: osdmap e576: 8 total, 8 up, 8 in 2024-06-11T13:41:38.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:38 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1434727939' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi057-77599-104"}]: dispatch 2024-06-11T13:41:38.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:38 smithi057 ceph-mon[40588]: from='client.36413 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi057-77599-104"}]: dispatch 2024-06-11T13:41:38.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:38 smithi057 ceph-mon[40588]: pgmap v894: 260 pgs: 260 active+clean; 583 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-06-11T13:41:38.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:38 smithi107 ceph-mon[33178]: from='client.36413 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi057-77599-104"}]': finished 2024-06-11T13:41:38.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:38 smithi107 ceph-mon[33178]: osdmap e576: 8 total, 8 up, 8 in 2024-06-11T13:41:38.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:38 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1434727939' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi057-77599-104"}]: dispatch 2024-06-11T13:41:38.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:38 smithi107 ceph-mon[33178]: from='client.36413 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi057-77599-104"}]: dispatch 2024-06-11T13:41:38.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:38 smithi107 ceph-mon[33178]: pgmap v894: 260 pgs: 260 active+clean; 583 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-06-11T13:41:39.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:39 smithi057 ceph-mon[40588]: from='client.36413 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi057-77599-104"}]': finished 2024-06-11T13:41:39.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:39 smithi057 ceph-mon[40588]: osdmap e577: 8 total, 8 up, 8 in 2024-06-11T13:41:39.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:39 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:41:39.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:39 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:41:39.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:39 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:41:39.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:39 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:41:39.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:39 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi057-77599-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:41:39.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:39 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi057-77599-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:41:39.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:39 smithi057 ceph-mon[29805]: from='client.36413 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi057-77599-104"}]': finished 2024-06-11T13:41:39.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:39 smithi057 ceph-mon[29805]: osdmap e577: 8 total, 8 up, 8 in 2024-06-11T13:41:39.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:39 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:41:39.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:39 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:41:39.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:39 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:41:39.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:39 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:41:39.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:39 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi057-77599-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:41:39.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:39 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi057-77599-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:41:39.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:39 smithi107 ceph-mon[33178]: from='client.36413 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi057-77599-104"}]': finished 2024-06-11T13:41:39.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:39 smithi107 ceph-mon[33178]: osdmap e577: 8 total, 8 up, 8 in 2024-06-11T13:41:39.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:39 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:41:39.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:39 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:41:39.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:39 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:41:39.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:39 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:41:39.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:39 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi057-77599-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:41:39.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:39 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi057-77599-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-06-11T13:41:40.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:40 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi057-77599-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:41:40.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:40 smithi057 ceph-mon[29805]: pgmap v896: 260 pgs: 260 active+clean; 583 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-06-11T13:41:40.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:40 smithi057 ceph-mon[29805]: osdmap e578: 8 total, 8 up, 8 in 2024-06-11T13:41:40.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:40 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi057-77599-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:41:40.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:40 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi057-77599-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:41:40.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:40 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi057-77599-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:41:40.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:40 smithi057 ceph-mon[40588]: pgmap v896: 260 pgs: 260 active+clean; 583 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-06-11T13:41:40.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:40 smithi057 ceph-mon[40588]: osdmap e578: 8 total, 8 up, 8 in 2024-06-11T13:41:40.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:40 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi057-77599-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:41:40.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:40 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi057-77599-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:41:40.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:40 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi057-77599-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-06-11T13:41:40.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:40 smithi107 ceph-mon[33178]: pgmap v896: 260 pgs: 260 active+clean; 583 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-06-11T13:41:40.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:40 smithi107 ceph-mon[33178]: osdmap e578: 8 total, 8 up, 8 in 2024-06-11T13:41:40.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:40 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi057-77599-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:41:40.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:40 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi057-77599-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:41:41.777 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:41 smithi107 ceph-mon[33178]: osdmap e579: 8 total, 8 up, 8 in 2024-06-11T13:41:41.778 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:41:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:41:41.784 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:41 smithi057 ceph-mon[29805]: osdmap e579: 8 total, 8 up, 8 in 2024-06-11T13:41:41.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:41 smithi057 ceph-mon[40588]: osdmap e579: 8 total, 8 up, 8 in 2024-06-11T13:41:42.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:42 smithi057 ceph-mon[29805]: pgmap v899: 260 pgs: 260 active+clean; 583 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:41:42.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:42 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi057-77599-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:41:42.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:42 smithi057 ceph-mon[29805]: osdmap e580: 8 total, 8 up, 8 in 2024-06-11T13:41:42.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:42 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:41:42.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:42 smithi057 ceph-mon[29805]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:41:42.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:42 smithi057 ceph-mon[40588]: pgmap v899: 260 pgs: 260 active+clean; 583 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:41:42.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:42 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi057-77599-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:41:42.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:42 smithi057 ceph-mon[40588]: osdmap e580: 8 total, 8 up, 8 in 2024-06-11T13:41:42.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:42 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:41:42.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:42 smithi057 ceph-mon[40588]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:41:42.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:42 smithi107 ceph-mon[33178]: pgmap v899: 260 pgs: 260 active+clean; 583 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:41:42.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:42 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi057-77599-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:41:42.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:42 smithi107 ceph-mon[33178]: osdmap e580: 8 total, 8 up, 8 in 2024-06-11T13:41:42.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:42 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:41:42.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:42 smithi107 ceph-mon[33178]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:41:43.785 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:41:43 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:41:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:41:43.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:43 smithi057 ceph-mon[29805]: osdmap e581: 8 total, 8 up, 8 in 2024-06-11T13:41:43.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:43 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:41:43.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:43 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:41:43.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:43 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:41:43.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:43 smithi057 ceph-mon[29805]: osdmap e582: 8 total, 8 up, 8 in 2024-06-11T13:41:43.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:43 smithi057 ceph-mon[40588]: osdmap e581: 8 total, 8 up, 8 in 2024-06-11T13:41:43.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:43 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:41:43.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:43 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:41:43.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:43 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:41:43.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:43 smithi057 ceph-mon[40588]: osdmap e582: 8 total, 8 up, 8 in 2024-06-11T13:41:43.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:43 smithi107 ceph-mon[33178]: osdmap e581: 8 total, 8 up, 8 in 2024-06-11T13:41:43.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:43 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:41:43.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:43 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:41:43.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:43 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:41:43.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:43 smithi107 ceph-mon[33178]: osdmap e582: 8 total, 8 up, 8 in 2024-06-11T13:41:44.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:44 smithi057 ceph-mon[29805]: pgmap v902: 300 pgs: 40 unknown, 260 active+clean; 583 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:41:44.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:44 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:41:44.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:44 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:41:44.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:44 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-112", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:41:44.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:44 smithi057 ceph-mon[29805]: osdmap e583: 8 total, 8 up, 8 in 2024-06-11T13:41:44.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:44 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-112"}]: dispatch 2024-06-11T13:41:44.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:44 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-112"}]: dispatch 2024-06-11T13:41:44.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:44 smithi057 ceph-mon[40588]: pgmap v902: 300 pgs: 40 unknown, 260 active+clean; 583 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:41:44.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:44 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:41:44.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:44 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:41:44.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:44 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-112", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:41:44.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:44 smithi057 ceph-mon[40588]: osdmap e583: 8 total, 8 up, 8 in 2024-06-11T13:41:44.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:44 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-112"}]: dispatch 2024-06-11T13:41:44.787 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:44 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-112"}]: dispatch 2024-06-11T13:41:44.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:44 smithi107 ceph-mon[33178]: pgmap v902: 300 pgs: 40 unknown, 260 active+clean; 583 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:41:44.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:44 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:41:44.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:44 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:41:44.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:44 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-112", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:41:44.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:44 smithi107 ceph-mon[33178]: osdmap e583: 8 total, 8 up, 8 in 2024-06-11T13:41:44.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:44 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-112"}]: dispatch 2024-06-11T13:41:44.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:44 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-112"}]: dispatch 2024-06-11T13:41:46.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:46 smithi057 ceph-mon[29805]: pgmap v905: 300 pgs: 40 unknown, 260 active+clean; 583 KiB data, 656 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:41:46.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:46 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-112"}]': finished 2024-06-11T13:41:46.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:46 smithi057 ceph-mon[29805]: osdmap e584: 8 total, 8 up, 8 in 2024-06-11T13:41:46.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:46 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:41:46.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:46 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:41:46.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:46 smithi057 ceph-mon[40588]: pgmap v905: 300 pgs: 40 unknown, 260 active+clean; 583 KiB data, 656 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:41:46.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:46 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-112"}]': finished 2024-06-11T13:41:46.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:46 smithi057 ceph-mon[40588]: osdmap e584: 8 total, 8 up, 8 in 2024-06-11T13:41:46.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:46 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:41:46.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:46 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:41:46.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:46 smithi107 ceph-mon[33178]: pgmap v905: 300 pgs: 40 unknown, 260 active+clean; 583 KiB data, 656 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:41:46.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:46 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-112"}]': finished 2024-06-11T13:41:46.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:46 smithi107 ceph-mon[33178]: osdmap e584: 8 total, 8 up, 8 in 2024-06-11T13:41:46.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:46 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:41:46.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:46 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:41:47.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:47 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:41:47.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:47 smithi057 ceph-mon[29805]: osdmap e585: 8 total, 8 up, 8 in 2024-06-11T13:41:47.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:47 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-112"}]: dispatch 2024-06-11T13:41:47.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:47 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-112"}]: dispatch 2024-06-11T13:41:47.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:47 smithi057 ceph-mon[29805]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:41:47.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:47 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:41:47.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:47 smithi057 ceph-mon[40588]: osdmap e585: 8 total, 8 up, 8 in 2024-06-11T13:41:47.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:47 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-112"}]: dispatch 2024-06-11T13:41:47.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:47 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-112"}]: dispatch 2024-06-11T13:41:47.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:47 smithi057 ceph-mon[40588]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:41:47.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:47 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:41:47.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:47 smithi107 ceph-mon[33178]: osdmap e585: 8 total, 8 up, 8 in 2024-06-11T13:41:47.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:47 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-112"}]: dispatch 2024-06-11T13:41:47.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:47 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-112"}]: dispatch 2024-06-11T13:41:47.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:47 smithi107 ceph-mon[33178]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:41:48.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:48 smithi057 ceph-mon[29805]: pgmap v908: 300 pgs: 300 active+clean; 583 KiB data, 657 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-06-11T13:41:48.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:48 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-112"}]': finished 2024-06-11T13:41:48.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:48 smithi057 ceph-mon[29805]: osdmap e586: 8 total, 8 up, 8 in 2024-06-11T13:41:48.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:48 smithi057 ceph-mon[40588]: pgmap v908: 300 pgs: 300 active+clean; 583 KiB data, 657 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-06-11T13:41:48.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:48 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-112"}]': finished 2024-06-11T13:41:48.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:48 smithi057 ceph-mon[40588]: osdmap e586: 8 total, 8 up, 8 in 2024-06-11T13:41:48.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:48 smithi107 ceph-mon[33178]: pgmap v908: 300 pgs: 300 active+clean; 583 KiB data, 657 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-06-11T13:41:48.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:48 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-112"}]': finished 2024-06-11T13:41:48.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:48 smithi107 ceph-mon[33178]: osdmap e586: 8 total, 8 up, 8 in 2024-06-11T13:41:49.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:49 smithi107 ceph-mon[33178]: osdmap e587: 8 total, 8 up, 8 in 2024-06-11T13:41:50.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:49 smithi057 ceph-mon[29805]: osdmap e587: 8 total, 8 up, 8 in 2024-06-11T13:41:50.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:49 smithi057 ceph-mon[40588]: osdmap e587: 8 total, 8 up, 8 in 2024-06-11T13:41:50.829 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.829 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.829 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.829 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.829 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.829 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.829 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.829 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.830 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.830 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.830 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.830 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.830 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.830 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.830 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.830 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.830 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.830 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.830 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.830 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.831 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.831 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.831 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.831 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.831 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.831 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.831 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.831 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.831 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.831 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.831 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.832 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.832 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.832 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.832 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.832 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.832 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.832 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.832 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.832 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.832 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.832 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.832 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.832 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.833 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.833 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.833 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.833 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.833 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.833 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.833 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.833 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.833 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.833 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.833 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.833 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.834 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.834 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.834 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.834 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.834 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.834 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.834 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.834 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.834 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.834 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.834 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.834 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.834 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.834 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.835 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.835 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.835 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.835 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.835 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.835 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.835 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.835 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.835 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.835 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.835 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.835 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.835 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.835 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.836 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.836 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.836 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.836 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.836 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.836 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.836 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.836 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.836 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.836 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.836 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.836 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.836 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.837 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:50.837 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:41:51.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:50 smithi057 ceph-mon[29805]: pgmap v911: 268 pgs: 268 active+clean; 583 KiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:41:51.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:50 smithi057 ceph-mon[29805]: osdmap e588: 8 total, 8 up, 8 in 2024-06-11T13:41:51.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:50 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:41:51.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:50 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:41:51.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:50 smithi057 ceph-mon[40588]: pgmap v911: 268 pgs: 268 active+clean; 583 KiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:41:51.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:50 smithi057 ceph-mon[40588]: osdmap e588: 8 total, 8 up, 8 in 2024-06-11T13:41:51.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:50 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:41:51.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:50 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:41:51.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:50 smithi107 ceph-mon[33178]: pgmap v911: 268 pgs: 268 active+clean; 583 KiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:41:51.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:50 smithi107 ceph-mon[33178]: osdmap e588: 8 total, 8 up, 8 in 2024-06-11T13:41:51.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:50 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:41:51.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:50 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:41:52.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:51 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:41:52.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:51 smithi057 ceph-mon[29805]: osdmap e589: 8 total, 8 up, 8 in 2024-06-11T13:41:52.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:51 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:41:52.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:51 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:41:52.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:51 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-114", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:41:52.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:51 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:41:52.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:51 smithi057 ceph-mon[40588]: osdmap e589: 8 total, 8 up, 8 in 2024-06-11T13:41:52.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:51 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:41:52.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:51 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:41:52.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:51 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-114", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:41:52.048 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:41:51 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:41:52.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:51 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:41:52.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:51 smithi107 ceph-mon[33178]: osdmap e589: 8 total, 8 up, 8 in 2024-06-11T13:41:52.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:51 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:41:52.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:51 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:41:52.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:51 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-114", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:41:53.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:52 smithi057 ceph-mon[29805]: pgmap v914: 300 pgs: 300 active+clean; 583 KiB data, 662 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-11T13:41:53.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:52 smithi057 ceph-mon[29805]: osdmap e590: 8 total, 8 up, 8 in 2024-06-11T13:41:53.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:52 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-114"}]: dispatch 2024-06-11T13:41:53.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:52 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-114"}]: dispatch 2024-06-11T13:41:53.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:52 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:41:53.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:52 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:41:53.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:52 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-114"}]': finished 2024-06-11T13:41:53.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:52 smithi057 ceph-mon[40588]: pgmap v914: 300 pgs: 300 active+clean; 583 KiB data, 662 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-11T13:41:53.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:52 smithi057 ceph-mon[40588]: osdmap e590: 8 total, 8 up, 8 in 2024-06-11T13:41:53.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:52 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-114"}]: dispatch 2024-06-11T13:41:53.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:52 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-114"}]: dispatch 2024-06-11T13:41:53.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:52 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:41:53.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:52 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:41:53.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:52 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-114"}]': finished 2024-06-11T13:41:53.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:52 smithi107 ceph-mon[33178]: pgmap v914: 300 pgs: 300 active+clean; 583 KiB data, 662 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-11T13:41:53.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:52 smithi107 ceph-mon[33178]: osdmap e590: 8 total, 8 up, 8 in 2024-06-11T13:41:53.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:52 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-114"}]: dispatch 2024-06-11T13:41:53.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:52 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-114"}]: dispatch 2024-06-11T13:41:53.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:52 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:41:53.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:52 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:41:53.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:52 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-114"}]': finished 2024-06-11T13:41:54.035 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:41:53 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:41:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:41:54.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:53 smithi057 ceph-mon[40588]: osdmap e591: 8 total, 8 up, 8 in 2024-06-11T13:41:54.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:53 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-114", "mode": "writeback"}]: dispatch 2024-06-11T13:41:54.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:53 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-114", "mode": "writeback"}]: dispatch 2024-06-11T13:41:54.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:53 smithi057 ceph-mon[40588]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:41:54.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:53 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-114", "mode": "writeback"}]': finished 2024-06-11T13:41:54.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:53 smithi057 ceph-mon[40588]: osdmap e592: 8 total, 8 up, 8 in 2024-06-11T13:41:54.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:53 smithi057 ceph-mon[29805]: osdmap e591: 8 total, 8 up, 8 in 2024-06-11T13:41:54.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:53 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-114", "mode": "writeback"}]: dispatch 2024-06-11T13:41:54.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:53 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-114", "mode": "writeback"}]: dispatch 2024-06-11T13:41:54.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:53 smithi057 ceph-mon[29805]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:41:54.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:53 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-114", "mode": "writeback"}]': finished 2024-06-11T13:41:54.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:53 smithi057 ceph-mon[29805]: osdmap e592: 8 total, 8 up, 8 in 2024-06-11T13:41:54.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:53 smithi107 ceph-mon[33178]: osdmap e591: 8 total, 8 up, 8 in 2024-06-11T13:41:54.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:53 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-114", "mode": "writeback"}]: dispatch 2024-06-11T13:41:54.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:53 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-114", "mode": "writeback"}]: dispatch 2024-06-11T13:41:54.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:53 smithi107 ceph-mon[33178]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:41:54.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:53 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-114", "mode": "writeback"}]': finished 2024-06-11T13:41:54.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:53 smithi107 ceph-mon[33178]: osdmap e592: 8 total, 8 up, 8 in 2024-06-11T13:41:54.936 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:54 smithi057 ceph-mon[40588]: pgmap v917: 300 pgs: 300 active+clean; 583 KiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:41:54.936 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:54 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:41:54.937 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:54 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:41:54.937 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:54 smithi057 ceph-mon[29805]: pgmap v917: 300 pgs: 300 active+clean; 583 KiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:41:54.937 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:54 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:41:54.937 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:54 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:41:55.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:54 smithi107 ceph-mon[33178]: pgmap v917: 300 pgs: 300 active+clean; 583 KiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:41:55.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:54 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:41:55.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:54 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:41:55.955 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:55 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:41:55.955 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:55 smithi057 ceph-mon[40588]: osdmap e593: 8 total, 8 up, 8 in 2024-06-11T13:41:55.955 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:55 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-114"}]: dispatch 2024-06-11T13:41:55.955 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:55 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-114"}]: dispatch 2024-06-11T13:41:55.955 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:55 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:41:55.955 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:55 smithi057 ceph-mon[29805]: osdmap e593: 8 total, 8 up, 8 in 2024-06-11T13:41:55.955 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:55 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-114"}]: dispatch 2024-06-11T13:41:55.955 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:55 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-114"}]: dispatch 2024-06-11T13:41:56.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:55 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:41:56.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:55 smithi107 ceph-mon[33178]: osdmap e593: 8 total, 8 up, 8 in 2024-06-11T13:41:56.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:55 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-114"}]: dispatch 2024-06-11T13:41:56.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:55 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-114"}]: dispatch 2024-06-11T13:41:57.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:57 smithi057 ceph-mon[40588]: pgmap v920: 300 pgs: 300 active+clean; 583 KiB data, 662 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:41:57.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:57 smithi057 ceph-mon[40588]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:41:57.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:57 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-114"}]': finished 2024-06-11T13:41:57.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:57 smithi057 ceph-mon[40588]: osdmap e594: 8 total, 8 up, 8 in 2024-06-11T13:41:57.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:57 smithi057 ceph-mon[29805]: pgmap v920: 300 pgs: 300 active+clean; 583 KiB data, 662 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:41:57.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:57 smithi057 ceph-mon[29805]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:41:57.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:57 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-114"}]': finished 2024-06-11T13:41:57.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:57 smithi057 ceph-mon[29805]: osdmap e594: 8 total, 8 up, 8 in 2024-06-11T13:41:57.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:57 smithi107 ceph-mon[33178]: pgmap v920: 300 pgs: 300 active+clean; 583 KiB data, 662 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:41:57.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:57 smithi107 ceph-mon[33178]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:41:57.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:57 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-114"}]': finished 2024-06-11T13:41:57.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:57 smithi107 ceph-mon[33178]: osdmap e594: 8 total, 8 up, 8 in 2024-06-11T13:41:58.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:58 smithi057 ceph-mon[29805]: osdmap e595: 8 total, 8 up, 8 in 2024-06-11T13:41:58.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:58 smithi057 ceph-mon[29805]: pgmap v923: 268 pgs: 268 active+clean; 583 KiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:41:58.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:58 smithi057 ceph-mon[29805]: osdmap e596: 8 total, 8 up, 8 in 2024-06-11T13:41:58.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:58 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:41:58.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:58 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:41:58.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:58 smithi057 ceph-mon[40588]: osdmap e595: 8 total, 8 up, 8 in 2024-06-11T13:41:58.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:58 smithi057 ceph-mon[40588]: pgmap v923: 268 pgs: 268 active+clean; 583 KiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:41:58.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:58 smithi057 ceph-mon[40588]: osdmap e596: 8 total, 8 up, 8 in 2024-06-11T13:41:58.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:58 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:41:58.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:58 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:41:58.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:58 smithi107 ceph-mon[33178]: osdmap e595: 8 total, 8 up, 8 in 2024-06-11T13:41:58.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:58 smithi107 ceph-mon[33178]: pgmap v923: 268 pgs: 268 active+clean; 583 KiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:41:58.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:58 smithi107 ceph-mon[33178]: osdmap e596: 8 total, 8 up, 8 in 2024-06-11T13:41:58.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:58 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:41:58.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:58 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:42:00.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:59 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:42:00.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:41:59 smithi057 ceph-mon[40588]: osdmap e597: 8 total, 8 up, 8 in 2024-06-11T13:42:00.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:59 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:42:00.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:41:59 smithi057 ceph-mon[29805]: osdmap e597: 8 total, 8 up, 8 in 2024-06-11T13:42:00.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:59 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:42:00.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:41:59 smithi107 ceph-mon[33178]: osdmap e597: 8 total, 8 up, 8 in 2024-06-11T13:42:01.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:00 smithi057 ceph-mon[29805]: pgmap v926: 300 pgs: 32 unknown, 268 active+clean; 583 KiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:42:01.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:00 smithi057 ceph-mon[29805]: osdmap e598: 8 total, 8 up, 8 in 2024-06-11T13:42:01.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:00 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:42:01.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:00 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:42:01.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:00 smithi057 ceph-mon[40588]: pgmap v926: 300 pgs: 32 unknown, 268 active+clean; 583 KiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:42:01.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:00 smithi057 ceph-mon[40588]: osdmap e598: 8 total, 8 up, 8 in 2024-06-11T13:42:01.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:00 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:42:01.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:00 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:42:01.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:00 smithi107 ceph-mon[33178]: pgmap v926: 300 pgs: 32 unknown, 268 active+clean; 583 KiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:42:01.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:00 smithi107 ceph-mon[33178]: osdmap e598: 8 total, 8 up, 8 in 2024-06-11T13:42:01.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:00 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:42:01.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:00 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:42:01.950 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:01 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-116", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:42:01.950 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:01 smithi107 ceph-mon[33178]: osdmap e599: 8 total, 8 up, 8 in 2024-06-11T13:42:01.950 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:01 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-116"}]: dispatch 2024-06-11T13:42:01.950 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:01 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-116"}]: dispatch 2024-06-11T13:42:01.950 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:01 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-116"}]': finished 2024-06-11T13:42:01.950 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:01 smithi107 ceph-mon[33178]: osdmap e600: 8 total, 8 up, 8 in 2024-06-11T13:42:01.951 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:42:01 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:42:02.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:01 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-116", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:42:02.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:01 smithi057 ceph-mon[29805]: osdmap e599: 8 total, 8 up, 8 in 2024-06-11T13:42:02.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:01 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-116"}]: dispatch 2024-06-11T13:42:02.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:01 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-116"}]: dispatch 2024-06-11T13:42:02.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:01 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-116"}]': finished 2024-06-11T13:42:02.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:01 smithi057 ceph-mon[29805]: osdmap e600: 8 total, 8 up, 8 in 2024-06-11T13:42:02.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:01 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-116", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:42:02.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:01 smithi057 ceph-mon[40588]: osdmap e599: 8 total, 8 up, 8 in 2024-06-11T13:42:02.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:01 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-116"}]: dispatch 2024-06-11T13:42:02.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:01 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-116"}]: dispatch 2024-06-11T13:42:02.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:01 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-116"}]': finished 2024-06-11T13:42:02.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:01 smithi057 ceph-mon[40588]: osdmap e600: 8 total, 8 up, 8 in 2024-06-11T13:42:03.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:02 smithi057 ceph-mon[40588]: pgmap v929: 300 pgs: 300 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-06-11T13:42:03.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:02 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-116", "mode": "writeback"}]: dispatch 2024-06-11T13:42:03.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:02 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-116", "mode": "writeback"}]: dispatch 2024-06-11T13:42:03.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:02 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:42:03.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:02 smithi057 ceph-mon[40588]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:42:03.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:02 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-116", "mode": "writeback"}]': finished 2024-06-11T13:42:03.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:02 smithi057 ceph-mon[40588]: osdmap e601: 8 total, 8 up, 8 in 2024-06-11T13:42:03.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:02 smithi057 ceph-mon[29805]: pgmap v929: 300 pgs: 300 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-06-11T13:42:03.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:02 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-116", "mode": "writeback"}]: dispatch 2024-06-11T13:42:03.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:02 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-116", "mode": "writeback"}]: dispatch 2024-06-11T13:42:03.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:02 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:42:03.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:02 smithi057 ceph-mon[29805]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:42:03.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:02 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-116", "mode": "writeback"}]': finished 2024-06-11T13:42:03.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:02 smithi057 ceph-mon[29805]: osdmap e601: 8 total, 8 up, 8 in 2024-06-11T13:42:03.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:02 smithi107 ceph-mon[33178]: pgmap v929: 300 pgs: 300 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-06-11T13:42:03.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:02 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-116", "mode": "writeback"}]: dispatch 2024-06-11T13:42:03.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:02 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-116", "mode": "writeback"}]: dispatch 2024-06-11T13:42:03.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:02 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:42:03.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:02 smithi107 ceph-mon[33178]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:42:03.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:02 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-116", "mode": "writeback"}]': finished 2024-06-11T13:42:03.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:02 smithi107 ceph-mon[33178]: osdmap e601: 8 total, 8 up, 8 in 2024-06-11T13:42:04.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:03 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "325.0"}]: dispatch 2024-06-11T13:42:04.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:03 smithi057 ceph-mon[29805]: from='mon.? v1:172.21.15.57:0/45435016' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "325.0"}]: dispatch 2024-06-11T13:42:04.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:03 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "325.0"}]: dispatch 2024-06-11T13:42:04.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:03 smithi057 ceph-mon[40588]: from='mon.? v1:172.21.15.57:0/45435016' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "325.0"}]: dispatch 2024-06-11T13:42:04.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:42:03 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:42:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:42:04.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:03 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "325.0"}]: dispatch 2024-06-11T13:42:04.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:03 smithi107 ceph-mon[33178]: from='mon.? v1:172.21.15.57:0/45435016' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "325.0"}]: dispatch 2024-06-11T13:42:05.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:04 smithi057 ceph-mon[40588]: 325.0 scrub starts 2024-06-11T13:42:05.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:04 smithi057 ceph-mon[40588]: 325.0 scrub ok 2024-06-11T13:42:05.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:04 smithi057 ceph-mon[40588]: pgmap v932: 300 pgs: 300 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-06-11T13:42:05.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:04 smithi057 ceph-mon[29805]: 325.0 scrub starts 2024-06-11T13:42:05.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:04 smithi057 ceph-mon[29805]: 325.0 scrub ok 2024-06-11T13:42:05.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:04 smithi057 ceph-mon[29805]: pgmap v932: 300 pgs: 300 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-06-11T13:42:05.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:04 smithi107 ceph-mon[33178]: 325.0 scrub starts 2024-06-11T13:42:05.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:04 smithi107 ceph-mon[33178]: 325.0 scrub ok 2024-06-11T13:42:05.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:04 smithi107 ceph-mon[33178]: pgmap v932: 300 pgs: 300 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-06-11T13:42:07.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:06 smithi057 ceph-mon[29805]: pgmap v933: 300 pgs: 300 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 883 B/s rd, 1.2 KiB/s wr, 2 op/s 2024-06-11T13:42:07.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:06 smithi057 ceph-mon[40588]: pgmap v933: 300 pgs: 300 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 883 B/s rd, 1.2 KiB/s wr, 2 op/s 2024-06-11T13:42:07.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:06 smithi107 ceph-mon[33178]: pgmap v933: 300 pgs: 300 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 883 B/s rd, 1.2 KiB/s wr, 2 op/s 2024-06-11T13:42:08.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:08 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:42:08.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:08 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:42:08.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:08 smithi057 ceph-mon[29805]: pgmap v934: 300 pgs: 300 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-06-11T13:42:08.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:08 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:42:08.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:08 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:42:08.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:08 smithi057 ceph-mon[40588]: pgmap v934: 300 pgs: 300 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-06-11T13:42:08.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:08 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:42:08.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:08 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:42:08.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:08 smithi107 ceph-mon[33178]: pgmap v934: 300 pgs: 300 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-06-11T13:42:10.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:10 smithi057 ceph-mon[29805]: pgmap v935: 300 pgs: 300 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 1 op/s 2024-06-11T13:42:10.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:10 smithi057 ceph-mon[40588]: pgmap v935: 300 pgs: 300 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 1 op/s 2024-06-11T13:42:10.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:10 smithi107 ceph-mon[33178]: pgmap v935: 300 pgs: 300 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 1 op/s 2024-06-11T13:42:12.048 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:42:11 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:42:12.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:12 smithi057 ceph-mon[29805]: pgmap v936: 300 pgs: 300 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:42:12.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:12 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:42:12.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:12 smithi057 ceph-mon[40588]: pgmap v936: 300 pgs: 300 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:42:12.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:12 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:42:12.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:12 smithi107 ceph-mon[33178]: pgmap v936: 300 pgs: 300 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:42:12.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:12 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:42:14.035 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:42:13 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:42:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:42:14.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:14 smithi057 ceph-mon[29805]: pgmap v937: 300 pgs: 300 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-06-11T13:42:14.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:14 smithi057 ceph-mon[40588]: pgmap v937: 300 pgs: 300 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-06-11T13:42:14.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:14 smithi107 ceph-mon[33178]: pgmap v937: 300 pgs: 300 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-06-11T13:42:16.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:16 smithi057 ceph-mon[29805]: pgmap v938: 300 pgs: 300 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-06-11T13:42:16.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:16 smithi057 ceph-mon[40588]: pgmap v938: 300 pgs: 300 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-06-11T13:42:16.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:16 smithi107 ceph-mon[33178]: pgmap v938: 300 pgs: 300 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-06-11T13:42:18.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:18 smithi057 ceph-mon[29805]: pgmap v939: 300 pgs: 300 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-06-11T13:42:18.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:18 smithi057 ceph-mon[40588]: pgmap v939: 300 pgs: 300 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-06-11T13:42:18.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:18 smithi107 ceph-mon[33178]: pgmap v939: 300 pgs: 300 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-06-11T13:42:20.038 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:19 smithi057 ceph-mon[29805]: osdmap e602: 8 total, 8 up, 8 in 2024-06-11T13:42:20.038 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:19 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:42:20.039 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:19 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:42:20.039 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:19 smithi057 ceph-mon[40588]: osdmap e602: 8 total, 8 up, 8 in 2024-06-11T13:42:20.039 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:19 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:42:20.039 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:19 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:42:20.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:19 smithi107 ceph-mon[33178]: osdmap e602: 8 total, 8 up, 8 in 2024-06-11T13:42:20.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:19 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:42:20.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:19 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:42:21.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:20 smithi057 ceph-mon[29805]: pgmap v941: 300 pgs: 300 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T13:42:21.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:20 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:42:21.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:20 smithi057 ceph-mon[29805]: osdmap e603: 8 total, 8 up, 8 in 2024-06-11T13:42:21.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:20 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-116"}]: dispatch 2024-06-11T13:42:21.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:20 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-116"}]: dispatch 2024-06-11T13:42:21.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:20 smithi057 ceph-mon[40588]: pgmap v941: 300 pgs: 300 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T13:42:21.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:20 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:42:21.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:20 smithi057 ceph-mon[40588]: osdmap e603: 8 total, 8 up, 8 in 2024-06-11T13:42:21.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:20 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-116"}]: dispatch 2024-06-11T13:42:21.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:20 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-116"}]: dispatch 2024-06-11T13:42:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:20 smithi107 ceph-mon[33178]: pgmap v941: 300 pgs: 300 active+clean; 583 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T13:42:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:20 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:42:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:20 smithi107 ceph-mon[33178]: osdmap e603: 8 total, 8 up, 8 in 2024-06-11T13:42:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:20 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-116"}]: dispatch 2024-06-11T13:42:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:20 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-116"}]: dispatch 2024-06-11T13:42:22.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:21 smithi057 ceph-mon[40588]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:42:22.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:21 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-116"}]': finished 2024-06-11T13:42:22.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:21 smithi057 ceph-mon[40588]: osdmap e604: 8 total, 8 up, 8 in 2024-06-11T13:42:22.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:21 smithi057 ceph-mon[29805]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:42:22.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:21 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-116"}]': finished 2024-06-11T13:42:22.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:21 smithi057 ceph-mon[29805]: osdmap e604: 8 total, 8 up, 8 in 2024-06-11T13:42:22.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:21 smithi107 ceph-mon[33178]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:42:22.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:21 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-116"}]': finished 2024-06-11T13:42:22.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:21 smithi107 ceph-mon[33178]: osdmap e604: 8 total, 8 up, 8 in 2024-06-11T13:42:22.049 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:42:21 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:42:23.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:22 smithi057 ceph-mon[29805]: pgmap v944: 300 pgs: 300 active+clean; 583 KiB data, 668 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 0 B/s wr, 3 op/s 2024-06-11T13:42:23.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:22 smithi057 ceph-mon[29805]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:42:23.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:22 smithi057 ceph-mon[29805]: osdmap e605: 8 total, 8 up, 8 in 2024-06-11T13:42:23.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:22 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:42:23.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:22 smithi057 ceph-mon[29805]: osdmap e606: 8 total, 8 up, 8 in 2024-06-11T13:42:23.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:22 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:42:23.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:22 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:42:23.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:22 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:42:23.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:22 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:42:23.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:22 smithi057 ceph-mon[40588]: pgmap v944: 300 pgs: 300 active+clean; 583 KiB data, 668 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 0 B/s wr, 3 op/s 2024-06-11T13:42:23.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:22 smithi057 ceph-mon[40588]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:42:23.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:22 smithi057 ceph-mon[40588]: osdmap e605: 8 total, 8 up, 8 in 2024-06-11T13:42:23.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:22 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:42:23.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:22 smithi057 ceph-mon[40588]: osdmap e606: 8 total, 8 up, 8 in 2024-06-11T13:42:23.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:22 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:42:23.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:22 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:42:23.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:22 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:42:23.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:22 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:42:23.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:22 smithi107 ceph-mon[33178]: pgmap v944: 300 pgs: 300 active+clean; 583 KiB data, 668 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 0 B/s wr, 3 op/s 2024-06-11T13:42:23.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:22 smithi107 ceph-mon[33178]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:42:23.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:22 smithi107 ceph-mon[33178]: osdmap e605: 8 total, 8 up, 8 in 2024-06-11T13:42:23.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:22 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:42:23.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:22 smithi107 ceph-mon[33178]: osdmap e606: 8 total, 8 up, 8 in 2024-06-11T13:42:23.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:22 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:42:23.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:22 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:42:23.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:22 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:42:23.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:22 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:42:24.035 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:42:23 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:42:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:42:24.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:24 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:42:24.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:24 smithi057 ceph-mon[40588]: osdmap e607: 8 total, 8 up, 8 in 2024-06-11T13:42:24.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:24 smithi057 ceph-mon[40588]: pgmap v948: 300 pgs: 32 unknown, 268 active+clean; 583 KiB data, 668 MiB used, 715 GiB / 715 GiB avail; 792 B/s rd, 1 op/s 2024-06-11T13:42:24.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:24 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:42:24.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:24 smithi057 ceph-mon[29805]: osdmap e607: 8 total, 8 up, 8 in 2024-06-11T13:42:24.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:24 smithi057 ceph-mon[29805]: pgmap v948: 300 pgs: 32 unknown, 268 active+clean; 583 KiB data, 668 MiB used, 715 GiB / 715 GiB avail; 792 B/s rd, 1 op/s 2024-06-11T13:42:24.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:24 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:42:24.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:24 smithi107 ceph-mon[33178]: osdmap e607: 8 total, 8 up, 8 in 2024-06-11T13:42:24.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:24 smithi107 ceph-mon[33178]: pgmap v948: 300 pgs: 32 unknown, 268 active+clean; 583 KiB data, 668 MiB used, 715 GiB / 715 GiB avail; 792 B/s rd, 1 op/s 2024-06-11T13:42:25.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:25 smithi057 ceph-mon[40588]: osdmap e608: 8 total, 8 up, 8 in 2024-06-11T13:42:25.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:25 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:42:25.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:25 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:42:25.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:25 smithi057 ceph-mon[29805]: osdmap e608: 8 total, 8 up, 8 in 2024-06-11T13:42:25.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:25 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:42:25.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:25 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:42:25.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:25 smithi107 ceph-mon[33178]: osdmap e608: 8 total, 8 up, 8 in 2024-06-11T13:42:25.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:25 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:42:25.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:25 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:42:26.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:26 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-118", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:42:26.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:26 smithi057 ceph-mon[29805]: osdmap e609: 8 total, 8 up, 8 in 2024-06-11T13:42:26.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:26 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-118"}]: dispatch 2024-06-11T13:42:26.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:26 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-118"}]: dispatch 2024-06-11T13:42:26.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:26 smithi057 ceph-mon[29805]: pgmap v951: 300 pgs: 32 unknown, 268 active+clean; 583 KiB data, 668 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:42:26.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:26 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-118", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:42:26.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:26 smithi057 ceph-mon[40588]: osdmap e609: 8 total, 8 up, 8 in 2024-06-11T13:42:26.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:26 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-118"}]: dispatch 2024-06-11T13:42:26.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:26 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-118"}]: dispatch 2024-06-11T13:42:26.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:26 smithi057 ceph-mon[40588]: pgmap v951: 300 pgs: 32 unknown, 268 active+clean; 583 KiB data, 668 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:42:26.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:26 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-118", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:42:26.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:26 smithi107 ceph-mon[33178]: osdmap e609: 8 total, 8 up, 8 in 2024-06-11T13:42:26.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:26 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-118"}]: dispatch 2024-06-11T13:42:26.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:26 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-118"}]: dispatch 2024-06-11T13:42:26.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:26 smithi107 ceph-mon[33178]: pgmap v951: 300 pgs: 32 unknown, 268 active+clean; 583 KiB data, 668 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:42:27.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:27 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-118"}]': finished 2024-06-11T13:42:27.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:27 smithi057 ceph-mon[40588]: osdmap e610: 8 total, 8 up, 8 in 2024-06-11T13:42:27.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:27 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-118", "mode": "writeback"}]: dispatch 2024-06-11T13:42:27.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:27 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-118", "mode": "writeback"}]: dispatch 2024-06-11T13:42:27.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:27 smithi057 ceph-mon[40588]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:42:27.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:27 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-118"}]': finished 2024-06-11T13:42:27.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:27 smithi057 ceph-mon[29805]: osdmap e610: 8 total, 8 up, 8 in 2024-06-11T13:42:27.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:27 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-118", "mode": "writeback"}]: dispatch 2024-06-11T13:42:27.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:27 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-118", "mode": "writeback"}]: dispatch 2024-06-11T13:42:27.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:27 smithi057 ceph-mon[29805]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:42:27.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:27 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-118"}]': finished 2024-06-11T13:42:27.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:27 smithi107 ceph-mon[33178]: osdmap e610: 8 total, 8 up, 8 in 2024-06-11T13:42:27.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:27 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-118", "mode": "writeback"}]: dispatch 2024-06-11T13:42:27.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:27 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-118", "mode": "writeback"}]: dispatch 2024-06-11T13:42:27.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:27 smithi107 ceph-mon[33178]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:42:28.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:28 smithi057 ceph-mon[29805]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:42:28.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:28 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-118", "mode": "writeback"}]': finished 2024-06-11T13:42:28.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:28 smithi057 ceph-mon[29805]: osdmap e611: 8 total, 8 up, 8 in 2024-06-11T13:42:28.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:28 smithi057 ceph-mon[29805]: pgmap v954: 300 pgs: 300 active+clean; 583 KiB data, 686 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-11T13:42:28.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:28 smithi057 ceph-mon[40588]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:42:28.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:28 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-118", "mode": "writeback"}]': finished 2024-06-11T13:42:28.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:28 smithi057 ceph-mon[40588]: osdmap e611: 8 total, 8 up, 8 in 2024-06-11T13:42:28.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:28 smithi057 ceph-mon[40588]: pgmap v954: 300 pgs: 300 active+clean; 583 KiB data, 686 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-11T13:42:28.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:28 smithi107 ceph-mon[33178]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:42:28.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:28 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-118", "mode": "writeback"}]': finished 2024-06-11T13:42:28.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:28 smithi107 ceph-mon[33178]: osdmap e611: 8 total, 8 up, 8 in 2024-06-11T13:42:28.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:28 smithi107 ceph-mon[33178]: pgmap v954: 300 pgs: 300 active+clean; 583 KiB data, 686 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-11T13:42:29.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:29 smithi057 ceph-mon[29805]: osdmap e612: 8 total, 8 up, 8 in 2024-06-11T13:42:29.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:29 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:42:29.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:29 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:42:29.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:29 smithi057 ceph-mon[40588]: osdmap e612: 8 total, 8 up, 8 in 2024-06-11T13:42:29.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:29 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:42:29.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:29 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:42:29.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:29 smithi107 ceph-mon[33178]: osdmap e612: 8 total, 8 up, 8 in 2024-06-11T13:42:29.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:29 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:42:29.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:29 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:42:30.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:30 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:42:30.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:30 smithi057 ceph-mon[29805]: osdmap e613: 8 total, 8 up, 8 in 2024-06-11T13:42:30.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:30 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-118"}]: dispatch 2024-06-11T13:42:30.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:30 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-118"}]: dispatch 2024-06-11T13:42:30.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:30 smithi057 ceph-mon[29805]: pgmap v957: 300 pgs: 300 active+clean; 583 KiB data, 686 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-11T13:42:30.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:30 smithi057 ceph-mon[29805]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:42:30.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:30 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-118"}]': finished 2024-06-11T13:42:30.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:30 smithi057 ceph-mon[29805]: osdmap e614: 8 total, 8 up, 8 in 2024-06-11T13:42:30.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:30 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:42:30.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:30 smithi057 ceph-mon[40588]: osdmap e613: 8 total, 8 up, 8 in 2024-06-11T13:42:30.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:30 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-118"}]: dispatch 2024-06-11T13:42:30.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:30 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-118"}]: dispatch 2024-06-11T13:42:30.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:30 smithi057 ceph-mon[40588]: pgmap v957: 300 pgs: 300 active+clean; 583 KiB data, 686 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-11T13:42:30.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:30 smithi057 ceph-mon[40588]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:42:30.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:30 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-118"}]': finished 2024-06-11T13:42:30.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:30 smithi057 ceph-mon[40588]: osdmap e614: 8 total, 8 up, 8 in 2024-06-11T13:42:30.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:30 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:42:30.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:30 smithi107 ceph-mon[33178]: osdmap e613: 8 total, 8 up, 8 in 2024-06-11T13:42:30.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:30 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-118"}]: dispatch 2024-06-11T13:42:30.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:30 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-118"}]: dispatch 2024-06-11T13:42:30.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:30 smithi107 ceph-mon[33178]: pgmap v957: 300 pgs: 300 active+clean; 583 KiB data, 686 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-11T13:42:30.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:30 smithi107 ceph-mon[33178]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:42:30.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:30 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-118"}]': finished 2024-06-11T13:42:30.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:30 smithi107 ceph-mon[33178]: osdmap e614: 8 total, 8 up, 8 in 2024-06-11T13:42:32.048 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:42:31 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:42:32.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:32 smithi057 ceph-mon[29805]: osdmap e615: 8 total, 8 up, 8 in 2024-06-11T13:42:32.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:32 smithi057 ceph-mon[29805]: pgmap v960: 268 pgs: 268 active+clean; 583 KiB data, 687 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:42:32.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:32 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:42:32.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:32 smithi057 ceph-mon[40588]: osdmap e615: 8 total, 8 up, 8 in 2024-06-11T13:42:32.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:32 smithi057 ceph-mon[40588]: pgmap v960: 268 pgs: 268 active+clean; 583 KiB data, 687 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:42:32.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:32 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:42:32.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:32 smithi107 ceph-mon[33178]: osdmap e615: 8 total, 8 up, 8 in 2024-06-11T13:42:32.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:32 smithi107 ceph-mon[33178]: pgmap v960: 268 pgs: 268 active+clean; 583 KiB data, 687 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:42:32.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:32 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:42:33.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:33 smithi057 ceph-mon[29805]: osdmap e616: 8 total, 8 up, 8 in 2024-06-11T13:42:33.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:33 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:42:33.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:33 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:42:33.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:33 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:42:33.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:33 smithi057 ceph-mon[40588]: osdmap e616: 8 total, 8 up, 8 in 2024-06-11T13:42:33.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:33 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:42:33.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:33 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:42:33.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:33 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:42:33.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:33 smithi107 ceph-mon[33178]: osdmap e616: 8 total, 8 up, 8 in 2024-06-11T13:42:33.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:33 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:42:33.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:33 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:42:33.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:33 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:42:34.035 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:42:33 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:42:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:42:34.538 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:34 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:42:34.538 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:34 smithi057 ceph-mon[40588]: osdmap e617: 8 total, 8 up, 8 in 2024-06-11T13:42:34.538 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:34 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:42:34.538 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:34 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:42:34.538 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:34 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:42:34.538 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:34 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:42:34.538 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:34 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:42:34.538 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:34 smithi057 ceph-mon[40588]: pgmap v963: 300 pgs: 32 unknown, 268 active+clean; 583 KiB data, 687 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:42:34.538 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:34 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-120", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:42:34.538 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:34 smithi057 ceph-mon[40588]: osdmap e618: 8 total, 8 up, 8 in 2024-06-11T13:42:34.539 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:34 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:42:34.539 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:34 smithi057 ceph-mon[29805]: osdmap e617: 8 total, 8 up, 8 in 2024-06-11T13:42:34.539 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:34 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:42:34.539 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:34 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:42:34.539 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:34 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:42:34.539 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:34 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:42:34.539 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:34 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:42:34.539 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:34 smithi057 ceph-mon[29805]: pgmap v963: 300 pgs: 32 unknown, 268 active+clean; 583 KiB data, 687 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:42:34.539 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:34 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-120", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:42:34.539 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:34 smithi057 ceph-mon[29805]: osdmap e618: 8 total, 8 up, 8 in 2024-06-11T13:42:34.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:34 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:42:34.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:34 smithi107 ceph-mon[33178]: osdmap e617: 8 total, 8 up, 8 in 2024-06-11T13:42:34.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:34 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:42:34.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:34 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:42:34.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:34 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:42:34.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:34 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:42:34.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:34 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:42:34.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:34 smithi107 ceph-mon[33178]: pgmap v963: 300 pgs: 32 unknown, 268 active+clean; 583 KiB data, 687 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:42:34.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:34 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-120", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:42:34.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:34 smithi107 ceph-mon[33178]: osdmap e618: 8 total, 8 up, 8 in 2024-06-11T13:42:35.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:35 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-120"}]: dispatch 2024-06-11T13:42:35.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:35 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-120"}]: dispatch 2024-06-11T13:42:35.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:35 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-120"}]: dispatch 2024-06-11T13:42:35.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:35 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-120"}]: dispatch 2024-06-11T13:42:35.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:35 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-120"}]: dispatch 2024-06-11T13:42:35.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:35 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-120"}]: dispatch 2024-06-11T13:42:36.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:36 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-120"}]': finished 2024-06-11T13:42:36.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:36 smithi107 ceph-mon[33178]: osdmap e619: 8 total, 8 up, 8 in 2024-06-11T13:42:36.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:36 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-120", "mode": "writeback"}]: dispatch 2024-06-11T13:42:36.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:36 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-120", "mode": "writeback"}]: dispatch 2024-06-11T13:42:36.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:36 smithi107 ceph-mon[33178]: pgmap v966: 300 pgs: 32 unknown, 268 active+clean; 583 KiB data, 687 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:42:36.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:36 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-120"}]': finished 2024-06-11T13:42:36.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:36 smithi057 ceph-mon[40588]: osdmap e619: 8 total, 8 up, 8 in 2024-06-11T13:42:36.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:36 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-120", "mode": "writeback"}]: dispatch 2024-06-11T13:42:36.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:36 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-120", "mode": "writeback"}]: dispatch 2024-06-11T13:42:36.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:36 smithi057 ceph-mon[40588]: pgmap v966: 300 pgs: 32 unknown, 268 active+clean; 583 KiB data, 687 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:42:36.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:36 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-120"}]': finished 2024-06-11T13:42:36.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:36 smithi057 ceph-mon[29805]: osdmap e619: 8 total, 8 up, 8 in 2024-06-11T13:42:36.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:36 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-120", "mode": "writeback"}]: dispatch 2024-06-11T13:42:36.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:36 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-120", "mode": "writeback"}]: dispatch 2024-06-11T13:42:36.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:36 smithi057 ceph-mon[29805]: pgmap v966: 300 pgs: 32 unknown, 268 active+clean; 583 KiB data, 687 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:42:37.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:37 smithi057 ceph-mon[40588]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:42:37.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:37 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-120", "mode": "writeback"}]': finished 2024-06-11T13:42:37.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:37 smithi057 ceph-mon[40588]: osdmap e620: 8 total, 8 up, 8 in 2024-06-11T13:42:37.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:37 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:42:37.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:37 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:42:37.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:37 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:42:37.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:37 smithi057 ceph-mon[40588]: osdmap e621: 8 total, 8 up, 8 in 2024-06-11T13:42:37.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:37 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-120"}]: dispatch 2024-06-11T13:42:37.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:37 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-120"}]: dispatch 2024-06-11T13:42:37.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:37 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:42:37.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:37 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:42:37.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:37 smithi057 ceph-mon[29805]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:42:37.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:37 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-120", "mode": "writeback"}]': finished 2024-06-11T13:42:37.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:37 smithi057 ceph-mon[29805]: osdmap e620: 8 total, 8 up, 8 in 2024-06-11T13:42:37.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:37 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:42:37.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:37 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:42:37.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:37 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:42:37.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:37 smithi057 ceph-mon[29805]: osdmap e621: 8 total, 8 up, 8 in 2024-06-11T13:42:37.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:37 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-120"}]: dispatch 2024-06-11T13:42:37.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:37 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-120"}]: dispatch 2024-06-11T13:42:37.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:37 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:42:37.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:37 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:42:37.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:37 smithi107 ceph-mon[33178]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:42:37.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:37 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-120", "mode": "writeback"}]': finished 2024-06-11T13:42:37.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:37 smithi107 ceph-mon[33178]: osdmap e620: 8 total, 8 up, 8 in 2024-06-11T13:42:37.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:37 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:42:37.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:37 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:42:37.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:37 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:42:37.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:37 smithi107 ceph-mon[33178]: osdmap e621: 8 total, 8 up, 8 in 2024-06-11T13:42:37.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:37 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-120"}]: dispatch 2024-06-11T13:42:37.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:37 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-120"}]: dispatch 2024-06-11T13:42:37.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:37 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:42:37.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:37 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:42:38.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:38 smithi107 ceph-mon[33178]: pgmap v969: 300 pgs: 300 active+clean; 583 KiB data, 687 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:42:38.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:38 smithi107 ceph-mon[33178]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:42:38.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:38 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-120"}]': finished 2024-06-11T13:42:38.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:38 smithi107 ceph-mon[33178]: osdmap e622: 8 total, 8 up, 8 in 2024-06-11T13:42:38.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:38 smithi057 ceph-mon[29805]: pgmap v969: 300 pgs: 300 active+clean; 583 KiB data, 687 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:42:38.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:38 smithi057 ceph-mon[29805]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:42:38.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:38 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-120"}]': finished 2024-06-11T13:42:38.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:38 smithi057 ceph-mon[29805]: osdmap e622: 8 total, 8 up, 8 in 2024-06-11T13:42:38.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:38 smithi057 ceph-mon[40588]: pgmap v969: 300 pgs: 300 active+clean; 583 KiB data, 687 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:42:38.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:38 smithi057 ceph-mon[40588]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:42:38.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:38 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-120"}]': finished 2024-06-11T13:42:38.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:38 smithi057 ceph-mon[40588]: osdmap e622: 8 total, 8 up, 8 in 2024-06-11T13:42:40.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:40 smithi057 ceph-mon[40588]: osdmap e623: 8 total, 8 up, 8 in 2024-06-11T13:42:40.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:40 smithi057 ceph-mon[40588]: pgmap v972: 268 pgs: 268 active+clean; 583 KiB data, 687 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:42:40.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:40 smithi057 ceph-mon[29805]: osdmap e623: 8 total, 8 up, 8 in 2024-06-11T13:42:40.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:40 smithi057 ceph-mon[29805]: pgmap v972: 268 pgs: 268 active+clean; 583 KiB data, 687 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:42:40.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:40 smithi107 ceph-mon[33178]: osdmap e623: 8 total, 8 up, 8 in 2024-06-11T13:42:40.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:40 smithi107 ceph-mon[33178]: pgmap v972: 268 pgs: 268 active+clean; 583 KiB data, 687 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:42:41.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:41 smithi057 ceph-mon[29805]: osdmap e624: 8 total, 8 up, 8 in 2024-06-11T13:42:41.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:41 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:42:41.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:41 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:42:41.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:41 smithi057 ceph-mon[40588]: osdmap e624: 8 total, 8 up, 8 in 2024-06-11T13:42:41.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:41 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:42:41.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:41 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:42:41.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:41 smithi107 ceph-mon[33178]: osdmap e624: 8 total, 8 up, 8 in 2024-06-11T13:42:41.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:41 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:42:41.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:41 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:42:42.048 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:42:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:42:42.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:42 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:42:42.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:42 smithi057 ceph-mon[29805]: osdmap e625: 8 total, 8 up, 8 in 2024-06-11T13:42:42.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:42 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:42:42.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:42 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:42:42.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:42 smithi057 ceph-mon[29805]: pgmap v975: 300 pgs: 32 creating+peering, 268 active+clean; 583 KiB data, 688 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:42:42.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:42 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:42:42.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:42 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:42:42.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:42 smithi057 ceph-mon[40588]: osdmap e625: 8 total, 8 up, 8 in 2024-06-11T13:42:42.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:42 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:42:42.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:42 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:42:42.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:42 smithi057 ceph-mon[40588]: pgmap v975: 300 pgs: 32 creating+peering, 268 active+clean; 583 KiB data, 688 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:42:42.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:42 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:42:42.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:42 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:42:42.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:42 smithi107 ceph-mon[33178]: osdmap e625: 8 total, 8 up, 8 in 2024-06-11T13:42:42.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:42 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:42:42.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:42 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:42:42.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:42 smithi107 ceph-mon[33178]: pgmap v975: 300 pgs: 32 creating+peering, 268 active+clean; 583 KiB data, 688 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:42:42.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:42 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:42:43.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:43 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-122", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:42:43.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:43 smithi057 ceph-mon[40588]: osdmap e626: 8 total, 8 up, 8 in 2024-06-11T13:42:43.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:43 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-122"}]: dispatch 2024-06-11T13:42:43.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:43 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-122"}]: dispatch 2024-06-11T13:42:43.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:43 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-122", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:42:43.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:43 smithi057 ceph-mon[29805]: osdmap e626: 8 total, 8 up, 8 in 2024-06-11T13:42:43.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:43 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-122"}]: dispatch 2024-06-11T13:42:43.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:43 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-122"}]: dispatch 2024-06-11T13:42:43.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:43 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-122", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:42:43.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:43 smithi107 ceph-mon[33178]: osdmap e626: 8 total, 8 up, 8 in 2024-06-11T13:42:43.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:43 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-122"}]: dispatch 2024-06-11T13:42:43.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:43 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-122"}]: dispatch 2024-06-11T13:42:44.035 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:42:43 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:42:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:42:44.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:44 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-122"}]': finished 2024-06-11T13:42:44.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:44 smithi057 ceph-mon[40588]: osdmap e627: 8 total, 8 up, 8 in 2024-06-11T13:42:44.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:44 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-122", "mode": "writeback"}]: dispatch 2024-06-11T13:42:44.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:44 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-122", "mode": "writeback"}]: dispatch 2024-06-11T13:42:44.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:44 smithi057 ceph-mon[40588]: pgmap v978: 300 pgs: 32 creating+peering, 268 active+clean; 583 KiB data, 688 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:42:44.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:44 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-122"}]': finished 2024-06-11T13:42:44.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:44 smithi057 ceph-mon[29805]: osdmap e627: 8 total, 8 up, 8 in 2024-06-11T13:42:44.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:44 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-122", "mode": "writeback"}]: dispatch 2024-06-11T13:42:44.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:44 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-122", "mode": "writeback"}]: dispatch 2024-06-11T13:42:44.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:44 smithi057 ceph-mon[29805]: pgmap v978: 300 pgs: 32 creating+peering, 268 active+clean; 583 KiB data, 688 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:42:44.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:44 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-122"}]': finished 2024-06-11T13:42:44.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:44 smithi107 ceph-mon[33178]: osdmap e627: 8 total, 8 up, 8 in 2024-06-11T13:42:44.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:44 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-122", "mode": "writeback"}]: dispatch 2024-06-11T13:42:44.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:44 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-122", "mode": "writeback"}]: dispatch 2024-06-11T13:42:44.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:44 smithi107 ceph-mon[33178]: pgmap v978: 300 pgs: 32 creating+peering, 268 active+clean; 583 KiB data, 688 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:42:45.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:45 smithi057 ceph-mon[29805]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:42:45.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:45 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-122", "mode": "writeback"}]': finished 2024-06-11T13:42:45.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:45 smithi057 ceph-mon[29805]: osdmap e628: 8 total, 8 up, 8 in 2024-06-11T13:42:45.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:45 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:42:45.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:45 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:42:45.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:45 smithi057 ceph-mon[40588]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:42:45.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:45 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-122", "mode": "writeback"}]': finished 2024-06-11T13:42:45.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:45 smithi057 ceph-mon[40588]: osdmap e628: 8 total, 8 up, 8 in 2024-06-11T13:42:45.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:45 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:42:45.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:45 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:42:45.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:45 smithi107 ceph-mon[33178]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:42:45.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:45 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-122", "mode": "writeback"}]': finished 2024-06-11T13:42:45.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:45 smithi107 ceph-mon[33178]: osdmap e628: 8 total, 8 up, 8 in 2024-06-11T13:42:45.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:45 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:42:45.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:45 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:42:46.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:46 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:42:46.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:46 smithi057 ceph-mon[29805]: osdmap e629: 8 total, 8 up, 8 in 2024-06-11T13:42:46.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:46 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-122"}]: dispatch 2024-06-11T13:42:46.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:46 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-122"}]: dispatch 2024-06-11T13:42:46.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:46 smithi057 ceph-mon[29805]: pgmap v981: 300 pgs: 32 creating+peering, 268 active+clean; 583 KiB data, 688 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:42:46.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:46 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:42:46.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:46 smithi057 ceph-mon[40588]: osdmap e629: 8 total, 8 up, 8 in 2024-06-11T13:42:46.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:46 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-122"}]: dispatch 2024-06-11T13:42:46.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:46 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-122"}]: dispatch 2024-06-11T13:42:46.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:46 smithi057 ceph-mon[40588]: pgmap v981: 300 pgs: 32 creating+peering, 268 active+clean; 583 KiB data, 688 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:42:46.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:46 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:42:46.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:46 smithi107 ceph-mon[33178]: osdmap e629: 8 total, 8 up, 8 in 2024-06-11T13:42:46.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:46 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-122"}]: dispatch 2024-06-11T13:42:46.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:46 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-122"}]: dispatch 2024-06-11T13:42:46.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:46 smithi107 ceph-mon[33178]: pgmap v981: 300 pgs: 32 creating+peering, 268 active+clean; 583 KiB data, 688 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:42:47.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:47 smithi057 ceph-mon[29805]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:42:47.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:47 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-122"}]': finished 2024-06-11T13:42:47.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:47 smithi057 ceph-mon[29805]: osdmap e630: 8 total, 8 up, 8 in 2024-06-11T13:42:47.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:47 smithi057 ceph-mon[40588]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:42:47.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:47 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-122"}]': finished 2024-06-11T13:42:47.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:47 smithi057 ceph-mon[40588]: osdmap e630: 8 total, 8 up, 8 in 2024-06-11T13:42:47.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:47 smithi107 ceph-mon[33178]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:42:47.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:47 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-122"}]': finished 2024-06-11T13:42:47.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:47 smithi107 ceph-mon[33178]: osdmap e630: 8 total, 8 up, 8 in 2024-06-11T13:42:48.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:48 smithi057 ceph-mon[29805]: osdmap e631: 8 total, 8 up, 8 in 2024-06-11T13:42:48.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:48 smithi057 ceph-mon[29805]: pgmap v984: 268 pgs: 268 active+clean; 583 KiB data, 693 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-06-11T13:42:48.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:48 smithi057 ceph-mon[40588]: osdmap e631: 8 total, 8 up, 8 in 2024-06-11T13:42:48.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:48 smithi057 ceph-mon[40588]: pgmap v984: 268 pgs: 268 active+clean; 583 KiB data, 693 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-06-11T13:42:48.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:48 smithi107 ceph-mon[33178]: osdmap e631: 8 total, 8 up, 8 in 2024-06-11T13:42:48.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:48 smithi107 ceph-mon[33178]: pgmap v984: 268 pgs: 268 active+clean; 583 KiB data, 693 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-06-11T13:42:49.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:49 smithi057 ceph-mon[29805]: osdmap e632: 8 total, 8 up, 8 in 2024-06-11T13:42:49.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:49 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:42:49.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:49 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:42:49.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:49 smithi057 ceph-mon[40588]: osdmap e632: 8 total, 8 up, 8 in 2024-06-11T13:42:49.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:49 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:42:49.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:49 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:42:49.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:49 smithi107 ceph-mon[33178]: osdmap e632: 8 total, 8 up, 8 in 2024-06-11T13:42:49.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:49 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:42:49.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:49 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:42:50.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:50 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:42:50.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:50 smithi057 ceph-mon[29805]: osdmap e633: 8 total, 8 up, 8 in 2024-06-11T13:42:50.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:50 smithi057 ceph-mon[29805]: pgmap v987: 300 pgs: 32 unknown, 268 active+clean; 583 KiB data, 693 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-06-11T13:42:50.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:50 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:42:50.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:50 smithi057 ceph-mon[40588]: osdmap e633: 8 total, 8 up, 8 in 2024-06-11T13:42:50.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:50 smithi057 ceph-mon[40588]: pgmap v987: 300 pgs: 32 unknown, 268 active+clean; 583 KiB data, 693 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-06-11T13:42:50.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:50 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:42:50.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:50 smithi107 ceph-mon[33178]: osdmap e633: 8 total, 8 up, 8 in 2024-06-11T13:42:50.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:50 smithi107 ceph-mon[33178]: pgmap v987: 300 pgs: 32 unknown, 268 active+clean; 583 KiB data, 693 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-06-11T13:42:51.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:51 smithi057 ceph-mon[29805]: osdmap e634: 8 total, 8 up, 8 in 2024-06-11T13:42:51.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:51 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:42:51.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:51 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:42:51.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:51 smithi057 ceph-mon[40588]: osdmap e634: 8 total, 8 up, 8 in 2024-06-11T13:42:51.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:51 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:42:51.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:51 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:42:51.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:51 smithi107 ceph-mon[33178]: osdmap e634: 8 total, 8 up, 8 in 2024-06-11T13:42:51.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:51 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:42:51.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:51 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:42:52.048 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:42:51 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:42:52.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:52 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-124", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:42:52.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:52 smithi107 ceph-mon[33178]: osdmap e635: 8 total, 8 up, 8 in 2024-06-11T13:42:52.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:52 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-124"}]: dispatch 2024-06-11T13:42:52.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:52 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-124"}]: dispatch 2024-06-11T13:42:52.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:52 smithi107 ceph-mon[33178]: pgmap v990: 300 pgs: 300 active+clean; 583 KiB data, 701 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-06-11T13:42:52.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:52 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:42:52.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:52 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:42:52.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:52 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:42:52.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:52 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-124", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:42:52.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:52 smithi057 ceph-mon[40588]: osdmap e635: 8 total, 8 up, 8 in 2024-06-11T13:42:52.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:52 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-124"}]: dispatch 2024-06-11T13:42:52.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:52 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-124"}]: dispatch 2024-06-11T13:42:52.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:52 smithi057 ceph-mon[40588]: pgmap v990: 300 pgs: 300 active+clean; 583 KiB data, 701 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-06-11T13:42:52.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:52 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:42:52.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:52 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:42:52.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:52 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:42:52.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:52 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-124", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:42:52.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:52 smithi057 ceph-mon[29805]: osdmap e635: 8 total, 8 up, 8 in 2024-06-11T13:42:52.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:52 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-124"}]: dispatch 2024-06-11T13:42:52.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:52 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-124"}]: dispatch 2024-06-11T13:42:52.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:52 smithi057 ceph-mon[29805]: pgmap v990: 300 pgs: 300 active+clean; 583 KiB data, 701 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-06-11T13:42:52.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:52 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:42:52.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:52 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:42:52.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:52 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:42:53.733 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:53 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-124"}]': finished 2024-06-11T13:42:53.733 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:53 smithi057 ceph-mon[40588]: osdmap e636: 8 total, 8 up, 8 in 2024-06-11T13:42:53.733 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:53 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-124", "mode": "writeback"}]: dispatch 2024-06-11T13:42:53.734 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:53 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-124", "mode": "writeback"}]: dispatch 2024-06-11T13:42:53.734 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:53 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-124"}]': finished 2024-06-11T13:42:53.734 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:53 smithi057 ceph-mon[29805]: osdmap e636: 8 total, 8 up, 8 in 2024-06-11T13:42:53.734 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:53 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-124", "mode": "writeback"}]: dispatch 2024-06-11T13:42:53.734 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:53 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-124", "mode": "writeback"}]: dispatch 2024-06-11T13:42:53.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:53 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-124"}]': finished 2024-06-11T13:42:53.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:53 smithi107 ceph-mon[33178]: osdmap e636: 8 total, 8 up, 8 in 2024-06-11T13:42:53.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:53 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-124", "mode": "writeback"}]: dispatch 2024-06-11T13:42:53.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:53 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-124", "mode": "writeback"}]: dispatch 2024-06-11T13:42:54.035 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:42:53 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:42:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:42:54.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:54 smithi057 ceph-mon[40588]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:42:54.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:54 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-124", "mode": "writeback"}]': finished 2024-06-11T13:42:54.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:54 smithi057 ceph-mon[40588]: osdmap e637: 8 total, 8 up, 8 in 2024-06-11T13:42:54.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:54 smithi057 ceph-mon[40588]: pgmap v993: 300 pgs: 300 active+clean; 583 KiB data, 701 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-06-11T13:42:54.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:54 smithi057 ceph-mon[29805]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:42:54.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:54 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-124", "mode": "writeback"}]': finished 2024-06-11T13:42:54.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:54 smithi057 ceph-mon[29805]: osdmap e637: 8 total, 8 up, 8 in 2024-06-11T13:42:54.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:54 smithi057 ceph-mon[29805]: pgmap v993: 300 pgs: 300 active+clean; 583 KiB data, 701 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-06-11T13:42:54.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:54 smithi107 ceph-mon[33178]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:42:54.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:54 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-124", "mode": "writeback"}]': finished 2024-06-11T13:42:54.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:54 smithi107 ceph-mon[33178]: osdmap e637: 8 total, 8 up, 8 in 2024-06-11T13:42:54.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:54 smithi107 ceph-mon[33178]: pgmap v993: 300 pgs: 300 active+clean; 583 KiB data, 701 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-06-11T13:42:55.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:55 smithi057 ceph-mon[40588]: osdmap e638: 8 total, 8 up, 8 in 2024-06-11T13:42:55.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:55 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:42:55.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:55 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:42:55.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:55 smithi057 ceph-mon[29805]: osdmap e638: 8 total, 8 up, 8 in 2024-06-11T13:42:55.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:55 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:42:55.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:55 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:42:55.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:55 smithi107 ceph-mon[33178]: osdmap e638: 8 total, 8 up, 8 in 2024-06-11T13:42:55.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:55 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:42:55.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:55 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:42:56.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:56 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:42:56.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:56 smithi057 ceph-mon[40588]: osdmap e639: 8 total, 8 up, 8 in 2024-06-11T13:42:56.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:56 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-124"}]: dispatch 2024-06-11T13:42:56.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:56 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-124"}]: dispatch 2024-06-11T13:42:56.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:56 smithi057 ceph-mon[40588]: pgmap v996: 300 pgs: 300 active+clean; 583 KiB data, 701 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:42:56.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:56 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:42:56.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:56 smithi057 ceph-mon[29805]: osdmap e639: 8 total, 8 up, 8 in 2024-06-11T13:42:56.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:56 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-124"}]: dispatch 2024-06-11T13:42:56.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:56 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-124"}]: dispatch 2024-06-11T13:42:56.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:56 smithi057 ceph-mon[29805]: pgmap v996: 300 pgs: 300 active+clean; 583 KiB data, 701 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:42:56.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:56 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:42:56.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:56 smithi107 ceph-mon[33178]: osdmap e639: 8 total, 8 up, 8 in 2024-06-11T13:42:56.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:56 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-124"}]: dispatch 2024-06-11T13:42:56.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:56 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-124"}]: dispatch 2024-06-11T13:42:56.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:56 smithi107 ceph-mon[33178]: pgmap v996: 300 pgs: 300 active+clean; 583 KiB data, 701 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:42:57.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:57 smithi057 ceph-mon[40588]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:42:57.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:57 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-124"}]': finished 2024-06-11T13:42:57.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:57 smithi057 ceph-mon[40588]: osdmap e640: 8 total, 8 up, 8 in 2024-06-11T13:42:57.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:57 smithi057 ceph-mon[29805]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:42:57.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:57 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-124"}]': finished 2024-06-11T13:42:57.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:57 smithi057 ceph-mon[29805]: osdmap e640: 8 total, 8 up, 8 in 2024-06-11T13:42:57.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:57 smithi107 ceph-mon[33178]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:42:57.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:57 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-124"}]': finished 2024-06-11T13:42:57.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:57 smithi107 ceph-mon[33178]: osdmap e640: 8 total, 8 up, 8 in 2024-06-11T13:42:58.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:58 smithi057 ceph-mon[40588]: osdmap e641: 8 total, 8 up, 8 in 2024-06-11T13:42:58.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:58 smithi057 ceph-mon[40588]: pgmap v999: 268 pgs: 268 active+clean; 583 KiB data, 702 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:42:58.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:58 smithi057 ceph-mon[29805]: osdmap e641: 8 total, 8 up, 8 in 2024-06-11T13:42:58.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:58 smithi057 ceph-mon[29805]: pgmap v999: 268 pgs: 268 active+clean; 583 KiB data, 702 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:42:58.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:58 smithi107 ceph-mon[33178]: osdmap e641: 8 total, 8 up, 8 in 2024-06-11T13:42:58.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:58 smithi107 ceph-mon[33178]: pgmap v999: 268 pgs: 268 active+clean; 583 KiB data, 702 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:42:59.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:59 smithi057 ceph-mon[40588]: osdmap e642: 8 total, 8 up, 8 in 2024-06-11T13:42:59.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:59 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:42:59.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:42:59 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:42:59.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:59 smithi057 ceph-mon[29805]: osdmap e642: 8 total, 8 up, 8 in 2024-06-11T13:42:59.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:59 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:42:59.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:42:59 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:42:59.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:59 smithi107 ceph-mon[33178]: osdmap e642: 8 total, 8 up, 8 in 2024-06-11T13:42:59.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:59 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:42:59.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:42:59 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:43:00.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:00 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:43:00.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:00 smithi057 ceph-mon[40588]: osdmap e643: 8 total, 8 up, 8 in 2024-06-11T13:43:00.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:00 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:43:00.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:00 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:43:00.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:00 smithi057 ceph-mon[40588]: pgmap v1002: 300 pgs: 32 unknown, 268 active+clean; 583 KiB data, 702 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:43:00.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:00 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:43:00.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:00 smithi057 ceph-mon[29805]: osdmap e643: 8 total, 8 up, 8 in 2024-06-11T13:43:00.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:00 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:43:00.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:00 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:43:00.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:00 smithi057 ceph-mon[29805]: pgmap v1002: 300 pgs: 32 unknown, 268 active+clean; 583 KiB data, 702 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:43:00.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:00 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:43:00.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:00 smithi107 ceph-mon[33178]: osdmap e643: 8 total, 8 up, 8 in 2024-06-11T13:43:00.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:00 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:43:00.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:00 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:43:00.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:00 smithi107 ceph-mon[33178]: pgmap v1002: 300 pgs: 32 unknown, 268 active+clean; 583 KiB data, 702 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:43:01.765 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:01 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-126", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:43:01.765 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:01 smithi107 ceph-mon[33178]: osdmap e644: 8 total, 8 up, 8 in 2024-06-11T13:43:01.765 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:01 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-126"}]: dispatch 2024-06-11T13:43:01.766 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:01 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-126"}]: dispatch 2024-06-11T13:43:01.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:01 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-126", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:43:01.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:01 smithi057 ceph-mon[40588]: osdmap e644: 8 total, 8 up, 8 in 2024-06-11T13:43:01.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:01 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-126"}]: dispatch 2024-06-11T13:43:01.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:01 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-126"}]: dispatch 2024-06-11T13:43:01.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:01 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-126", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:43:01.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:01 smithi057 ceph-mon[29805]: osdmap e644: 8 total, 8 up, 8 in 2024-06-11T13:43:01.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:01 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-126"}]: dispatch 2024-06-11T13:43:01.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:01 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-126"}]: dispatch 2024-06-11T13:43:02.048 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:43:01 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:43:02.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:02 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-126"}]': finished 2024-06-11T13:43:02.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:02 smithi057 ceph-mon[40588]: osdmap e645: 8 total, 8 up, 8 in 2024-06-11T13:43:02.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:02 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-126", "mode": "writeback"}]: dispatch 2024-06-11T13:43:02.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:02 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-126", "mode": "writeback"}]: dispatch 2024-06-11T13:43:02.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:02 smithi057 ceph-mon[40588]: pgmap v1005: 300 pgs: 300 active+clean; 583 KiB data, 707 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:43:02.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:02 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:43:02.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:02 smithi057 ceph-mon[40588]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:43:02.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:02 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-126", "mode": "writeback"}]': finished 2024-06-11T13:43:02.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:02 smithi057 ceph-mon[40588]: osdmap e646: 8 total, 8 up, 8 in 2024-06-11T13:43:02.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:02 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:43:02.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:02 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:43:02.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:02 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-126"}]': finished 2024-06-11T13:43:02.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:02 smithi057 ceph-mon[29805]: osdmap e645: 8 total, 8 up, 8 in 2024-06-11T13:43:02.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:02 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-126", "mode": "writeback"}]: dispatch 2024-06-11T13:43:02.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:02 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-126", "mode": "writeback"}]: dispatch 2024-06-11T13:43:02.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:02 smithi057 ceph-mon[29805]: pgmap v1005: 300 pgs: 300 active+clean; 583 KiB data, 707 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:43:02.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:02 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:43:02.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:02 smithi057 ceph-mon[29805]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:43:02.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:02 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-126", "mode": "writeback"}]': finished 2024-06-11T13:43:02.788 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:02 smithi057 ceph-mon[29805]: osdmap e646: 8 total, 8 up, 8 in 2024-06-11T13:43:02.788 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:02 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:43:02.788 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:02 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:43:02.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:02 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-126"}]': finished 2024-06-11T13:43:02.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:02 smithi107 ceph-mon[33178]: osdmap e645: 8 total, 8 up, 8 in 2024-06-11T13:43:02.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:02 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-126", "mode": "writeback"}]: dispatch 2024-06-11T13:43:02.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:02 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-126", "mode": "writeback"}]: dispatch 2024-06-11T13:43:02.800 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:02 smithi107 ceph-mon[33178]: pgmap v1005: 300 pgs: 300 active+clean; 583 KiB data, 707 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:43:02.800 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:02 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:43:02.800 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:02 smithi107 ceph-mon[33178]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:43:02.800 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:02 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-126", "mode": "writeback"}]': finished 2024-06-11T13:43:02.800 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:02 smithi107 ceph-mon[33178]: osdmap e646: 8 total, 8 up, 8 in 2024-06-11T13:43:02.800 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:02 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:43:02.800 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:02 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:43:04.035 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:43:03 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:43:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:43:04.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:04 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:43:04.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:04 smithi057 ceph-mon[29805]: osdmap e647: 8 total, 8 up, 8 in 2024-06-11T13:43:04.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:04 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-126"}]: dispatch 2024-06-11T13:43:04.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:04 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-126"}]: dispatch 2024-06-11T13:43:04.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:04 smithi057 ceph-mon[29805]: pgmap v1008: 300 pgs: 300 active+clean; 583 KiB data, 707 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:43:04.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:04 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:43:04.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:04 smithi057 ceph-mon[40588]: osdmap e647: 8 total, 8 up, 8 in 2024-06-11T13:43:04.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:04 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-126"}]: dispatch 2024-06-11T13:43:04.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:04 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-126"}]: dispatch 2024-06-11T13:43:04.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:04 smithi057 ceph-mon[40588]: pgmap v1008: 300 pgs: 300 active+clean; 583 KiB data, 707 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:43:04.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:04 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:43:04.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:04 smithi107 ceph-mon[33178]: osdmap e647: 8 total, 8 up, 8 in 2024-06-11T13:43:04.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:04 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-126"}]: dispatch 2024-06-11T13:43:04.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:04 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-126"}]: dispatch 2024-06-11T13:43:04.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:04 smithi107 ceph-mon[33178]: pgmap v1008: 300 pgs: 300 active+clean; 583 KiB data, 707 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:43:05.138 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 se api_tier_pp: require_osd_release = squid 2024-06-11T13:43:05.138 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringFlush (9077 ms) 2024-06-11T13:43:05.138 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2024-06-11T13:43:05.138 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: require_osd_release = squid 2024-06-11T13:43:05.138 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapHasChunk (6057 ms) 2024-06-11T13:43:05.138 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollback 2024-06-11T13:43:05.138 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: require_osd_release = squid 2024-06-11T13:43:05.139 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollback (5034 ms) 2024-06-11T13:43:05.139 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollbackRefcount 2024-06-11T13:43:05.139 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: require_osd_release = squid 2024-06-11T13:43:05.139 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollbackRefcount (25514 ms) 2024-06-11T13:43:05.139 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictRollback 2024-06-11T13:43:05.139 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: require_osd_release = squid 2024-06-11T13:43:05.139 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictRollback (13101 ms) 2024-06-11T13:43:05.139 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PropagateBaseTierError 2024-06-11T13:43:05.139 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PropagateBaseTierError (12130 ms) 2024-06-11T13:43:05.139 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HelloWriteReturn 2024-06-11T13:43:05.140 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: 00000000 79 6f 75 20 6d 69 67 68 74 20 73 65 65 20 74 68 |you might see th| 2024-06-11T13:43:05.140 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: 00000010 69 73 |is| 2024-06-11T13:43:05.140 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: 00000012 2024-06-11T13:43:05.140 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HelloWriteReturn (12115 ms) 2024-06-11T13:43:05.140 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2024-06-11T13:43:05.140 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: require_osd_release = squid 2024-06-11T13:43:05.140 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier (6097 ms) 2024-06-11T13:43:05.140 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP (565528 ms total) 2024-06-11T13:43:05.140 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: 2024-06-11T13:43:05.140 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP 2024-06-11T13:43:05.140 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.Dirty 2024-06-11T13:43:05.140 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.Dirty (1011 ms) 2024-06-11T13:43:05.140 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.FlushWriteRaces 2024-06-11T13:43:05.140 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.FlushWriteRaces (11112 ms) 2024-06-11T13:43:05.141 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.CallForcesPromote 2024-06-11T13:43:05.141 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.CallForcesPromote (18162 ms) 2024-06-11T13:43:05.141 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.HitSetNone 2024-06-11T13:43:05.141 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.HitSetNone (3 ms) 2024-06-11T13:43:05.141 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP (30288 ms total) 2024-06-11T13:43:05.141 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: 2024-06-11T13:43:05.141 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP 2024-06-11T13:43:05.141 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Overlay 2024-06-11T13:43:05.141 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Overlay (7048 ms) 2024-06-11T13:43:05.141 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Promote 2024-06-11T13:43:05.141 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Promote (8123 ms) 2024-06-11T13:43:05.141 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnap 2024-06-11T13:43:05.142 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: waiting for scrub... 2024-06-11T13:43:05.142 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: done waiting 2024-06-11T13:43:05.142 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnap (24957 ms) 2024-06-11T13:43:05.142 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2024-06-11T13:43:05.143 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace (9613 ms) 2024-06-11T13:43:05.143 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Whiteout 2024-06-11T13:43:05.143 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Whiteout (7909 ms) 2024-06-11T13:43:05.143 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Evict 2024-06-11T13:43:05.143 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Evict (8115 ms) 2024-06-11T13:43:05.143 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.EvictSnap 2024-06-11T13:43:05.143 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.EvictSnap (10156 ms) 2024-06-11T13:43:05.143 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlush 2024-06-11T13:43:05.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:05 smithi057 ceph-mon[40588]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:43:05.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:05 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-126"}]': finished 2024-06-11T13:43:05.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:05 smithi057 ceph-mon[40588]: osdmap e648: 8 total, 8 up, 8 in 2024-06-11T13:43:05.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:05 smithi057 ceph-mon[29805]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:43:05.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:05 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-126"}]': finished 2024-06-11T13:43:05.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:05 smithi057 ceph-mon[29805]: osdmap e648: 8 total, 8 up, 8 in 2024-06-11T13:43:05.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:05 smithi107 ceph-mon[33178]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:43:05.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:05 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-126"}]': finished 2024-06-11T13:43:05.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:05 smithi107 ceph-mon[33178]: osdmap e648: 8 total, 8 up, 8 in 2024-06-11T13:43:06.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:06 smithi057 ceph-mon[40588]: osdmap e649: 8 total, 8 up, 8 in 2024-06-11T13:43:06.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:06 smithi057 ceph-mon[40588]: pgmap v1011: 268 pgs: 268 active+clean; 583 KiB data, 707 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:43:06.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:06 smithi057 ceph-mon[29805]: osdmap e649: 8 total, 8 up, 8 in 2024-06-11T13:43:06.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:06 smithi057 ceph-mon[29805]: pgmap v1011: 268 pgs: 268 active+clean; 583 KiB data, 707 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:43:06.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:06 smithi107 ceph-mon[33178]: osdmap e649: 8 total, 8 up, 8 in 2024-06-11T13:43:06.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:06 smithi107 ceph-mon[33178]: pgmap v1011: 268 pgs: 268 active+clean; 583 KiB data, 707 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:43:07.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:07 smithi057 ceph-mon[40588]: osdmap e650: 8 total, 8 up, 8 in 2024-06-11T13:43:07.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:07 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:43:07.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:07 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:43:07.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:07 smithi057 ceph-mon[29805]: osdmap e650: 8 total, 8 up, 8 in 2024-06-11T13:43:07.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:07 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:43:07.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:07 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:43:07.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:07 smithi107 ceph-mon[33178]: osdmap e650: 8 total, 8 up, 8 in 2024-06-11T13:43:07.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:07 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:43:07.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:07 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:43:08.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:08 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:43:08.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:08 smithi057 ceph-mon[29805]: osdmap e651: 8 total, 8 up, 8 in 2024-06-11T13:43:08.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:08 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:43:08.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:08 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:43:08.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:08 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:43:08.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:08 smithi057 ceph-mon[29805]: pgmap v1014: 300 pgs: 32 unknown, 268 active+clean; 583 KiB data, 707 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:43:08.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:08 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:43:08.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:08 smithi057 ceph-mon[40588]: osdmap e651: 8 total, 8 up, 8 in 2024-06-11T13:43:08.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:08 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:43:08.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:08 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:43:08.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:08 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:43:08.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:08 smithi057 ceph-mon[40588]: pgmap v1014: 300 pgs: 32 unknown, 268 active+clean; 583 KiB data, 707 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:43:08.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:08 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:43:08.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:08 smithi107 ceph-mon[33178]: osdmap e651: 8 total, 8 up, 8 in 2024-06-11T13:43:08.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:08 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:43:08.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:08 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:43:08.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:08 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:43:08.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:08 smithi107 ceph-mon[33178]: pgmap v1014: 300 pgs: 32 unknown, 268 active+clean; 583 KiB data, 707 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:43:09.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:09 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-128", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:43:09.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:09 smithi057 ceph-mon[40588]: osdmap e652: 8 total, 8 up, 8 in 2024-06-11T13:43:09.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:09 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-128"}]: dispatch 2024-06-11T13:43:09.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:09 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-128"}]: dispatch 2024-06-11T13:43:09.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:09 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-128", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:43:09.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:09 smithi057 ceph-mon[29805]: osdmap e652: 8 total, 8 up, 8 in 2024-06-11T13:43:09.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:09 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-128"}]: dispatch 2024-06-11T13:43:09.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:09 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-128"}]: dispatch 2024-06-11T13:43:09.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:09 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-128", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:43:09.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:09 smithi107 ceph-mon[33178]: osdmap e652: 8 total, 8 up, 8 in 2024-06-11T13:43:09.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:09 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-128"}]: dispatch 2024-06-11T13:43:09.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:09 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-128"}]: dispatch 2024-06-11T13:43:10.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:10 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-128"}]': finished 2024-06-11T13:43:10.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:10 smithi057 ceph-mon[29805]: osdmap e653: 8 total, 8 up, 8 in 2024-06-11T13:43:10.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:10 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-128", "mode": "writeback"}]: dispatch 2024-06-11T13:43:10.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:10 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-128", "mode": "writeback"}]: dispatch 2024-06-11T13:43:10.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:10 smithi057 ceph-mon[29805]: pgmap v1017: 300 pgs: 32 unknown, 268 active+clean; 583 KiB data, 707 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:43:10.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:10 smithi057 ceph-mon[29805]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:43:10.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:10 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-128", "mode": "writeback"}]': finished 2024-06-11T13:43:10.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:10 smithi057 ceph-mon[29805]: osdmap e654: 8 total, 8 up, 8 in 2024-06-11T13:43:10.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:10 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-128"}]': finished 2024-06-11T13:43:10.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:10 smithi057 ceph-mon[40588]: osdmap e653: 8 total, 8 up, 8 in 2024-06-11T13:43:10.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:10 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-128", "mode": "writeback"}]: dispatch 2024-06-11T13:43:10.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:10 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-128", "mode": "writeback"}]: dispatch 2024-06-11T13:43:10.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:10 smithi057 ceph-mon[40588]: pgmap v1017: 300 pgs: 32 unknown, 268 active+clean; 583 KiB data, 707 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:43:10.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:10 smithi057 ceph-mon[40588]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:43:10.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:10 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-128", "mode": "writeback"}]': finished 2024-06-11T13:43:10.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:10 smithi057 ceph-mon[40588]: osdmap e654: 8 total, 8 up, 8 in 2024-06-11T13:43:10.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:10 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-128"}]': finished 2024-06-11T13:43:10.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:10 smithi107 ceph-mon[33178]: osdmap e653: 8 total, 8 up, 8 in 2024-06-11T13:43:10.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:10 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-128", "mode": "writeback"}]: dispatch 2024-06-11T13:43:10.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:10 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-128", "mode": "writeback"}]: dispatch 2024-06-11T13:43:10.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:10 smithi107 ceph-mon[33178]: pgmap v1017: 300 pgs: 32 unknown, 268 active+clean; 583 KiB data, 707 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:43:10.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:10 smithi107 ceph-mon[33178]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:43:10.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:10 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-128", "mode": "writeback"}]': finished 2024-06-11T13:43:10.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:10 smithi107 ceph-mon[33178]: osdmap e654: 8 total, 8 up, 8 in 2024-06-11T13:43:12.048 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:43:11 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:43:12.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:12 smithi057 ceph-mon[29805]: pgmap v1019: 300 pgs: 300 active+clean; 583 KiB data, 696 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 475 B/s wr, 1 op/s 2024-06-11T13:43:12.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:12 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:43:12.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:12 smithi057 ceph-mon[40588]: pgmap v1019: 300 pgs: 300 active+clean; 583 KiB data, 696 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 475 B/s wr, 1 op/s 2024-06-11T13:43:12.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:12 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:43:12.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:12 smithi107 ceph-mon[33178]: pgmap v1019: 300 pgs: 300 active+clean; 583 KiB data, 696 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 475 B/s wr, 1 op/s 2024-06-11T13:43:12.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:12 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:43:14.035 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:43:13 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:43:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:43:14.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:14 smithi057 ceph-mon[29805]: pgmap v1020: 300 pgs: 300 active+clean; 583 KiB data, 696 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 341 B/s wr, 1 op/s 2024-06-11T13:43:14.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:14 smithi057 ceph-mon[40588]: pgmap v1020: 300 pgs: 300 active+clean; 583 KiB data, 696 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 341 B/s wr, 1 op/s 2024-06-11T13:43:14.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:14 smithi107 ceph-mon[33178]: pgmap v1020: 300 pgs: 300 active+clean; 583 KiB data, 696 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 341 B/s wr, 1 op/s 2024-06-11T13:43:15.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:15 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:43:15.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:15 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:43:15.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:15 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:43:15.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:15 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:43:15.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:15 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:43:15.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:15 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:43:16.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:16 smithi107 ceph-mon[33178]: pgmap v1021: 300 pgs: 300 active+clean; 583 KiB data, 696 MiB used, 715 GiB / 715 GiB avail; 842 B/s rd, 280 B/s wr, 1 op/s 2024-06-11T13:43:16.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:16 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:43:16.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:16 smithi107 ceph-mon[33178]: osdmap e655: 8 total, 8 up, 8 in 2024-06-11T13:43:16.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:16 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-128"}]: dispatch 2024-06-11T13:43:16.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:16 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-128"}]: dispatch 2024-06-11T13:43:17.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:16 smithi057 ceph-mon[29805]: pgmap v1021: 300 pgs: 300 active+clean; 583 KiB data, 696 MiB used, 715 GiB / 715 GiB avail; 842 B/s rd, 280 B/s wr, 1 op/s 2024-06-11T13:43:17.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:16 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:43:17.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:16 smithi057 ceph-mon[29805]: osdmap e655: 8 total, 8 up, 8 in 2024-06-11T13:43:17.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:16 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-128"}]: dispatch 2024-06-11T13:43:17.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:16 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-128"}]: dispatch 2024-06-11T13:43:17.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:16 smithi057 ceph-mon[40588]: pgmap v1021: 300 pgs: 300 active+clean; 583 KiB data, 696 MiB used, 715 GiB / 715 GiB avail; 842 B/s rd, 280 B/s wr, 1 op/s 2024-06-11T13:43:17.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:16 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:43:17.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:16 smithi057 ceph-mon[40588]: osdmap e655: 8 total, 8 up, 8 in 2024-06-11T13:43:17.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:16 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-128"}]: dispatch 2024-06-11T13:43:17.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:16 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-128"}]: dispatch 2024-06-11T13:43:18.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:17 smithi057 ceph-mon[29805]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:43:18.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:17 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-128"}]': finished 2024-06-11T13:43:18.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:17 smithi057 ceph-mon[29805]: osdmap e656: 8 total, 8 up, 8 in 2024-06-11T13:43:18.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:17 smithi057 ceph-mon[40588]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:43:18.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:17 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-128"}]': finished 2024-06-11T13:43:18.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:17 smithi057 ceph-mon[40588]: osdmap e656: 8 total, 8 up, 8 in 2024-06-11T13:43:18.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:17 smithi107 ceph-mon[33178]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:43:18.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:17 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-128"}]': finished 2024-06-11T13:43:18.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:17 smithi107 ceph-mon[33178]: osdmap e656: 8 total, 8 up, 8 in 2024-06-11T13:43:19.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:18 smithi057 ceph-mon[29805]: pgmap v1024: 300 pgs: 300 active+clean; 583 KiB data, 696 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 281 B/s wr, 2 op/s 2024-06-11T13:43:19.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:18 smithi057 ceph-mon[29805]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:43:19.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:18 smithi057 ceph-mon[29805]: osdmap e657: 8 total, 8 up, 8 in 2024-06-11T13:43:19.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:18 smithi057 ceph-mon[40588]: pgmap v1024: 300 pgs: 300 active+clean; 583 KiB data, 696 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 281 B/s wr, 2 op/s 2024-06-11T13:43:19.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:18 smithi057 ceph-mon[40588]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:43:19.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:18 smithi057 ceph-mon[40588]: osdmap e657: 8 total, 8 up, 8 in 2024-06-11T13:43:19.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:18 smithi107 ceph-mon[33178]: pgmap v1024: 300 pgs: 300 active+clean; 583 KiB data, 696 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 281 B/s wr, 2 op/s 2024-06-11T13:43:19.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:18 smithi107 ceph-mon[33178]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:43:19.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:18 smithi107 ceph-mon[33178]: osdmap e657: 8 total, 8 up, 8 in 2024-06-11T13:43:20.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:19 smithi057 ceph-mon[29805]: osdmap e658: 8 total, 8 up, 8 in 2024-06-11T13:43:20.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:19 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:43:20.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:19 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:43:20.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:19 smithi057 ceph-mon[40588]: osdmap e658: 8 total, 8 up, 8 in 2024-06-11T13:43:20.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:19 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:43:20.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:19 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:43:20.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:19 smithi107 ceph-mon[33178]: osdmap e658: 8 total, 8 up, 8 in 2024-06-11T13:43:20.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:19 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:43:20.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:19 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:43:21.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:20 smithi057 ceph-mon[29805]: pgmap v1027: 300 pgs: 32 unknown, 268 active+clean; 583 KiB data, 696 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T13:43:21.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:20 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:43:21.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:20 smithi057 ceph-mon[29805]: osdmap e659: 8 total, 8 up, 8 in 2024-06-11T13:43:21.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:20 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:43:21.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:20 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:43:21.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:20 smithi057 ceph-mon[40588]: pgmap v1027: 300 pgs: 32 unknown, 268 active+clean; 583 KiB data, 696 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T13:43:21.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:20 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:43:21.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:20 smithi057 ceph-mon[40588]: osdmap e659: 8 total, 8 up, 8 in 2024-06-11T13:43:21.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:20 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:43:21.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:20 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:43:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:20 smithi107 ceph-mon[33178]: pgmap v1027: 300 pgs: 32 unknown, 268 active+clean; 583 KiB data, 696 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T13:43:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:20 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:43:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:20 smithi107 ceph-mon[33178]: osdmap e659: 8 total, 8 up, 8 in 2024-06-11T13:43:21.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:20 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:43:21.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:20 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:43:22.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:21 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-130", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:43:22.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:21 smithi057 ceph-mon[29805]: osdmap e660: 8 total, 8 up, 8 in 2024-06-11T13:43:22.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:21 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-130"}]: dispatch 2024-06-11T13:43:22.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:21 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-130"}]: dispatch 2024-06-11T13:43:22.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:21 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-130", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:43:22.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:21 smithi057 ceph-mon[40588]: osdmap e660: 8 total, 8 up, 8 in 2024-06-11T13:43:22.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:21 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-130"}]: dispatch 2024-06-11T13:43:22.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:21 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-130"}]: dispatch 2024-06-11T13:43:22.048 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:43:21 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:43:22.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:21 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-130", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:43:22.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:21 smithi107 ceph-mon[33178]: osdmap e660: 8 total, 8 up, 8 in 2024-06-11T13:43:22.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:21 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-130"}]: dispatch 2024-06-11T13:43:22.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:21 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-130"}]: dispatch 2024-06-11T13:43:23.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:22 smithi057 ceph-mon[29805]: pgmap v1030: 300 pgs: 300 active+clean; 583 KiB data, 697 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T13:43:23.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:22 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-130"}]': finished 2024-06-11T13:43:23.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:22 smithi057 ceph-mon[29805]: osdmap e661: 8 total, 8 up, 8 in 2024-06-11T13:43:23.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:22 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-130", "mode": "writeback"}]: dispatch 2024-06-11T13:43:23.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:22 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-130", "mode": "writeback"}]: dispatch 2024-06-11T13:43:23.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:22 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:43:23.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:22 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:43:23.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:22 smithi057 ceph-mon[40588]: pgmap v1030: 300 pgs: 300 active+clean; 583 KiB data, 697 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T13:43:23.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:22 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-130"}]': finished 2024-06-11T13:43:23.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:22 smithi057 ceph-mon[40588]: osdmap e661: 8 total, 8 up, 8 in 2024-06-11T13:43:23.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:22 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-130", "mode": "writeback"}]: dispatch 2024-06-11T13:43:23.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:22 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-130", "mode": "writeback"}]: dispatch 2024-06-11T13:43:23.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:22 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:43:23.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:22 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:43:23.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:22 smithi107 ceph-mon[33178]: pgmap v1030: 300 pgs: 300 active+clean; 583 KiB data, 697 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T13:43:23.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:22 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-130"}]': finished 2024-06-11T13:43:23.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:22 smithi107 ceph-mon[33178]: osdmap e661: 8 total, 8 up, 8 in 2024-06-11T13:43:23.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:22 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-130", "mode": "writeback"}]: dispatch 2024-06-11T13:43:23.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:22 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-130", "mode": "writeback"}]: dispatch 2024-06-11T13:43:23.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:22 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:43:23.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:22 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:43:24.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:23 smithi057 ceph-mon[29805]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:43:24.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:23 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-130", "mode": "writeback"}]': finished 2024-06-11T13:43:24.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:23 smithi057 ceph-mon[29805]: osdmap e662: 8 total, 8 up, 8 in 2024-06-11T13:43:24.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:23 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:43:24.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:23 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:43:24.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:43:23 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:43:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:43:24.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:23 smithi057 ceph-mon[40588]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:43:24.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:23 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-130", "mode": "writeback"}]': finished 2024-06-11T13:43:24.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:23 smithi057 ceph-mon[40588]: osdmap e662: 8 total, 8 up, 8 in 2024-06-11T13:43:24.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:23 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:43:24.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:23 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:43:24.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:23 smithi107 ceph-mon[33178]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:43:24.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:23 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-130", "mode": "writeback"}]': finished 2024-06-11T13:43:24.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:23 smithi107 ceph-mon[33178]: osdmap e662: 8 total, 8 up, 8 in 2024-06-11T13:43:24.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:23 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:43:24.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:23 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:43:24.937 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:24 smithi057 ceph-mon[29805]: pgmap v1033: 300 pgs: 300 active+clean; 583 KiB data, 697 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:43:24.937 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:24 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:43:24.937 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:24 smithi057 ceph-mon[29805]: osdmap e663: 8 total, 8 up, 8 in 2024-06-11T13:43:24.938 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:24 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-130"}]: dispatch 2024-06-11T13:43:24.938 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:24 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-130"}]: dispatch 2024-06-11T13:43:24.938 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:24 smithi057 ceph-mon[40588]: pgmap v1033: 300 pgs: 300 active+clean; 583 KiB data, 697 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:43:24.938 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:24 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:43:24.939 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:24 smithi057 ceph-mon[40588]: osdmap e663: 8 total, 8 up, 8 in 2024-06-11T13:43:24.939 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:24 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-130"}]: dispatch 2024-06-11T13:43:24.939 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:24 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-130"}]: dispatch 2024-06-11T13:43:25.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:24 smithi107 ceph-mon[33178]: pgmap v1033: 300 pgs: 300 active+clean; 583 KiB data, 697 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:43:25.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:24 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:43:25.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:24 smithi107 ceph-mon[33178]: osdmap e663: 8 total, 8 up, 8 in 2024-06-11T13:43:25.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:24 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-130"}]: dispatch 2024-06-11T13:43:25.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:24 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-130"}]: dispatch 2024-06-11T13:43:26.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:25 smithi057 ceph-mon[29805]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:43:26.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:25 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-130"}]': finished 2024-06-11T13:43:26.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:25 smithi057 ceph-mon[29805]: osdmap e664: 8 total, 8 up, 8 in 2024-06-11T13:43:26.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:25 smithi057 ceph-mon[40588]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:43:26.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:25 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-130"}]': finished 2024-06-11T13:43:26.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:25 smithi057 ceph-mon[40588]: osdmap e664: 8 total, 8 up, 8 in 2024-06-11T13:43:26.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:25 smithi107 ceph-mon[33178]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:43:26.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:25 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-130"}]': finished 2024-06-11T13:43:26.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:25 smithi107 ceph-mon[33178]: osdmap e664: 8 total, 8 up, 8 in 2024-06-11T13:43:27.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:26 smithi057 ceph-mon[29805]: pgmap v1036: 300 pgs: 300 active+clean; 583 KiB data, 697 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:43:27.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:26 smithi057 ceph-mon[29805]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:43:27.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:26 smithi057 ceph-mon[29805]: osdmap e665: 8 total, 8 up, 8 in 2024-06-11T13:43:27.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:26 smithi057 ceph-mon[40588]: pgmap v1036: 300 pgs: 300 active+clean; 583 KiB data, 697 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:43:27.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:26 smithi057 ceph-mon[40588]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:43:27.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:26 smithi057 ceph-mon[40588]: osdmap e665: 8 total, 8 up, 8 in 2024-06-11T13:43:27.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:26 smithi107 ceph-mon[33178]: pgmap v1036: 300 pgs: 300 active+clean; 583 KiB data, 697 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:43:27.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:26 smithi107 ceph-mon[33178]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:43:27.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:26 smithi107 ceph-mon[33178]: osdmap e665: 8 total, 8 up, 8 in 2024-06-11T13:43:28.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:27 smithi057 ceph-mon[29805]: osdmap e666: 8 total, 8 up, 8 in 2024-06-11T13:43:28.054 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:27 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:43:28.054 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:27 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:43:28.054 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:27 smithi057 ceph-mon[40588]: osdmap e666: 8 total, 8 up, 8 in 2024-06-11T13:43:28.054 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:27 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:43:28.055 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:27 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:43:28.055 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:27 smithi107 ceph-mon[33178]: osdmap e666: 8 total, 8 up, 8 in 2024-06-11T13:43:28.055 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:27 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:43:28.055 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:27 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:43:29.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:28 smithi057 ceph-mon[29805]: pgmap v1039: 300 pgs: 32 unknown, 268 active+clean; 583 KiB data, 697 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:43:29.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:28 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:43:29.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:28 smithi057 ceph-mon[29805]: osdmap e667: 8 total, 8 up, 8 in 2024-06-11T13:43:29.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:28 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:43:29.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:28 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:43:29.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:28 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-132", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:43:29.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:28 smithi057 ceph-mon[29805]: osdmap e668: 8 total, 8 up, 8 in 2024-06-11T13:43:29.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:28 smithi057 ceph-mon[40588]: pgmap v1039: 300 pgs: 32 unknown, 268 active+clean; 583 KiB data, 697 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:43:29.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:28 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:43:29.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:28 smithi057 ceph-mon[40588]: osdmap e667: 8 total, 8 up, 8 in 2024-06-11T13:43:29.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:28 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:43:29.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:28 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:43:29.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:28 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-132", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:43:29.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:28 smithi057 ceph-mon[40588]: osdmap e668: 8 total, 8 up, 8 in 2024-06-11T13:43:29.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:28 smithi107 ceph-mon[33178]: pgmap v1039: 300 pgs: 32 unknown, 268 active+clean; 583 KiB data, 697 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:43:29.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:28 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:43:29.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:28 smithi107 ceph-mon[33178]: osdmap e667: 8 total, 8 up, 8 in 2024-06-11T13:43:29.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:28 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:43:29.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:28 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:43:29.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:28 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-132", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:43:29.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:28 smithi107 ceph-mon[33178]: osdmap e668: 8 total, 8 up, 8 in 2024-06-11T13:43:30.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:29 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-132"}]: dispatch 2024-06-11T13:43:30.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:29 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-132"}]: dispatch 2024-06-11T13:43:30.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:29 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-132"}]': finished 2024-06-11T13:43:30.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:29 smithi057 ceph-mon[29805]: osdmap e669: 8 total, 8 up, 8 in 2024-06-11T13:43:30.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:29 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-132"}]: dispatch 2024-06-11T13:43:30.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:29 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-132"}]: dispatch 2024-06-11T13:43:30.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:29 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-132"}]': finished 2024-06-11T13:43:30.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:29 smithi057 ceph-mon[40588]: osdmap e669: 8 total, 8 up, 8 in 2024-06-11T13:43:30.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:29 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-132"}]: dispatch 2024-06-11T13:43:30.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:29 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-132"}]: dispatch 2024-06-11T13:43:30.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:29 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-132"}]': finished 2024-06-11T13:43:30.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:29 smithi107 ceph-mon[33178]: osdmap e669: 8 total, 8 up, 8 in 2024-06-11T13:43:30.842 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OKc 2024-06-11T13:43:30.842 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.843 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.843 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.843 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.843 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.843 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.843 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.843 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.843 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.843 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.843 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.843 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.843 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.843 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.844 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.844 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.844 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.844 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.844 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.844 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.844 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.844 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.844 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.844 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.844 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.844 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.845 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.845 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.845 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.845 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.845 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.845 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.845 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.845 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.845 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.845 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.845 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.845 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.845 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.846 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.846 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.846 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.846 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.846 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.846 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.846 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.846 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.846 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.846 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.846 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.846 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.846 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.847 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.847 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.847 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.847 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.847 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.847 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.847 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.847 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.847 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.847 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.847 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.847 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.847 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.848 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.848 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.848 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.848 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.848 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.848 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.848 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.848 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.848 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.848 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.848 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.848 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.848 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.849 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.849 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.849 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.849 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.849 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.849 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.849 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.849 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.849 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.849 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.849 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.849 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.849 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.850 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.850 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.850 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.850 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.850 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.850 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.850 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:30.850 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:43:31.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:30 smithi057 ceph-mon[40588]: pgmap v1042: 300 pgs: 32 unknown, 268 active+clean; 583 KiB data, 697 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:43:31.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:30 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-132", "mode": "writeback"}]: dispatch 2024-06-11T13:43:31.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:30 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-132", "mode": "writeback"}]: dispatch 2024-06-11T13:43:31.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:30 smithi057 ceph-mon[40588]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:43:31.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:30 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-132", "mode": "writeback"}]': finished 2024-06-11T13:43:31.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:30 smithi057 ceph-mon[40588]: osdmap e670: 8 total, 8 up, 8 in 2024-06-11T13:43:31.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:30 smithi057 ceph-mon[29805]: pgmap v1042: 300 pgs: 32 unknown, 268 active+clean; 583 KiB data, 697 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:43:31.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:30 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-132", "mode": "writeback"}]: dispatch 2024-06-11T13:43:31.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:30 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-132", "mode": "writeback"}]: dispatch 2024-06-11T13:43:31.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:30 smithi057 ceph-mon[29805]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:43:31.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:30 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-132", "mode": "writeback"}]': finished 2024-06-11T13:43:31.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:30 smithi057 ceph-mon[29805]: osdmap e670: 8 total, 8 up, 8 in 2024-06-11T13:43:31.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:30 smithi107 ceph-mon[33178]: pgmap v1042: 300 pgs: 32 unknown, 268 active+clean; 583 KiB data, 697 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:43:31.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:30 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-132", "mode": "writeback"}]: dispatch 2024-06-11T13:43:31.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:30 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-132", "mode": "writeback"}]: dispatch 2024-06-11T13:43:31.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:30 smithi107 ceph-mon[33178]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:43:31.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:30 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-132", "mode": "writeback"}]': finished 2024-06-11T13:43:31.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:30 smithi107 ceph-mon[33178]: osdmap e670: 8 total, 8 up, 8 in 2024-06-11T13:43:32.048 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:43:31 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:43:33.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:32 smithi107 ceph-mon[33178]: pgmap v1045: 300 pgs: 300 active+clean; 583 KiB data, 698 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:43:33.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:32 smithi107 ceph-mon[33178]: osdmap e671: 8 total, 8 up, 8 in 2024-06-11T13:43:33.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:32 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:43:33.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:32 smithi107 ceph-mon[33178]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:43:33.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:32 smithi057 ceph-mon[40588]: pgmap v1045: 300 pgs: 300 active+clean; 583 KiB data, 698 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:43:33.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:32 smithi057 ceph-mon[40588]: osdmap e671: 8 total, 8 up, 8 in 2024-06-11T13:43:33.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:32 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:43:33.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:32 smithi057 ceph-mon[40588]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:43:33.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:32 smithi057 ceph-mon[29805]: pgmap v1045: 300 pgs: 300 active+clean; 583 KiB data, 698 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:43:33.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:32 smithi057 ceph-mon[29805]: osdmap e671: 8 total, 8 up, 8 in 2024-06-11T13:43:33.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:32 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:43:33.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:32 smithi057 ceph-mon[29805]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:43:34.035 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:43:33 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:43:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:43:34.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:33 smithi057 ceph-mon[40588]: osdmap e672: 8 total, 8 up, 8 in 2024-06-11T13:43:34.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:33 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:43:34.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:33 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:43:34.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:33 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:43:34.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:33 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:43:34.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:33 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:43:34.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:33 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:43:34.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:33 smithi057 ceph-mon[29805]: osdmap e672: 8 total, 8 up, 8 in 2024-06-11T13:43:34.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:33 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:43:34.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:33 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:43:34.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:33 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:43:34.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:33 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:43:34.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:33 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:43:34.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:33 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:43:34.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:33 smithi107 ceph-mon[33178]: osdmap e672: 8 total, 8 up, 8 in 2024-06-11T13:43:34.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:33 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:43:34.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:33 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:43:34.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:33 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:43:34.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:33 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:43:34.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:33 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:43:34.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:33 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:43:35.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:34 smithi057 ceph-mon[40588]: pgmap v1048: 300 pgs: 300 active+clean; 583 KiB data, 698 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:43:35.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:34 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:43:35.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:34 smithi057 ceph-mon[40588]: osdmap e673: 8 total, 8 up, 8 in 2024-06-11T13:43:35.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:34 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-132"}]: dispatch 2024-06-11T13:43:35.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:34 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-132"}]: dispatch 2024-06-11T13:43:35.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:34 smithi057 ceph-mon[29805]: pgmap v1048: 300 pgs: 300 active+clean; 583 KiB data, 698 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:43:35.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:34 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:43:35.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:34 smithi057 ceph-mon[29805]: osdmap e673: 8 total, 8 up, 8 in 2024-06-11T13:43:35.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:34 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-132"}]: dispatch 2024-06-11T13:43:35.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:34 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-132"}]: dispatch 2024-06-11T13:43:35.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:34 smithi107 ceph-mon[33178]: pgmap v1048: 300 pgs: 300 active+clean; 583 KiB data, 698 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:43:35.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:34 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:43:35.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:34 smithi107 ceph-mon[33178]: osdmap e673: 8 total, 8 up, 8 in 2024-06-11T13:43:35.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:34 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-132"}]: dispatch 2024-06-11T13:43:35.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:34 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-132"}]: dispatch 2024-06-11T13:43:36.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:35 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-132"}]': finished 2024-06-11T13:43:36.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:35 smithi057 ceph-mon[29805]: osdmap e674: 8 total, 8 up, 8 in 2024-06-11T13:43:36.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:35 smithi057 ceph-mon[29805]: osdmap e675: 8 total, 8 up, 8 in 2024-06-11T13:43:36.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:35 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-132"}]': finished 2024-06-11T13:43:36.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:35 smithi057 ceph-mon[40588]: osdmap e674: 8 total, 8 up, 8 in 2024-06-11T13:43:36.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:35 smithi057 ceph-mon[40588]: osdmap e675: 8 total, 8 up, 8 in 2024-06-11T13:43:36.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:35 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-132"}]': finished 2024-06-11T13:43:36.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:35 smithi107 ceph-mon[33178]: osdmap e674: 8 total, 8 up, 8 in 2024-06-11T13:43:36.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:35 smithi107 ceph-mon[33178]: osdmap e675: 8 total, 8 up, 8 in 2024-06-11T13:43:37.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:36 smithi057 ceph-mon[29805]: pgmap v1051: 300 pgs: 300 active+clean; 583 KiB data, 698 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:43:37.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:36 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:43:37.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:36 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:43:37.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:36 smithi057 ceph-mon[40588]: pgmap v1051: 300 pgs: 300 active+clean; 583 KiB data, 698 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:43:37.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:36 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:43:37.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:36 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:43:37.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:36 smithi107 ceph-mon[33178]: pgmap v1051: 300 pgs: 300 active+clean; 583 KiB data, 698 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:43:37.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:36 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:43:37.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:36 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:43:38.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:37 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:43:38.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:37 smithi057 ceph-mon[29805]: osdmap e676: 8 total, 8 up, 8 in 2024-06-11T13:43:38.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:37 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-132"}]: dispatch 2024-06-11T13:43:38.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:37 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-132"}]: dispatch 2024-06-11T13:43:38.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:37 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:43:38.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:37 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:43:38.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:37 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:43:38.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:37 smithi057 ceph-mon[40588]: osdmap e676: 8 total, 8 up, 8 in 2024-06-11T13:43:38.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:37 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-132"}]: dispatch 2024-06-11T13:43:38.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:37 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-132"}]: dispatch 2024-06-11T13:43:38.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:37 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:43:38.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:37 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:43:38.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:37 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:43:38.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:37 smithi107 ceph-mon[33178]: osdmap e676: 8 total, 8 up, 8 in 2024-06-11T13:43:38.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:37 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-132"}]: dispatch 2024-06-11T13:43:38.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:37 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-132"}]: dispatch 2024-06-11T13:43:38.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:37 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:43:38.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:37 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:43:39.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:38 smithi057 ceph-mon[29805]: pgmap v1054: 300 pgs: 300 active+clean; 583 KiB data, 699 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-06-11T13:43:39.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:38 smithi057 ceph-mon[29805]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:43:39.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:38 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-132"}]': finished 2024-06-11T13:43:39.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:38 smithi057 ceph-mon[29805]: osdmap e677: 8 total, 8 up, 8 in 2024-06-11T13:43:39.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:38 smithi057 ceph-mon[40588]: pgmap v1054: 300 pgs: 300 active+clean; 583 KiB data, 699 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-06-11T13:43:39.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:38 smithi057 ceph-mon[40588]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:43:39.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:38 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-132"}]': finished 2024-06-11T13:43:39.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:38 smithi057 ceph-mon[40588]: osdmap e677: 8 total, 8 up, 8 in 2024-06-11T13:43:39.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:38 smithi107 ceph-mon[33178]: pgmap v1054: 300 pgs: 300 active+clean; 583 KiB data, 699 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-06-11T13:43:39.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:38 smithi107 ceph-mon[33178]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:43:39.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:38 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-132"}]': finished 2024-06-11T13:43:39.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:38 smithi107 ceph-mon[33178]: osdmap e677: 8 total, 8 up, 8 in 2024-06-11T13:43:40.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:39 smithi057 ceph-mon[29805]: osdmap e678: 8 total, 8 up, 8 in 2024-06-11T13:43:40.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:39 smithi057 ceph-mon[40588]: osdmap e678: 8 total, 8 up, 8 in 2024-06-11T13:43:40.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:39 smithi107 ceph-mon[33178]: osdmap e678: 8 total, 8 up, 8 in 2024-06-11T13:43:41.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:40 smithi057 ceph-mon[29805]: pgmap v1057: 268 pgs: 268 active+clean; 583 KiB data, 699 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-06-11T13:43:41.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:40 smithi057 ceph-mon[29805]: osdmap e679: 8 total, 8 up, 8 in 2024-06-11T13:43:41.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:40 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:43:41.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:40 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:43:41.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:40 smithi057 ceph-mon[40588]: pgmap v1057: 268 pgs: 268 active+clean; 583 KiB data, 699 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-06-11T13:43:41.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:40 smithi057 ceph-mon[40588]: osdmap e679: 8 total, 8 up, 8 in 2024-06-11T13:43:41.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:40 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:43:41.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:40 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:43:41.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:40 smithi107 ceph-mon[33178]: pgmap v1057: 268 pgs: 268 active+clean; 583 KiB data, 699 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-06-11T13:43:41.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:40 smithi107 ceph-mon[33178]: osdmap e679: 8 total, 8 up, 8 in 2024-06-11T13:43:41.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:40 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:43:41.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:40 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:43:42.049 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:43:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:43:42.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:41 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:43:42.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:41 smithi107 ceph-mon[33178]: osdmap e680: 8 total, 8 up, 8 in 2024-06-11T13:43:42.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:41 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:43:42.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:41 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:43:42.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:41 smithi107 ceph-mon[33178]: pgmap v1060: 300 pgs: 300 active+clean; 583 KiB data, 717 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-06-11T13:43:42.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:41 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:43:42.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:41 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:43:42.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:41 smithi057 ceph-mon[29805]: osdmap e680: 8 total, 8 up, 8 in 2024-06-11T13:43:42.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:41 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:43:42.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:41 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:43:42.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:41 smithi057 ceph-mon[29805]: pgmap v1060: 300 pgs: 300 active+clean; 583 KiB data, 717 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-06-11T13:43:42.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:41 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:43:42.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:41 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:43:42.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:41 smithi057 ceph-mon[40588]: osdmap e680: 8 total, 8 up, 8 in 2024-06-11T13:43:42.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:41 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:43:42.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:41 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:43:42.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:41 smithi057 ceph-mon[40588]: pgmap v1060: 300 pgs: 300 active+clean; 583 KiB data, 717 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-06-11T13:43:42.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:41 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:43:43.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:42 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-134", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:43:43.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:42 smithi057 ceph-mon[29805]: osdmap e681: 8 total, 8 up, 8 in 2024-06-11T13:43:43.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:42 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-134"}]: dispatch 2024-06-11T13:43:43.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:42 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-134"}]: dispatch 2024-06-11T13:43:43.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:42 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-134"}]': finished 2024-06-11T13:43:43.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:42 smithi057 ceph-mon[29805]: osdmap e682: 8 total, 8 up, 8 in 2024-06-11T13:43:43.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:42 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-134", "mode": "writeback"}]: dispatch 2024-06-11T13:43:43.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:42 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-134", "mode": "writeback"}]: dispatch 2024-06-11T13:43:43.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:42 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-134", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:43:43.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:42 smithi057 ceph-mon[40588]: osdmap e681: 8 total, 8 up, 8 in 2024-06-11T13:43:43.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:42 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-134"}]: dispatch 2024-06-11T13:43:43.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:42 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-134"}]: dispatch 2024-06-11T13:43:43.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:42 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-134"}]': finished 2024-06-11T13:43:43.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:42 smithi057 ceph-mon[40588]: osdmap e682: 8 total, 8 up, 8 in 2024-06-11T13:43:43.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:42 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-134", "mode": "writeback"}]: dispatch 2024-06-11T13:43:43.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:42 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-134", "mode": "writeback"}]: dispatch 2024-06-11T13:43:43.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:42 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-134", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:43:43.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:42 smithi107 ceph-mon[33178]: osdmap e681: 8 total, 8 up, 8 in 2024-06-11T13:43:43.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:42 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-134"}]: dispatch 2024-06-11T13:43:43.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:42 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-134"}]: dispatch 2024-06-11T13:43:43.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:42 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-134"}]': finished 2024-06-11T13:43:43.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:42 smithi107 ceph-mon[33178]: osdmap e682: 8 total, 8 up, 8 in 2024-06-11T13:43:43.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:42 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-134", "mode": "writeback"}]: dispatch 2024-06-11T13:43:43.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:42 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-134", "mode": "writeback"}]: dispatch 2024-06-11T13:43:44.035 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:43:43 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:43:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:43:44.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:43 smithi057 ceph-mon[40588]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:43:44.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:43 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-134", "mode": "writeback"}]': finished 2024-06-11T13:43:44.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:43 smithi057 ceph-mon[40588]: osdmap e683: 8 total, 8 up, 8 in 2024-06-11T13:43:44.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:43 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:43:44.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:43 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:43:44.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:43 smithi057 ceph-mon[40588]: pgmap v1064: 300 pgs: 300 active+clean; 583 KiB data, 717 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-06-11T13:43:44.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:43 smithi057 ceph-mon[29805]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:43:44.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:43 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-134", "mode": "writeback"}]': finished 2024-06-11T13:43:44.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:43 smithi057 ceph-mon[29805]: osdmap e683: 8 total, 8 up, 8 in 2024-06-11T13:43:44.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:43 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:43:44.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:43 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:43:44.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:43 smithi057 ceph-mon[29805]: pgmap v1064: 300 pgs: 300 active+clean; 583 KiB data, 717 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-06-11T13:43:44.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:43 smithi107 ceph-mon[33178]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:43:44.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:43 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-134", "mode": "writeback"}]': finished 2024-06-11T13:43:44.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:43 smithi107 ceph-mon[33178]: osdmap e683: 8 total, 8 up, 8 in 2024-06-11T13:43:44.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:43 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:43:44.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:43 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:43:44.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:43 smithi107 ceph-mon[33178]: pgmap v1064: 300 pgs: 300 active+clean; 583 KiB data, 717 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-06-11T13:43:45.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:45 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:43:45.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:45 smithi057 ceph-mon[40588]: osdmap e684: 8 total, 8 up, 8 in 2024-06-11T13:43:45.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:45 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-134"}]: dispatch 2024-06-11T13:43:45.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:45 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-134"}]: dispatch 2024-06-11T13:43:45.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:45 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:43:45.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:45 smithi057 ceph-mon[29805]: osdmap e684: 8 total, 8 up, 8 in 2024-06-11T13:43:45.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:45 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-134"}]: dispatch 2024-06-11T13:43:45.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:45 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-134"}]: dispatch 2024-06-11T13:43:45.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:45 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:43:45.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:45 smithi107 ceph-mon[33178]: osdmap e684: 8 total, 8 up, 8 in 2024-06-11T13:43:45.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:45 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-134"}]: dispatch 2024-06-11T13:43:45.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:45 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-134"}]: dispatch 2024-06-11T13:43:46.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:46 smithi057 ceph-mon[29805]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:43:46.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:46 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-134"}]': finished 2024-06-11T13:43:46.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:46 smithi057 ceph-mon[29805]: osdmap e685: 8 total, 8 up, 8 in 2024-06-11T13:43:46.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:46 smithi057 ceph-mon[29805]: pgmap v1067: 300 pgs: 300 active+clean; 583 KiB data, 717 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:43:46.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:46 smithi057 ceph-mon[40588]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:43:46.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:46 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-134"}]': finished 2024-06-11T13:43:46.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:46 smithi057 ceph-mon[40588]: osdmap e685: 8 total, 8 up, 8 in 2024-06-11T13:43:46.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:46 smithi057 ceph-mon[40588]: pgmap v1067: 300 pgs: 300 active+clean; 583 KiB data, 717 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:43:46.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:46 smithi107 ceph-mon[33178]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:43:46.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:46 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-134"}]': finished 2024-06-11T13:43:46.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:46 smithi107 ceph-mon[33178]: osdmap e685: 8 total, 8 up, 8 in 2024-06-11T13:43:46.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:46 smithi107 ceph-mon[33178]: pgmap v1067: 300 pgs: 300 active+clean; 583 KiB data, 717 MiB used, 715 GiB / 715 GiB avail 2024-06-11T13:43:47.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:47 smithi057 ceph-mon[29805]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:43:47.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:47 smithi057 ceph-mon[29805]: osdmap e686: 8 total, 8 up, 8 in 2024-06-11T13:43:47.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:47 smithi057 ceph-mon[40588]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:43:47.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:47 smithi057 ceph-mon[40588]: osdmap e686: 8 total, 8 up, 8 in 2024-06-11T13:43:47.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:47 smithi107 ceph-mon[33178]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:43:47.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:47 smithi107 ceph-mon[33178]: osdmap e686: 8 total, 8 up, 8 in 2024-06-11T13:43:48.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:48 smithi057 ceph-mon[40588]: osdmap e687: 8 total, 8 up, 8 in 2024-06-11T13:43:48.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:48 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:43:48.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:48 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:43:48.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:48 smithi057 ceph-mon[40588]: pgmap v1070: 300 pgs: 32 unknown, 268 active+clean; 583 KiB data, 739 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-06-11T13:43:48.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:48 smithi057 ceph-mon[29805]: osdmap e687: 8 total, 8 up, 8 in 2024-06-11T13:43:48.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:48 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:43:48.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:48 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:43:48.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:48 smithi057 ceph-mon[29805]: pgmap v1070: 300 pgs: 32 unknown, 268 active+clean; 583 KiB data, 739 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-06-11T13:43:48.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:48 smithi107 ceph-mon[33178]: osdmap e687: 8 total, 8 up, 8 in 2024-06-11T13:43:48.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:48 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:43:48.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:48 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:43:48.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:48 smithi107 ceph-mon[33178]: pgmap v1070: 300 pgs: 32 unknown, 268 active+clean; 583 KiB data, 739 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-06-11T13:43:49.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:49 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:43:49.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:49 smithi057 ceph-mon[29805]: osdmap e688: 8 total, 8 up, 8 in 2024-06-11T13:43:49.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:49 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:43:49.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:49 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:43:49.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:49 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:43:49.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:49 smithi057 ceph-mon[40588]: osdmap e688: 8 total, 8 up, 8 in 2024-06-11T13:43:49.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:49 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:43:49.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:49 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:43:49.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:49 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:43:49.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:49 smithi107 ceph-mon[33178]: osdmap e688: 8 total, 8 up, 8 in 2024-06-11T13:43:49.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:49 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:43:49.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:49 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:43:50.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:50 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-136", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:43:50.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:50 smithi057 ceph-mon[40588]: osdmap e689: 8 total, 8 up, 8 in 2024-06-11T13:43:50.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:50 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-136"}]: dispatch 2024-06-11T13:43:50.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:50 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-136"}]: dispatch 2024-06-11T13:43:50.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:50 smithi057 ceph-mon[40588]: pgmap v1073: 300 pgs: 32 unknown, 268 active+clean; 583 KiB data, 739 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-06-11T13:43:50.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:50 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-136", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:43:50.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:50 smithi057 ceph-mon[29805]: osdmap e689: 8 total, 8 up, 8 in 2024-06-11T13:43:50.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:50 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-136"}]: dispatch 2024-06-11T13:43:50.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:50 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-136"}]: dispatch 2024-06-11T13:43:50.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:50 smithi057 ceph-mon[29805]: pgmap v1073: 300 pgs: 32 unknown, 268 active+clean; 583 KiB data, 739 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-06-11T13:43:50.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:50 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-136", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:43:50.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:50 smithi107 ceph-mon[33178]: osdmap e689: 8 total, 8 up, 8 in 2024-06-11T13:43:50.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:50 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-136"}]: dispatch 2024-06-11T13:43:50.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:50 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-136"}]: dispatch 2024-06-11T13:43:50.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:50 smithi107 ceph-mon[33178]: pgmap v1073: 300 pgs: 32 unknown, 268 active+clean; 583 KiB data, 739 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-06-11T13:43:51.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:51 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-136"}]': finished 2024-06-11T13:43:51.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:51 smithi057 ceph-mon[40588]: osdmap e690: 8 total, 8 up, 8 in 2024-06-11T13:43:51.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:51 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-136", "mode": "writeback"}]: dispatch 2024-06-11T13:43:51.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:51 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-136", "mode": "writeback"}]: dispatch 2024-06-11T13:43:51.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:51 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-136"}]': finished 2024-06-11T13:43:51.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:51 smithi057 ceph-mon[29805]: osdmap e690: 8 total, 8 up, 8 in 2024-06-11T13:43:51.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:51 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-136", "mode": "writeback"}]: dispatch 2024-06-11T13:43:51.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:51 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-136", "mode": "writeback"}]: dispatch 2024-06-11T13:43:51.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:51 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-136"}]': finished 2024-06-11T13:43:51.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:51 smithi107 ceph-mon[33178]: osdmap e690: 8 total, 8 up, 8 in 2024-06-11T13:43:51.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:51 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-136", "mode": "writeback"}]: dispatch 2024-06-11T13:43:51.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:51 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-136", "mode": "writeback"}]: dispatch 2024-06-11T13:43:52.250 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:43:51 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:43:52.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:52 smithi057 ceph-mon[40588]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:43:52.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:52 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-136", "mode": "writeback"}]': finished 2024-06-11T13:43:52.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:52 smithi057 ceph-mon[40588]: osdmap e691: 8 total, 8 up, 8 in 2024-06-11T13:43:52.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:52 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:43:52.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:52 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:43:52.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:52 smithi057 ceph-mon[40588]: pgmap v1076: 300 pgs: 300 active+clean; 583 KiB data, 740 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:43:52.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:52 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:43:52.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:52 smithi057 ceph-mon[40588]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:43:52.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:52 smithi057 ceph-mon[29805]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:43:52.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:52 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-136", "mode": "writeback"}]': finished 2024-06-11T13:43:52.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:52 smithi057 ceph-mon[29805]: osdmap e691: 8 total, 8 up, 8 in 2024-06-11T13:43:52.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:52 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:43:52.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:52 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:43:52.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:52 smithi057 ceph-mon[29805]: pgmap v1076: 300 pgs: 300 active+clean; 583 KiB data, 740 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:43:52.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:52 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:43:52.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:52 smithi057 ceph-mon[29805]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:43:52.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:52 smithi107 ceph-mon[33178]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:43:52.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:52 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-136", "mode": "writeback"}]': finished 2024-06-11T13:43:52.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:52 smithi107 ceph-mon[33178]: osdmap e691: 8 total, 8 up, 8 in 2024-06-11T13:43:52.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:52 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:43:52.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:52 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:43:52.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:52 smithi107 ceph-mon[33178]: pgmap v1076: 300 pgs: 300 active+clean; 583 KiB data, 740 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:43:52.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:52 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:43:52.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:52 smithi107 ceph-mon[33178]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:43:53.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:53 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:43:53.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:53 smithi057 ceph-mon[40588]: osdmap e692: 8 total, 8 up, 8 in 2024-06-11T13:43:53.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:53 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-136"}]: dispatch 2024-06-11T13:43:53.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:53 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-136"}]: dispatch 2024-06-11T13:43:53.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:53 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:43:53.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:53 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:43:53.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:53 smithi057 ceph-mon[29805]: osdmap e692: 8 total, 8 up, 8 in 2024-06-11T13:43:53.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:53 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-136"}]: dispatch 2024-06-11T13:43:53.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:53 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-136"}]: dispatch 2024-06-11T13:43:53.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:53 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:43:53.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:53 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:43:53.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:53 smithi107 ceph-mon[33178]: osdmap e692: 8 total, 8 up, 8 in 2024-06-11T13:43:53.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:53 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-136"}]: dispatch 2024-06-11T13:43:53.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:53 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-136"}]: dispatch 2024-06-11T13:43:53.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:53 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:43:54.035 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:43:53 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:43:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:43:54.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:54 smithi107 ceph-mon[33178]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:43:54.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:54 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-136"}]': finished 2024-06-11T13:43:54.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:54 smithi107 ceph-mon[33178]: osdmap e693: 8 total, 8 up, 8 in 2024-06-11T13:43:54.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:54 smithi107 ceph-mon[33178]: pgmap v1079: 300 pgs: 300 active+clean; 583 KiB data, 740 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:43:54.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:54 smithi057 ceph-mon[29805]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:43:54.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:54 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-136"}]': finished 2024-06-11T13:43:54.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:54 smithi057 ceph-mon[29805]: osdmap e693: 8 total, 8 up, 8 in 2024-06-11T13:43:54.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:54 smithi057 ceph-mon[29805]: pgmap v1079: 300 pgs: 300 active+clean; 583 KiB data, 740 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:43:54.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:54 smithi057 ceph-mon[40588]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:43:54.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:54 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-136"}]': finished 2024-06-11T13:43:54.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:54 smithi057 ceph-mon[40588]: osdmap e693: 8 total, 8 up, 8 in 2024-06-11T13:43:54.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:54 smithi057 ceph-mon[40588]: pgmap v1079: 300 pgs: 300 active+clean; 583 KiB data, 740 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:43:55.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:55 smithi057 ceph-mon[29805]: osdmap e694: 8 total, 8 up, 8 in 2024-06-11T13:43:55.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:55 smithi057 ceph-mon[40588]: osdmap e694: 8 total, 8 up, 8 in 2024-06-11T13:43:55.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:55 smithi107 ceph-mon[33178]: osdmap e694: 8 total, 8 up, 8 in 2024-06-11T13:43:56.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:56 smithi057 ceph-mon[29805]: osdmap e695: 8 total, 8 up, 8 in 2024-06-11T13:43:56.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:56 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:43:56.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:56 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:43:56.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:56 smithi057 ceph-mon[29805]: pgmap v1082: 300 pgs: 32 unknown, 268 active+clean; 583 KiB data, 740 MiB used, 714 GiB / 715 GiB avail 2024-06-11T13:43:56.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:56 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:43:56.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:56 smithi057 ceph-mon[29805]: osdmap e696: 8 total, 8 up, 8 in 2024-06-11T13:43:56.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:56 smithi057 ceph-mon[40588]: osdmap e695: 8 total, 8 up, 8 in 2024-06-11T13:43:56.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:56 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:43:56.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:56 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:43:56.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:56 smithi057 ceph-mon[40588]: pgmap v1082: 300 pgs: 32 unknown, 268 active+clean; 583 KiB data, 740 MiB used, 714 GiB / 715 GiB avail 2024-06-11T13:43:56.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:56 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:43:56.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:56 smithi057 ceph-mon[40588]: osdmap e696: 8 total, 8 up, 8 in 2024-06-11T13:43:56.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:56 smithi107 ceph-mon[33178]: osdmap e695: 8 total, 8 up, 8 in 2024-06-11T13:43:56.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:56 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:43:56.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:56 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:43:56.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:56 smithi107 ceph-mon[33178]: pgmap v1082: 300 pgs: 32 unknown, 268 active+clean; 583 KiB data, 740 MiB used, 714 GiB / 715 GiB avail 2024-06-11T13:43:56.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:56 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:43:56.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:56 smithi107 ceph-mon[33178]: osdmap e696: 8 total, 8 up, 8 in 2024-06-11T13:43:57.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:57 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:43:57.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:57 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:43:57.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:57 smithi057 ceph-mon[40588]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:43:57.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:57 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-138", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:43:57.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:57 smithi057 ceph-mon[40588]: osdmap e697: 8 total, 8 up, 8 in 2024-06-11T13:43:57.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:57 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-138","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:43:57.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:57 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-138","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:43:57.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:57 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:43:57.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:57 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:43:57.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:57 smithi057 ceph-mon[29805]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:43:57.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:57 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-138", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:43:57.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:57 smithi057 ceph-mon[29805]: osdmap e697: 8 total, 8 up, 8 in 2024-06-11T13:43:57.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:57 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-138","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:43:57.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:57 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-138","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:43:57.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:57 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:43:57.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:57 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:43:57.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:57 smithi107 ceph-mon[33178]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:43:57.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:57 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-138", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:43:57.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:57 smithi107 ceph-mon[33178]: osdmap e697: 8 total, 8 up, 8 in 2024-06-11T13:43:57.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:57 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-138","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:43:57.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:57 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-138","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:43:58.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:58 smithi057 ceph-mon[29805]: pgmap v1085: 300 pgs: 32 creating+peering, 268 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-06-11T13:43:58.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:58 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-138","var": "hit_set_count","val": "2"}]': finished 2024-06-11T13:43:58.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:58 smithi057 ceph-mon[29805]: osdmap e698: 8 total, 8 up, 8 in 2024-06-11T13:43:58.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:58 smithi057 ceph-mon[40588]: pgmap v1085: 300 pgs: 32 creating+peering, 268 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-06-11T13:43:58.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:58 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-138","var": "hit_set_count","val": "2"}]': finished 2024-06-11T13:43:58.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:58 smithi057 ceph-mon[40588]: osdmap e698: 8 total, 8 up, 8 in 2024-06-11T13:43:58.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:58 smithi107 ceph-mon[33178]: pgmap v1085: 300 pgs: 32 creating+peering, 268 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-06-11T13:43:58.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:58 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-138","var": "hit_set_count","val": "2"}]': finished 2024-06-11T13:43:58.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:58 smithi107 ceph-mon[33178]: osdmap e698: 8 total, 8 up, 8 in 2024-06-11T13:43:59.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:59 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-138","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:43:59.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:59 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-138","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:43:59.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:43:59 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-138","var": "hit_set_period","val": "600"}]': finished 2024-06-11T13:43:59.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:59 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-138","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:43:59.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:59 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-138","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:43:59.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:43:59 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-138","var": "hit_set_period","val": "600"}]': finished 2024-06-11T13:43:59.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:59 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-138","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:43:59.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:59 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-138","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:43:59.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:43:59 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-138","var": "hit_set_period","val": "600"}]': finished 2024-06-11T13:44:00.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:00 smithi057 ceph-mon[29805]: osdmap e699: 8 total, 8 up, 8 in 2024-06-11T13:44:00.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:00 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-11T13:44:00.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:00 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-11T13:44:00.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:00 smithi057 ceph-mon[29805]: pgmap v1088: 300 pgs: 32 creating+peering, 268 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-06-11T13:44:00.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:00 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-06-11T13:44:00.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:00 smithi057 ceph-mon[40588]: osdmap e699: 8 total, 8 up, 8 in 2024-06-11T13:44:00.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:00 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-11T13:44:00.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:00 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-11T13:44:00.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:00 smithi057 ceph-mon[40588]: pgmap v1088: 300 pgs: 32 creating+peering, 268 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-06-11T13:44:00.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:00 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-06-11T13:44:00.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:00 smithi107 ceph-mon[33178]: osdmap e699: 8 total, 8 up, 8 in 2024-06-11T13:44:00.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:00 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-11T13:44:00.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:00 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-06-11T13:44:00.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:00 smithi107 ceph-mon[33178]: pgmap v1088: 300 pgs: 32 creating+peering, 268 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-06-11T13:44:00.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:00 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-06-11T13:44:01.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:01 smithi057 ceph-mon[29805]: osdmap e700: 8 total, 8 up, 8 in 2024-06-11T13:44:01.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:01 smithi057 ceph-mon[40588]: osdmap e700: 8 total, 8 up, 8 in 2024-06-11T13:44:01.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:01 smithi107 ceph-mon[33178]: osdmap e700: 8 total, 8 up, 8 in 2024-06-11T13:44:02.298 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:44:01 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:44:02.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:02 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:44:02.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:02 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:44:02.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:02 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-138"}]: dispatch 2024-06-11T13:44:02.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:02 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-138"}]: dispatch 2024-06-11T13:44:02.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:02 smithi057 ceph-mon[40588]: pgmap v1090: 300 pgs: 300 active+clean; 4.4 MiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:44:02.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:02 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:44:02.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:02 smithi057 ceph-mon[40588]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:44:02.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:02 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:44:02.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:02 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:44:02.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:02 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-138"}]: dispatch 2024-06-11T13:44:02.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:02 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-138"}]: dispatch 2024-06-11T13:44:02.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:02 smithi057 ceph-mon[29805]: pgmap v1090: 300 pgs: 300 active+clean; 4.4 MiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:44:02.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:02 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:44:02.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:02 smithi057 ceph-mon[29805]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:44:02.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:02 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:44:02.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:02 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:44:02.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:02 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-138"}]: dispatch 2024-06-11T13:44:02.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:02 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-138"}]: dispatch 2024-06-11T13:44:02.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:02 smithi107 ceph-mon[33178]: pgmap v1090: 300 pgs: 300 active+clean; 4.4 MiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:44:02.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:02 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:44:02.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:02 smithi107 ceph-mon[33178]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:44:03.733 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:03 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-138"}]': finished 2024-06-11T13:44:03.733 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:03 smithi057 ceph-mon[40588]: osdmap e701: 8 total, 8 up, 8 in 2024-06-11T13:44:03.734 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:03 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-138"}]': finished 2024-06-11T13:44:03.734 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:03 smithi057 ceph-mon[29805]: osdmap e701: 8 total, 8 up, 8 in 2024-06-11T13:44:03.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:03 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-138"}]': finished 2024-06-11T13:44:03.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:03 smithi107 ceph-mon[33178]: osdmap e701: 8 total, 8 up, 8 in 2024-06-11T13:44:04.035 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:44:03 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:44:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:44:04.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:04 smithi057 ceph-mon[29805]: osdmap e702: 8 total, 8 up, 8 in 2024-06-11T13:44:04.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:04 smithi057 ceph-mon[29805]: pgmap v1093: 268 pgs: 268 active+clean; 4.4 MiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:44:04.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:04 smithi057 ceph-mon[40588]: osdmap e702: 8 total, 8 up, 8 in 2024-06-11T13:44:04.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:04 smithi057 ceph-mon[40588]: pgmap v1093: 268 pgs: 268 active+clean; 4.4 MiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:44:04.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:04 smithi107 ceph-mon[33178]: osdmap e702: 8 total, 8 up, 8 in 2024-06-11T13:44:04.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:04 smithi107 ceph-mon[33178]: pgmap v1093: 268 pgs: 268 active+clean; 4.4 MiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:44:05.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:05 smithi057 ceph-mon[29805]: osdmap e703: 8 total, 8 up, 8 in 2024-06-11T13:44:05.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:05 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:44:05.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:05 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:44:05.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:05 smithi057 ceph-mon[40588]: osdmap e703: 8 total, 8 up, 8 in 2024-06-11T13:44:05.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:05 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:44:05.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:05 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:44:05.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:05 smithi107 ceph-mon[33178]: osdmap e703: 8 total, 8 up, 8 in 2024-06-11T13:44:05.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:05 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:44:05.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:05 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:44:06.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:06 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:44:06.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:06 smithi057 ceph-mon[29805]: osdmap e704: 8 total, 8 up, 8 in 2024-06-11T13:44:06.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:06 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:44:06.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:06 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:44:06.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:06 smithi057 ceph-mon[29805]: pgmap v1096: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 786 MiB used, 714 GiB / 715 GiB avail 2024-06-11T13:44:06.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:06 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:44:06.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:06 smithi057 ceph-mon[40588]: osdmap e704: 8 total, 8 up, 8 in 2024-06-11T13:44:06.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:06 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:44:06.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:06 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:44:06.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:06 smithi057 ceph-mon[40588]: pgmap v1096: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 786 MiB used, 714 GiB / 715 GiB avail 2024-06-11T13:44:06.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:06 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:44:06.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:06 smithi107 ceph-mon[33178]: osdmap e704: 8 total, 8 up, 8 in 2024-06-11T13:44:06.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:06 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:44:06.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:06 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:44:06.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:06 smithi107 ceph-mon[33178]: pgmap v1096: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 786 MiB used, 714 GiB / 715 GiB avail 2024-06-11T13:44:07.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:07 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-140", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:44:07.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:07 smithi057 ceph-mon[29805]: osdmap e705: 8 total, 8 up, 8 in 2024-06-11T13:44:07.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:07 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-140","var": "hit_set_count","val": "3"}]: dispatch 2024-06-11T13:44:07.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:07 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-140","var": "hit_set_count","val": "3"}]: dispatch 2024-06-11T13:44:07.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:07 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:44:07.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:07 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:44:07.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:07 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-140", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:44:07.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:07 smithi057 ceph-mon[40588]: osdmap e705: 8 total, 8 up, 8 in 2024-06-11T13:44:07.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:07 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-140","var": "hit_set_count","val": "3"}]: dispatch 2024-06-11T13:44:07.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:07 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-140","var": "hit_set_count","val": "3"}]: dispatch 2024-06-11T13:44:07.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:07 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:44:07.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:07 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:44:07.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:07 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-140", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:44:07.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:07 smithi107 ceph-mon[33178]: osdmap e705: 8 total, 8 up, 8 in 2024-06-11T13:44:07.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:07 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-140","var": "hit_set_count","val": "3"}]: dispatch 2024-06-11T13:44:07.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:07 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-140","var": "hit_set_count","val": "3"}]: dispatch 2024-06-11T13:44:07.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:07 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:44:07.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:07 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:44:08.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:08 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-140","var": "hit_set_count","val": "3"}]': finished 2024-06-11T13:44:08.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:08 smithi057 ceph-mon[40588]: osdmap e706: 8 total, 8 up, 8 in 2024-06-11T13:44:08.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:08 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-140","var": "hit_set_period","val": "3"}]: dispatch 2024-06-11T13:44:08.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:08 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-140","var": "hit_set_period","val": "3"}]: dispatch 2024-06-11T13:44:08.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:08 smithi057 ceph-mon[40588]: pgmap v1099: 300 pgs: 300 active+clean; 4.4 MiB data, 787 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:44:08.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:08 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-140","var": "hit_set_period","val": "3"}]': finished 2024-06-11T13:44:08.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:08 smithi057 ceph-mon[40588]: osdmap e707: 8 total, 8 up, 8 in 2024-06-11T13:44:08.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:08 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-140","var": "hit_set_count","val": "3"}]': finished 2024-06-11T13:44:08.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:08 smithi057 ceph-mon[29805]: osdmap e706: 8 total, 8 up, 8 in 2024-06-11T13:44:08.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:08 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-140","var": "hit_set_period","val": "3"}]: dispatch 2024-06-11T13:44:08.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:08 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-140","var": "hit_set_period","val": "3"}]: dispatch 2024-06-11T13:44:08.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:08 smithi057 ceph-mon[29805]: pgmap v1099: 300 pgs: 300 active+clean; 4.4 MiB data, 787 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:44:08.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:08 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-140","var": "hit_set_period","val": "3"}]': finished 2024-06-11T13:44:08.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:08 smithi057 ceph-mon[29805]: osdmap e707: 8 total, 8 up, 8 in 2024-06-11T13:44:08.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:08 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-140","var": "hit_set_count","val": "3"}]': finished 2024-06-11T13:44:08.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:08 smithi107 ceph-mon[33178]: osdmap e706: 8 total, 8 up, 8 in 2024-06-11T13:44:08.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:08 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-140","var": "hit_set_period","val": "3"}]: dispatch 2024-06-11T13:44:08.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:08 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-140","var": "hit_set_period","val": "3"}]: dispatch 2024-06-11T13:44:08.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:08 smithi107 ceph-mon[33178]: pgmap v1099: 300 pgs: 300 active+clean; 4.4 MiB data, 787 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:44:08.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:08 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-140","var": "hit_set_period","val": "3"}]': finished 2024-06-11T13:44:08.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:08 smithi107 ceph-mon[33178]: osdmap e707: 8 total, 8 up, 8 in 2024-06-11T13:44:09.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:09 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-11T13:44:09.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:09 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-11T13:44:09.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:09 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-140","var": "hit_set_type","val": "bloom"}]': finished 2024-06-11T13:44:09.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:09 smithi057 ceph-mon[29805]: osdmap e708: 8 total, 8 up, 8 in 2024-06-11T13:44:09.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:09 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-06-11T13:44:09.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:09 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-06-11T13:44:09.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:09 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-11T13:44:09.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:09 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-11T13:44:09.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:09 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-140","var": "hit_set_type","val": "bloom"}]': finished 2024-06-11T13:44:09.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:09 smithi057 ceph-mon[40588]: osdmap e708: 8 total, 8 up, 8 in 2024-06-11T13:44:09.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:09 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-06-11T13:44:09.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:09 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-06-11T13:44:09.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:09 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-11T13:44:09.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:09 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-11T13:44:09.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:09 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-140","var": "hit_set_type","val": "bloom"}]': finished 2024-06-11T13:44:09.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:09 smithi107 ceph-mon[33178]: osdmap e708: 8 total, 8 up, 8 in 2024-06-11T13:44:09.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:09 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-06-11T13:44:09.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:09 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-06-11T13:44:10.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:10 smithi057 ceph-mon[29805]: pgmap v1102: 300 pgs: 300 active+clean; 4.4 MiB data, 787 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:44:10.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:10 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-06-11T13:44:10.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:10 smithi057 ceph-mon[29805]: osdmap e709: 8 total, 8 up, 8 in 2024-06-11T13:44:10.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:10 smithi057 ceph-mon[40588]: pgmap v1102: 300 pgs: 300 active+clean; 4.4 MiB data, 787 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:44:10.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:10 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-06-11T13:44:10.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:10 smithi057 ceph-mon[40588]: osdmap e709: 8 total, 8 up, 8 in 2024-06-11T13:44:10.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:10 smithi107 ceph-mon[33178]: pgmap v1102: 300 pgs: 300 active+clean; 4.4 MiB data, 787 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:44:10.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:10 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-06-11T13:44:10.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:10 smithi107 ceph-mon[33178]: osdmap e709: 8 total, 8 up, 8 in 2024-06-11T13:44:12.298 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:44:11 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:44:12.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:12 smithi107 ceph-mon[33178]: pgmap v1104: 300 pgs: 300 active+clean; 4.4 MiB data, 787 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-06-11T13:44:12.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:12 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:44:13.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:12 smithi057 ceph-mon[40588]: pgmap v1104: 300 pgs: 300 active+clean; 4.4 MiB data, 787 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-06-11T13:44:13.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:12 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:44:13.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:12 smithi057 ceph-mon[29805]: pgmap v1104: 300 pgs: 300 active+clean; 4.4 MiB data, 787 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-06-11T13:44:13.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:12 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:44:14.035 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:44:13 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:44:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:44:14.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:14 smithi107 ceph-mon[33178]: pgmap v1105: 300 pgs: 300 active+clean; 4.4 MiB data, 787 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 1.3 KiB/s wr, 0 op/s 2024-06-11T13:44:15.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:14 smithi057 ceph-mon[40588]: pgmap v1105: 300 pgs: 300 active+clean; 4.4 MiB data, 787 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 1.3 KiB/s wr, 0 op/s 2024-06-11T13:44:15.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:14 smithi057 ceph-mon[29805]: pgmap v1105: 300 pgs: 300 active+clean; 4.4 MiB data, 787 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 1.3 KiB/s wr, 0 op/s 2024-06-11T13:44:17.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:16 smithi057 ceph-mon[40588]: pgmap v1106: 300 pgs: 300 active+clean; 4.4 MiB data, 787 MiB used, 714 GiB / 715 GiB avail; 728 B/s rd, 1.1 KiB/s wr, 0 op/s 2024-06-11T13:44:17.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:16 smithi057 ceph-mon[29805]: pgmap v1106: 300 pgs: 300 active+clean; 4.4 MiB data, 787 MiB used, 714 GiB / 715 GiB avail; 728 B/s rd, 1.1 KiB/s wr, 0 op/s 2024-06-11T13:44:17.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:16 smithi107 ceph-mon[33178]: pgmap v1106: 300 pgs: 300 active+clean; 4.4 MiB data, 787 MiB used, 714 GiB / 715 GiB avail; 728 B/s rd, 1.1 KiB/s wr, 0 op/s 2024-06-11T13:44:19.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:18 smithi057 ceph-mon[29805]: pgmap v1107: 300 pgs: 300 active+clean; 4.4 MiB data, 787 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.1 KiB/s wr, 2 op/s 2024-06-11T13:44:19.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:18 smithi057 ceph-mon[40588]: pgmap v1107: 300 pgs: 300 active+clean; 4.4 MiB data, 787 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.1 KiB/s wr, 2 op/s 2024-06-11T13:44:19.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:18 smithi107 ceph-mon[33178]: pgmap v1107: 300 pgs: 300 active+clean; 4.4 MiB data, 787 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.1 KiB/s wr, 2 op/s 2024-06-11T13:44:21.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:20 smithi057 ceph-mon[29805]: pgmap v1108: 300 pgs: 300 active+clean; 4.4 MiB data, 787 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 4.9 KiB/s wr, 1 op/s 2024-06-11T13:44:21.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:20 smithi057 ceph-mon[40588]: pgmap v1108: 300 pgs: 300 active+clean; 4.4 MiB data, 787 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 4.9 KiB/s wr, 1 op/s 2024-06-11T13:44:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:20 smithi107 ceph-mon[33178]: pgmap v1108: 300 pgs: 300 active+clean; 4.4 MiB data, 787 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 4.9 KiB/s wr, 1 op/s 2024-06-11T13:44:22.298 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:44:21 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:44:23.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:22 smithi057 ceph-mon[40588]: pgmap v1109: 300 pgs: 300 active+clean; 4.5 MiB data, 787 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 8.2 KiB/s wr, 2 op/s 2024-06-11T13:44:23.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:22 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:44:23.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:22 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:44:23.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:22 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:44:23.039 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:22 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:44:23.039 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:22 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-140"}]: dispatch 2024-06-11T13:44:23.039 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:22 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-140"}]: dispatch 2024-06-11T13:44:23.039 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:22 smithi057 ceph-mon[29805]: pgmap v1109: 300 pgs: 300 active+clean; 4.5 MiB data, 787 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 8.2 KiB/s wr, 2 op/s 2024-06-11T13:44:23.039 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:22 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:44:23.039 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:22 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:44:23.039 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:22 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:44:23.039 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:22 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:44:23.039 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:22 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-140"}]: dispatch 2024-06-11T13:44:23.039 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:22 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-140"}]: dispatch 2024-06-11T13:44:23.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:22 smithi107 ceph-mon[33178]: pgmap v1109: 300 pgs: 300 active+clean; 4.5 MiB data, 787 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 8.2 KiB/s wr, 2 op/s 2024-06-11T13:44:23.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:22 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:44:23.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:22 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:44:23.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:22 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:44:23.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:22 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:44:23.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:22 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-140"}]: dispatch 2024-06-11T13:44:23.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:22 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-140"}]: dispatch 2024-06-11T13:44:24.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:23 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-140"}]': finished 2024-06-11T13:44:24.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:23 smithi057 ceph-mon[40588]: osdmap e710: 8 total, 8 up, 8 in 2024-06-11T13:44:24.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:23 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-140"}]': finished 2024-06-11T13:44:24.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:23 smithi057 ceph-mon[29805]: osdmap e710: 8 total, 8 up, 8 in 2024-06-11T13:44:24.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:44:23 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:44:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:44:24.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:23 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-140"}]': finished 2024-06-11T13:44:24.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:23 smithi107 ceph-mon[33178]: osdmap e710: 8 total, 8 up, 8 in 2024-06-11T13:44:24.936 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:24 smithi057 ceph-mon[40588]: pgmap v1111: 300 pgs: 300 active+clean; 4.5 MiB data, 787 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 8.3 KiB/s wr, 2 op/s 2024-06-11T13:44:24.937 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:24 smithi057 ceph-mon[40588]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:44:24.937 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:24 smithi057 ceph-mon[40588]: osdmap e711: 8 total, 8 up, 8 in 2024-06-11T13:44:24.937 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:24 smithi057 ceph-mon[29805]: pgmap v1111: 300 pgs: 300 active+clean; 4.5 MiB data, 787 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 8.3 KiB/s wr, 2 op/s 2024-06-11T13:44:24.937 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:24 smithi057 ceph-mon[29805]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:44:24.937 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:24 smithi057 ceph-mon[29805]: osdmap e711: 8 total, 8 up, 8 in 2024-06-11T13:44:25.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:24 smithi107 ceph-mon[33178]: pgmap v1111: 300 pgs: 300 active+clean; 4.5 MiB data, 787 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 8.3 KiB/s wr, 2 op/s 2024-06-11T13:44:25.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:24 smithi107 ceph-mon[33178]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:44:25.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:24 smithi107 ceph-mon[33178]: osdmap e711: 8 total, 8 up, 8 in 2024-06-11T13:44:26.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:25 smithi057 ceph-mon[29805]: osdmap e712: 8 total, 8 up, 8 in 2024-06-11T13:44:26.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:25 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:44:26.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:25 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:44:26.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:25 smithi057 ceph-mon[40588]: osdmap e712: 8 total, 8 up, 8 in 2024-06-11T13:44:26.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:25 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:44:26.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:25 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:44:26.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:25 smithi107 ceph-mon[33178]: osdmap e712: 8 total, 8 up, 8 in 2024-06-11T13:44:26.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:25 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:44:26.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:25 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:44:27.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:26 smithi057 ceph-mon[29805]: pgmap v1114: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 787 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:44:27.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:26 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:44:27.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:26 smithi057 ceph-mon[29805]: osdmap e713: 8 total, 8 up, 8 in 2024-06-11T13:44:27.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:26 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:44:27.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:26 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:44:27.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:26 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-142", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:44:27.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:26 smithi057 ceph-mon[29805]: osdmap e714: 8 total, 8 up, 8 in 2024-06-11T13:44:27.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:26 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-142"}]: dispatch 2024-06-11T13:44:27.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:26 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-142"}]: dispatch 2024-06-11T13:44:27.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:26 smithi057 ceph-mon[40588]: pgmap v1114: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 787 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:44:27.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:26 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:44:27.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:26 smithi057 ceph-mon[40588]: osdmap e713: 8 total, 8 up, 8 in 2024-06-11T13:44:27.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:26 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:44:27.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:26 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:44:27.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:26 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-142", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:44:27.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:26 smithi057 ceph-mon[40588]: osdmap e714: 8 total, 8 up, 8 in 2024-06-11T13:44:27.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:26 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-142"}]: dispatch 2024-06-11T13:44:27.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:26 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-142"}]: dispatch 2024-06-11T13:44:27.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:26 smithi107 ceph-mon[33178]: pgmap v1114: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 787 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:44:27.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:26 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:44:27.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:26 smithi107 ceph-mon[33178]: osdmap e713: 8 total, 8 up, 8 in 2024-06-11T13:44:27.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:26 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:44:27.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:26 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:44:27.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:26 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-142", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:44:27.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:26 smithi107 ceph-mon[33178]: osdmap e714: 8 total, 8 up, 8 in 2024-06-11T13:44:27.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:26 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-142"}]: dispatch 2024-06-11T13:44:27.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:26 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-142"}]: dispatch 2024-06-11T13:44:29.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:28 smithi057 ceph-mon[29805]: pgmap v1117: 300 pgs: 300 active+clean; 4.4 MiB data, 788 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-06-11T13:44:29.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:28 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-142"}]': finished 2024-06-11T13:44:29.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:28 smithi057 ceph-mon[29805]: osdmap e715: 8 total, 8 up, 8 in 2024-06-11T13:44:29.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:28 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-142", "mode": "writeback"}]: dispatch 2024-06-11T13:44:29.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:28 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-142", "mode": "writeback"}]: dispatch 2024-06-11T13:44:29.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:28 smithi057 ceph-mon[40588]: pgmap v1117: 300 pgs: 300 active+clean; 4.4 MiB data, 788 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-06-11T13:44:29.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:28 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-142"}]': finished 2024-06-11T13:44:29.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:28 smithi057 ceph-mon[40588]: osdmap e715: 8 total, 8 up, 8 in 2024-06-11T13:44:29.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:28 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-142", "mode": "writeback"}]: dispatch 2024-06-11T13:44:29.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:28 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-142", "mode": "writeback"}]: dispatch 2024-06-11T13:44:29.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:28 smithi107 ceph-mon[33178]: pgmap v1117: 300 pgs: 300 active+clean; 4.4 MiB data, 788 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-06-11T13:44:29.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:28 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-142"}]': finished 2024-06-11T13:44:29.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:28 smithi107 ceph-mon[33178]: osdmap e715: 8 total, 8 up, 8 in 2024-06-11T13:44:29.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:28 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-142", "mode": "writeback"}]: dispatch 2024-06-11T13:44:29.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:28 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-142", "mode": "writeback"}]: dispatch 2024-06-11T13:44:30.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:29 smithi057 ceph-mon[40588]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:44:30.178 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:29 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-142", "mode": "writeback"}]': finished 2024-06-11T13:44:30.178 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:29 smithi057 ceph-mon[40588]: osdmap e716: 8 total, 8 up, 8 in 2024-06-11T13:44:30.178 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:29 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:44:30.178 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:29 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:44:30.178 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:29 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "hit_set_count","val": "2"}]': finished 2024-06-11T13:44:30.178 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:29 smithi057 ceph-mon[40588]: osdmap e717: 8 total, 8 up, 8 in 2024-06-11T13:44:30.179 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:29 smithi057 ceph-mon[29805]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:44:30.179 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:29 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-142", "mode": "writeback"}]': finished 2024-06-11T13:44:30.179 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:29 smithi057 ceph-mon[29805]: osdmap e716: 8 total, 8 up, 8 in 2024-06-11T13:44:30.179 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:29 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:44:30.179 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:29 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:44:30.179 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:29 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "hit_set_count","val": "2"}]': finished 2024-06-11T13:44:30.179 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:29 smithi057 ceph-mon[29805]: osdmap e717: 8 total, 8 up, 8 in 2024-06-11T13:44:30.179 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:29 smithi107 ceph-mon[33178]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:44:30.180 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:29 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-142", "mode": "writeback"}]': finished 2024-06-11T13:44:30.180 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:29 smithi107 ceph-mon[33178]: osdmap e716: 8 total, 8 up, 8 in 2024-06-11T13:44:30.180 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:29 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:44:30.180 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:29 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:44:30.180 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:29 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "hit_set_count","val": "2"}]': finished 2024-06-11T13:44:30.180 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:29 smithi107 ceph-mon[33178]: osdmap e717: 8 total, 8 up, 8 in 2024-06-11T13:44:31.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:30 smithi057 ceph-mon[40588]: pgmap v1120: 300 pgs: 300 active+clean; 4.4 MiB data, 788 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-06-11T13:44:31.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:30 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:44:31.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:30 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:44:31.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:30 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "hit_set_period","val": "600"}]': finished 2024-06-11T13:44:31.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:30 smithi057 ceph-mon[40588]: osdmap e718: 8 total, 8 up, 8 in 2024-06-11T13:44:31.038 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:30 smithi057 ceph-mon[29805]: pgmap v1120: 300 pgs: 300 active+clean; 4.4 MiB data, 788 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-06-11T13:44:31.038 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:30 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:44:31.038 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:30 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:44:31.038 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:30 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "hit_set_period","val": "600"}]': finished 2024-06-11T13:44:31.039 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:30 smithi057 ceph-mon[29805]: osdmap e718: 8 total, 8 up, 8 in 2024-06-11T13:44:31.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:30 smithi107 ceph-mon[33178]: pgmap v1120: 300 pgs: 300 active+clean; 4.4 MiB data, 788 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-06-11T13:44:31.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:30 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:44:31.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:30 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:44:31.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:30 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "hit_set_period","val": "600"}]': finished 2024-06-11T13:44:31.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:30 smithi107 ceph-mon[33178]: osdmap e718: 8 total, 8 up, 8 in 2024-06-11T13:44:32.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:31 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-11T13:44:32.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:31 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-11T13:44:32.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:31 smithi057 ceph-mon[40588]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:44:32.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:31 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "hit_set_type","val": "bloom"}]': finished 2024-06-11T13:44:32.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:31 smithi057 ceph-mon[40588]: osdmap e719: 8 total, 8 up, 8 in 2024-06-11T13:44:32.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:31 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-11T13:44:32.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:31 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-11T13:44:32.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:31 smithi057 ceph-mon[29805]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:44:32.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:31 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "hit_set_type","val": "bloom"}]': finished 2024-06-11T13:44:32.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:31 smithi057 ceph-mon[29805]: osdmap e719: 8 total, 8 up, 8 in 2024-06-11T13:44:32.048 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:44:31 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:44:32.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:31 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-11T13:44:32.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:31 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-11T13:44:32.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:31 smithi107 ceph-mon[33178]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:44:32.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:31 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "hit_set_type","val": "bloom"}]': finished 2024-06-11T13:44:32.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:31 smithi107 ceph-mon[33178]: osdmap e719: 8 total, 8 up, 8 in 2024-06-11T13:44:32.950 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:32 smithi107 ceph-mon[33178]: pgmap v1123: 300 pgs: 300 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:44:32.951 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:32 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-11T13:44:32.951 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:32 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-11T13:44:32.951 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:32 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:44:32.951 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:32 smithi107 ceph-mon[33178]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:44:32.951 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:32 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-06-11T13:44:32.951 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:32 smithi107 ceph-mon[33178]: osdmap e720: 8 total, 8 up, 8 in 2024-06-11T13:44:32.951 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:32 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-06-11T13:44:32.951 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:32 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-06-11T13:44:33.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:32 smithi057 ceph-mon[40588]: pgmap v1123: 300 pgs: 300 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:44:33.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:32 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-11T13:44:33.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:32 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-11T13:44:33.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:32 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:44:33.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:32 smithi057 ceph-mon[40588]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:44:33.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:32 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-06-11T13:44:33.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:32 smithi057 ceph-mon[40588]: osdmap e720: 8 total, 8 up, 8 in 2024-06-11T13:44:33.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:32 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-06-11T13:44:33.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:32 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-06-11T13:44:33.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:32 smithi057 ceph-mon[29805]: pgmap v1123: 300 pgs: 300 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:44:33.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:32 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-11T13:44:33.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:32 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-11T13:44:33.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:32 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:44:33.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:32 smithi057 ceph-mon[29805]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:44:33.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:32 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-06-11T13:44:33.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:32 smithi057 ceph-mon[29805]: osdmap e720: 8 total, 8 up, 8 in 2024-06-11T13:44:33.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:32 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-06-11T13:44:33.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:32 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-06-11T13:44:34.035 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:44:33 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:44:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:44:35.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:34 smithi057 ceph-mon[29805]: pgmap v1126: 300 pgs: 300 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:44:35.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:34 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-06-11T13:44:35.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:34 smithi057 ceph-mon[29805]: osdmap e721: 8 total, 8 up, 8 in 2024-06-11T13:44:35.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:34 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-06-11T13:44:35.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:34 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-06-11T13:44:35.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:34 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:44:35.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:34 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:44:35.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:34 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:44:35.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:34 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:44:35.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:34 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:44:35.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:34 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:44:35.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:34 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:44:35.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:34 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:44:35.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:34 smithi057 ceph-mon[40588]: pgmap v1126: 300 pgs: 300 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:44:35.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:34 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-06-11T13:44:35.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:34 smithi057 ceph-mon[40588]: osdmap e721: 8 total, 8 up, 8 in 2024-06-11T13:44:35.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:34 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-06-11T13:44:35.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:34 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-06-11T13:44:35.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:34 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:44:35.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:34 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:44:35.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:34 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:44:35.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:34 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:44:35.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:34 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:44:35.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:34 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:44:35.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:34 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:44:35.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:34 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:44:35.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:34 smithi107 ceph-mon[33178]: pgmap v1126: 300 pgs: 300 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:44:35.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:34 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-06-11T13:44:35.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:34 smithi107 ceph-mon[33178]: osdmap e721: 8 total, 8 up, 8 in 2024-06-11T13:44:35.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:34 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-06-11T13:44:35.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:34 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-06-11T13:44:35.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:34 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:44:35.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:34 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:44:35.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:34 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:44:35.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:34 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:44:35.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:34 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:44:35.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:34 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:44:35.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:34 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:44:35.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:34 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:44:36.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:35 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-06-11T13:44:36.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:35 smithi057 ceph-mon[40588]: osdmap e722: 8 total, 8 up, 8 in 2024-06-11T13:44:36.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:35 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:44:36.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:35 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:44:36.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:35 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-06-11T13:44:36.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:35 smithi057 ceph-mon[29805]: osdmap e722: 8 total, 8 up, 8 in 2024-06-11T13:44:36.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:35 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:44:36.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:35 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:44:36.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:35 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-06-11T13:44:36.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:35 smithi107 ceph-mon[33178]: osdmap e722: 8 total, 8 up, 8 in 2024-06-11T13:44:36.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:35 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:44:36.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:35 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:44:37.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:36 smithi057 ceph-mon[29805]: pgmap v1129: 300 pgs: 300 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail 2024-06-11T13:44:37.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:36 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:44:37.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:36 smithi057 ceph-mon[29805]: osdmap e723: 8 total, 8 up, 8 in 2024-06-11T13:44:37.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:36 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-142"}]: dispatch 2024-06-11T13:44:37.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:36 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-142"}]: dispatch 2024-06-11T13:44:37.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:36 smithi057 ceph-mon[40588]: pgmap v1129: 300 pgs: 300 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail 2024-06-11T13:44:37.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:36 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:44:37.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:36 smithi057 ceph-mon[40588]: osdmap e723: 8 total, 8 up, 8 in 2024-06-11T13:44:37.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:36 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-142"}]: dispatch 2024-06-11T13:44:37.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:36 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-142"}]: dispatch 2024-06-11T13:44:37.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:36 smithi107 ceph-mon[33178]: pgmap v1129: 300 pgs: 300 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail 2024-06-11T13:44:37.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:36 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:44:37.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:36 smithi107 ceph-mon[33178]: osdmap e723: 8 total, 8 up, 8 in 2024-06-11T13:44:37.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:36 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-142"}]: dispatch 2024-06-11T13:44:37.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:36 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-142"}]: dispatch 2024-06-11T13:44:38.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:37 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-142"}]': finished 2024-06-11T13:44:38.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:37 smithi057 ceph-mon[40588]: osdmap e724: 8 total, 8 up, 8 in 2024-06-11T13:44:38.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:37 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:44:38.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:37 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:44:38.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:37 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-142"}]: dispatch 2024-06-11T13:44:38.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:37 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-142"}]: dispatch 2024-06-11T13:44:38.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:37 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:44:38.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:37 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-142"}]': finished 2024-06-11T13:44:38.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:37 smithi057 ceph-mon[29805]: osdmap e724: 8 total, 8 up, 8 in 2024-06-11T13:44:38.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:37 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:44:38.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:37 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:44:38.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:37 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-142"}]: dispatch 2024-06-11T13:44:38.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:37 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-142"}]: dispatch 2024-06-11T13:44:38.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:37 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:44:38.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:37 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-142"}]': finished 2024-06-11T13:44:38.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:37 smithi107 ceph-mon[33178]: osdmap e724: 8 total, 8 up, 8 in 2024-06-11T13:44:38.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:37 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:44:38.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:37 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:44:38.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:37 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-142"}]: dispatch 2024-06-11T13:44:38.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:37 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-142"}]: dispatch 2024-06-11T13:44:38.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:37 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:44:39.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:38 smithi057 ceph-mon[40588]: pgmap v1132: 300 pgs: 300 active+clean; 4.4 MiB data, 807 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-06-11T13:44:39.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:38 smithi057 ceph-mon[40588]: osdmap e725: 8 total, 8 up, 8 in 2024-06-11T13:44:39.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:38 smithi057 ceph-mon[40588]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:44:39.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:38 smithi057 ceph-mon[29805]: pgmap v1132: 300 pgs: 300 active+clean; 4.4 MiB data, 807 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-06-11T13:44:39.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:38 smithi057 ceph-mon[29805]: osdmap e725: 8 total, 8 up, 8 in 2024-06-11T13:44:39.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:38 smithi057 ceph-mon[29805]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:44:39.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:38 smithi107 ceph-mon[33178]: pgmap v1132: 300 pgs: 300 active+clean; 4.4 MiB data, 807 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-06-11T13:44:39.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:38 smithi107 ceph-mon[33178]: osdmap e725: 8 total, 8 up, 8 in 2024-06-11T13:44:39.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:38 smithi107 ceph-mon[33178]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:44:40.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:39 smithi107 ceph-mon[33178]: osdmap e726: 8 total, 8 up, 8 in 2024-06-11T13:44:40.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:39 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:44:40.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:39 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:44:40.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:39 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:44:40.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:39 smithi107 ceph-mon[33178]: osdmap e727: 8 total, 8 up, 8 in 2024-06-11T13:44:40.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:39 smithi057 ceph-mon[40588]: osdmap e726: 8 total, 8 up, 8 in 2024-06-11T13:44:40.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:39 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:44:40.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:39 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:44:40.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:39 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:44:40.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:39 smithi057 ceph-mon[40588]: osdmap e727: 8 total, 8 up, 8 in 2024-06-11T13:44:40.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:39 smithi057 ceph-mon[29805]: osdmap e726: 8 total, 8 up, 8 in 2024-06-11T13:44:40.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:39 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:44:40.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:39 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:44:40.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:39 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:44:40.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:39 smithi057 ceph-mon[29805]: osdmap e727: 8 total, 8 up, 8 in 2024-06-11T13:44:41.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:40 smithi107 ceph-mon[33178]: pgmap v1135: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 807 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-06-11T13:44:41.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:40 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:44:41.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:40 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:44:41.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:40 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-144", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:44:41.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:40 smithi107 ceph-mon[33178]: osdmap e728: 8 total, 8 up, 8 in 2024-06-11T13:44:41.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:40 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-144"}]: dispatch 2024-06-11T13:44:41.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:40 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-144"}]: dispatch 2024-06-11T13:44:41.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:40 smithi057 ceph-mon[40588]: pgmap v1135: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 807 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-06-11T13:44:41.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:40 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:44:41.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:40 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:44:41.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:40 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-144", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:44:41.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:40 smithi057 ceph-mon[40588]: osdmap e728: 8 total, 8 up, 8 in 2024-06-11T13:44:41.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:40 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-144"}]: dispatch 2024-06-11T13:44:41.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:40 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-144"}]: dispatch 2024-06-11T13:44:41.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:40 smithi057 ceph-mon[29805]: pgmap v1135: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 807 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-06-11T13:44:41.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:40 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:44:41.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:40 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:44:41.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:40 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-144", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:44:41.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:40 smithi057 ceph-mon[29805]: osdmap e728: 8 total, 8 up, 8 in 2024-06-11T13:44:41.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:40 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-144"}]: dispatch 2024-06-11T13:44:41.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:40 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-144"}]: dispatch 2024-06-11T13:44:42.298 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:44:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:44:42.299 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:44:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=sqlstore.transactions t=2024-06-11T13:44:42.077129283Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-06-11T13:44:43.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:42 smithi057 ceph-mon[40588]: pgmap v1138: 300 pgs: 300 active+clean; 4.4 MiB data, 829 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:44:43.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:42 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-144"}]': finished 2024-06-11T13:44:43.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:42 smithi057 ceph-mon[40588]: osdmap e729: 8 total, 8 up, 8 in 2024-06-11T13:44:43.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:42 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-144", "mode": "readproxy"}]: dispatch 2024-06-11T13:44:43.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:42 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-144", "mode": "readproxy"}]: dispatch 2024-06-11T13:44:43.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:42 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:44:43.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:42 smithi057 ceph-mon[29805]: pgmap v1138: 300 pgs: 300 active+clean; 4.4 MiB data, 829 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:44:43.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:42 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-144"}]': finished 2024-06-11T13:44:43.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:42 smithi057 ceph-mon[29805]: osdmap e729: 8 total, 8 up, 8 in 2024-06-11T13:44:43.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:42 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-144", "mode": "readproxy"}]: dispatch 2024-06-11T13:44:43.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:42 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-144", "mode": "readproxy"}]: dispatch 2024-06-11T13:44:43.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:42 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:44:43.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:42 smithi107 ceph-mon[33178]: pgmap v1138: 300 pgs: 300 active+clean; 4.4 MiB data, 829 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:44:43.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:42 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-144"}]': finished 2024-06-11T13:44:43.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:42 smithi107 ceph-mon[33178]: osdmap e729: 8 total, 8 up, 8 in 2024-06-11T13:44:43.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:42 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-144", "mode": "readproxy"}]: dispatch 2024-06-11T13:44:43.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:42 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-144", "mode": "readproxy"}]: dispatch 2024-06-11T13:44:43.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:42 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:44:44.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:43 smithi057 ceph-mon[29805]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:44:44.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:43 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-144", "mode": "readproxy"}]': finished 2024-06-11T13:44:44.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:43 smithi057 ceph-mon[29805]: osdmap e730: 8 total, 8 up, 8 in 2024-06-11T13:44:44.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:43 smithi057 ceph-mon[40588]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:44:44.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:43 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-144", "mode": "readproxy"}]': finished 2024-06-11T13:44:44.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:43 smithi057 ceph-mon[40588]: osdmap e730: 8 total, 8 up, 8 in 2024-06-11T13:44:44.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:44:43 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:44:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:44:44.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:43 smithi107 ceph-mon[33178]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:44:44.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:43 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-144", "mode": "readproxy"}]': finished 2024-06-11T13:44:44.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:43 smithi107 ceph-mon[33178]: osdmap e730: 8 total, 8 up, 8 in 2024-06-11T13:44:45.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:44 smithi057 ceph-mon[29805]: pgmap v1141: 300 pgs: 300 active+clean; 4.4 MiB data, 829 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:44:45.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:44 smithi057 ceph-mon[40588]: pgmap v1141: 300 pgs: 300 active+clean; 4.4 MiB data, 829 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:44:45.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:44 smithi107 ceph-mon[33178]: pgmap v1141: 300 pgs: 300 active+clean; 4.4 MiB data, 829 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-06-11T13:44:47.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:46 smithi057 ceph-mon[29805]: pgmap v1142: 300 pgs: 300 active+clean; 4.4 MiB data, 829 MiB used, 714 GiB / 715 GiB avail; 893 B/s rd, 178 B/s wr, 1 op/s 2024-06-11T13:44:47.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:46 smithi057 ceph-mon[40588]: pgmap v1142: 300 pgs: 300 active+clean; 4.4 MiB data, 829 MiB used, 714 GiB / 715 GiB avail; 893 B/s rd, 178 B/s wr, 1 op/s 2024-06-11T13:44:47.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:46 smithi107 ceph-mon[33178]: pgmap v1142: 300 pgs: 300 active+clean; 4.4 MiB data, 829 MiB used, 714 GiB / 715 GiB avail; 893 B/s rd, 178 B/s wr, 1 op/s 2024-06-11T13:44:48.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:47 smithi057 ceph-mon[29805]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:44:48.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:47 smithi057 ceph-mon[40588]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:44:48.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:47 smithi107 ceph-mon[33178]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:44:49.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:48 smithi057 ceph-mon[29805]: pgmap v1143: 300 pgs: 300 active+clean; 4.4 MiB data, 830 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 152 B/s wr, 1 op/s 2024-06-11T13:44:49.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:48 smithi057 ceph-mon[40588]: pgmap v1143: 300 pgs: 300 active+clean; 4.4 MiB data, 830 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 152 B/s wr, 1 op/s 2024-06-11T13:44:49.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:48 smithi107 ceph-mon[33178]: pgmap v1143: 300 pgs: 300 active+clean; 4.4 MiB data, 830 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 152 B/s wr, 1 op/s 2024-06-11T13:44:51.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:50 smithi057 ceph-mon[29805]: pgmap v1144: 300 pgs: 300 active+clean; 4.4 MiB data, 830 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-11T13:44:51.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:50 smithi057 ceph-mon[40588]: pgmap v1144: 300 pgs: 300 active+clean; 4.4 MiB data, 830 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-11T13:44:51.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:50 smithi107 ceph-mon[33178]: pgmap v1144: 300 pgs: 300 active+clean; 4.4 MiB data, 830 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-11T13:44:52.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:51 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "339.c", "id": [2, 6]}]: dispatch 2024-06-11T13:44:52.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:51 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "339.c", "id": [2, 6]}]: dispatch 2024-06-11T13:44:52.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:51 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "339.c", "id": [2, 6]}]: dispatch 2024-06-11T13:44:52.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:51 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "339.c", "id": [2, 6]}]: dispatch 2024-06-11T13:44:52.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:51 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "339.c", "id": [2, 6]}]: dispatch 2024-06-11T13:44:52.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:51 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "339.c", "id": [2, 6]}]: dispatch 2024-06-11T13:44:52.299 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:44:51 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:44:53.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:52 smithi057 ceph-mon[29805]: pgmap v1145: 300 pgs: 300 active+clean; 4.4 MiB data, 830 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-06-11T13:44:53.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:52 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:44:53.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:52 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "339.c", "id": [2, 6]}]': finished 2024-06-11T13:44:53.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:52 smithi057 ceph-mon[29805]: osdmap e731: 8 total, 8 up, 8 in 2024-06-11T13:44:53.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:52 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:44:53.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:52 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:44:53.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:52 smithi057 ceph-mon[40588]: pgmap v1145: 300 pgs: 300 active+clean; 4.4 MiB data, 830 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-06-11T13:44:53.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:52 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:44:53.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:52 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "339.c", "id": [2, 6]}]': finished 2024-06-11T13:44:53.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:52 smithi057 ceph-mon[40588]: osdmap e731: 8 total, 8 up, 8 in 2024-06-11T13:44:53.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:52 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:44:53.288 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:52 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:44:53.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:52 smithi107 ceph-mon[33178]: pgmap v1145: 300 pgs: 300 active+clean; 4.4 MiB data, 830 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-06-11T13:44:53.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:52 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:44:53.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:52 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "339.c", "id": [2, 6]}]': finished 2024-06-11T13:44:53.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:52 smithi107 ceph-mon[33178]: osdmap e731: 8 total, 8 up, 8 in 2024-06-11T13:44:53.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:52 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:44:53.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:52 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:44:54.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:53 smithi057 ceph-mon[40588]: osdmap e732: 8 total, 8 up, 8 in 2024-06-11T13:44:54.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:53 smithi057 ceph-mon[40588]: pgmap v1148: 300 pgs: 300 active+clean; 4.4 MiB data, 830 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-06-11T13:44:54.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:53 smithi057 ceph-mon[29805]: osdmap e732: 8 total, 8 up, 8 in 2024-06-11T13:44:54.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:53 smithi057 ceph-mon[29805]: pgmap v1148: 300 pgs: 300 active+clean; 4.4 MiB data, 830 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-06-11T13:44:54.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:44:53 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:44:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:44:54.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:53 smithi107 ceph-mon[33178]: osdmap e732: 8 total, 8 up, 8 in 2024-06-11T13:44:54.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:53 smithi107 ceph-mon[33178]: pgmap v1148: 300 pgs: 300 active+clean; 4.4 MiB data, 830 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-06-11T13:44:55.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:54 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:44:55.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:54 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:44:55.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:54 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:44:55.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:54 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:44:55.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:54 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:44:55.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:54 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:44:56.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:55 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:44:56.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:55 smithi057 ceph-mon[29805]: osdmap e733: 8 total, 8 up, 8 in 2024-06-11T13:44:56.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:55 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-144"}]: dispatch 2024-06-11T13:44:56.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:55 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-144"}]: dispatch 2024-06-11T13:44:56.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:55 smithi057 ceph-mon[29805]: pgmap v1150: 300 pgs: 300 active+clean; 4.4 MiB data, 830 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:44:56.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:55 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:44:56.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:55 smithi057 ceph-mon[40588]: osdmap e733: 8 total, 8 up, 8 in 2024-06-11T13:44:56.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:55 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-144"}]: dispatch 2024-06-11T13:44:56.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:55 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-144"}]: dispatch 2024-06-11T13:44:56.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:55 smithi057 ceph-mon[40588]: pgmap v1150: 300 pgs: 300 active+clean; 4.4 MiB data, 830 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:44:56.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:55 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:44:56.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:55 smithi107 ceph-mon[33178]: osdmap e733: 8 total, 8 up, 8 in 2024-06-11T13:44:56.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:55 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-144"}]: dispatch 2024-06-11T13:44:56.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:55 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-144"}]: dispatch 2024-06-11T13:44:56.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:55 smithi107 ceph-mon[33178]: pgmap v1150: 300 pgs: 300 active+clean; 4.4 MiB data, 830 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:44:57.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:56 smithi057 ceph-mon[40588]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:44:57.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:56 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-144"}]': finished 2024-06-11T13:44:57.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:56 smithi057 ceph-mon[40588]: osdmap e734: 8 total, 8 up, 8 in 2024-06-11T13:44:57.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:56 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:44:57.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:56 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:44:57.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:56 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-144"}]: dispatch 2024-06-11T13:44:57.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:56 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-144"}]: dispatch 2024-06-11T13:44:57.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:56 smithi057 ceph-mon[29805]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:44:57.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:56 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-144"}]': finished 2024-06-11T13:44:57.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:56 smithi057 ceph-mon[29805]: osdmap e734: 8 total, 8 up, 8 in 2024-06-11T13:44:57.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:56 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:44:57.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:56 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:44:57.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:56 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-144"}]: dispatch 2024-06-11T13:44:57.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:56 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-144"}]: dispatch 2024-06-11T13:44:57.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:56 smithi107 ceph-mon[33178]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:44:57.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:56 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-144"}]': finished 2024-06-11T13:44:57.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:56 smithi107 ceph-mon[33178]: osdmap e734: 8 total, 8 up, 8 in 2024-06-11T13:44:57.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:56 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:44:57.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:56 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:44:57.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:56 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-144"}]: dispatch 2024-06-11T13:44:57.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:56 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-144"}]: dispatch 2024-06-11T13:44:58.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:57 smithi057 ceph-mon[29805]: osdmap e735: 8 total, 8 up, 8 in 2024-06-11T13:44:58.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:57 smithi057 ceph-mon[29805]: pgmap v1153: 268 pgs: 268 active+clean; 4.4 MiB data, 830 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-06-11T13:44:58.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:57 smithi057 ceph-mon[40588]: osdmap e735: 8 total, 8 up, 8 in 2024-06-11T13:44:58.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:57 smithi057 ceph-mon[40588]: pgmap v1153: 268 pgs: 268 active+clean; 4.4 MiB data, 830 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-06-11T13:44:58.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:57 smithi107 ceph-mon[33178]: osdmap e735: 8 total, 8 up, 8 in 2024-06-11T13:44:58.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:57 smithi107 ceph-mon[33178]: pgmap v1153: 268 pgs: 268 active+clean; 4.4 MiB data, 830 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-06-11T13:44:59.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:58 smithi057 ceph-mon[40588]: osdmap e736: 8 total, 8 up, 8 in 2024-06-11T13:44:59.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:58 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:44:59.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:58 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:44:59.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:58 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:44:59.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:58 smithi057 ceph-mon[40588]: osdmap e737: 8 total, 8 up, 8 in 2024-06-11T13:44:59.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:58 smithi057 ceph-mon[29805]: osdmap e736: 8 total, 8 up, 8 in 2024-06-11T13:44:59.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:58 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:44:59.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:58 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:44:59.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:58 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:44:59.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:58 smithi057 ceph-mon[29805]: osdmap e737: 8 total, 8 up, 8 in 2024-06-11T13:44:59.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:58 smithi107 ceph-mon[33178]: osdmap e736: 8 total, 8 up, 8 in 2024-06-11T13:44:59.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:58 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:44:59.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:58 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:44:59.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:58 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:44:59.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:58 smithi107 ceph-mon[33178]: osdmap e737: 8 total, 8 up, 8 in 2024-06-11T13:45:00.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:59 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:45:00.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:59 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:45:00.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:44:59 smithi057 ceph-mon[29805]: pgmap v1156: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 830 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:45:00.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:59 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:45:00.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:59 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:45:00.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:44:59 smithi057 ceph-mon[40588]: pgmap v1156: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 830 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:45:00.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:59 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:45:00.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:59 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:45:00.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:44:59 smithi107 ceph-mon[33178]: pgmap v1156: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 830 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:45:01.288 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:01 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-146", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:45:01.288 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:01 smithi057 ceph-mon[29805]: osdmap e738: 8 total, 8 up, 8 in 2024-06-11T13:45:01.291 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:01 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-146"}]: dispatch 2024-06-11T13:45:01.292 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:01 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-146"}]: dispatch 2024-06-11T13:45:01.293 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:01 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-146", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:45:01.293 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:01 smithi057 ceph-mon[40588]: osdmap e738: 8 total, 8 up, 8 in 2024-06-11T13:45:01.293 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:01 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-146"}]: dispatch 2024-06-11T13:45:01.294 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:01 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-146"}]: dispatch 2024-06-11T13:45:01.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:01 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-146", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:45:01.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:01 smithi107 ceph-mon[33178]: osdmap e738: 8 total, 8 up, 8 in 2024-06-11T13:45:01.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:01 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-146"}]: dispatch 2024-06-11T13:45:01.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:01 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-146"}]: dispatch 2024-06-11T13:45:02.201 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:02 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-146"}]': finished 2024-06-11T13:45:02.201 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:02 smithi107 ceph-mon[33178]: osdmap e739: 8 total, 8 up, 8 in 2024-06-11T13:45:02.201 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:02 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-146", "mode": "writeback"}]: dispatch 2024-06-11T13:45:02.201 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:02 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-146", "mode": "writeback"}]: dispatch 2024-06-11T13:45:02.201 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:02 smithi107 ceph-mon[33178]: pgmap v1159: 300 pgs: 300 active+clean; 4.4 MiB data, 852 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-06-11T13:45:02.202 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:02 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:45:02.202 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:45:01 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:45:02.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:02 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-146"}]': finished 2024-06-11T13:45:02.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:02 smithi057 ceph-mon[40588]: osdmap e739: 8 total, 8 up, 8 in 2024-06-11T13:45:02.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:02 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-146", "mode": "writeback"}]: dispatch 2024-06-11T13:45:02.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:02 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-146", "mode": "writeback"}]: dispatch 2024-06-11T13:45:02.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:02 smithi057 ceph-mon[40588]: pgmap v1159: 300 pgs: 300 active+clean; 4.4 MiB data, 852 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-06-11T13:45:02.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:02 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:45:02.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:02 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi057-77599-111", "overlaypool": "test-rados-api-smithi057-77599-146"}]': finished 2024-06-11T13:45:02.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:02 smithi057 ceph-mon[29805]: osdmap e739: 8 total, 8 up, 8 in 2024-06-11T13:45:02.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:02 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-146", "mode": "writeback"}]: dispatch 2024-06-11T13:45:02.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:02 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-146", "mode": "writeback"}]: dispatch 2024-06-11T13:45:02.288 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:02 smithi057 ceph-mon[29805]: pgmap v1159: 300 pgs: 300 active+clean; 4.4 MiB data, 852 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-06-11T13:45:02.288 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:02 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:45:03.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:03 smithi057 ceph-mon[40588]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:45:03.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:03 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-146", "mode": "writeback"}]': finished 2024-06-11T13:45:03.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:03 smithi057 ceph-mon[40588]: osdmap e740: 8 total, 8 up, 8 in 2024-06-11T13:45:03.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:03 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-146","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:45:03.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:03 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-146","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:45:03.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:03 smithi057 ceph-mon[29805]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:45:03.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:03 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-146", "mode": "writeback"}]': finished 2024-06-11T13:45:03.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:03 smithi057 ceph-mon[29805]: osdmap e740: 8 total, 8 up, 8 in 2024-06-11T13:45:03.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:03 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-146","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:45:03.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:03 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-146","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:45:03.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:03 smithi107 ceph-mon[33178]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-06-11T13:45:03.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:03 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi057-77599-146", "mode": "writeback"}]': finished 2024-06-11T13:45:03.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:03 smithi107 ceph-mon[33178]: osdmap e740: 8 total, 8 up, 8 in 2024-06-11T13:45:03.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:03 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-146","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:45:03.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:03 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-146","var": "hit_set_count","val": "2"}]: dispatch 2024-06-11T13:45:04.035 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:45:03 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:45:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:45:04.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:04 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-146","var": "hit_set_count","val": "2"}]': finished 2024-06-11T13:45:04.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:04 smithi107 ceph-mon[33178]: osdmap e741: 8 total, 8 up, 8 in 2024-06-11T13:45:04.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:04 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-146","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:45:04.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:04 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-146","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:45:04.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:04 smithi107 ceph-mon[33178]: pgmap v1162: 300 pgs: 300 active+clean; 4.4 MiB data, 852 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-06-11T13:45:04.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:04 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-146","var": "hit_set_period","val": "600"}]': finished 2024-06-11T13:45:04.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:04 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-146","var": "hit_set_count","val": "2"}]': finished 2024-06-11T13:45:04.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:04 smithi057 ceph-mon[29805]: osdmap e741: 8 total, 8 up, 8 in 2024-06-11T13:45:04.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:04 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-146","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:45:04.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:04 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-146","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:45:04.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:04 smithi057 ceph-mon[29805]: pgmap v1162: 300 pgs: 300 active+clean; 4.4 MiB data, 852 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-06-11T13:45:04.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:04 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-146","var": "hit_set_period","val": "600"}]': finished 2024-06-11T13:45:04.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:04 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-146","var": "hit_set_count","val": "2"}]': finished 2024-06-11T13:45:04.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:04 smithi057 ceph-mon[40588]: osdmap e741: 8 total, 8 up, 8 in 2024-06-11T13:45:04.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:04 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-146","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:45:04.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:04 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-146","var": "hit_set_period","val": "600"}]: dispatch 2024-06-11T13:45:04.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:04 smithi057 ceph-mon[40588]: pgmap v1162: 300 pgs: 300 active+clean; 4.4 MiB data, 852 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-06-11T13:45:04.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:04 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-146","var": "hit_set_period","val": "600"}]': finished 2024-06-11T13:45:05.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:05 smithi107 ceph-mon[33178]: osdmap e742: 8 total, 8 up, 8 in 2024-06-11T13:45:05.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:05 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-11T13:45:05.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:05 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-11T13:45:05.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:05 smithi107 ceph-mon[33178]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:45:05.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:05 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-146","var": "hit_set_type","val": "bloom"}]': finished 2024-06-11T13:45:05.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:05 smithi107 ceph-mon[33178]: osdmap e743: 8 total, 8 up, 8 in 2024-06-11T13:45:05.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:05 smithi057 ceph-mon[29805]: osdmap e742: 8 total, 8 up, 8 in 2024-06-11T13:45:05.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:05 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-11T13:45:05.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:05 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-11T13:45:05.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:05 smithi057 ceph-mon[29805]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:45:05.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:05 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-146","var": "hit_set_type","val": "bloom"}]': finished 2024-06-11T13:45:05.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:05 smithi057 ceph-mon[29805]: osdmap e743: 8 total, 8 up, 8 in 2024-06-11T13:45:05.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:05 smithi057 ceph-mon[40588]: osdmap e742: 8 total, 8 up, 8 in 2024-06-11T13:45:05.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:05 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-11T13:45:05.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:05 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-06-11T13:45:05.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:05 smithi057 ceph-mon[40588]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-06-11T13:45:05.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:05 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-146","var": "hit_set_type","val": "bloom"}]': finished 2024-06-11T13:45:05.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:05 smithi057 ceph-mon[40588]: osdmap e743: 8 total, 8 up, 8 in 2024-06-11T13:45:06.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:06 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-11T13:45:06.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:06 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-11T13:45:06.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:06 smithi057 ceph-mon[29805]: pgmap v1165: 300 pgs: 300 active+clean; 4.4 MiB data, 852 MiB used, 714 GiB / 715 GiB avail 2024-06-11T13:45:06.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:06 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-06-11T13:45:06.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:06 smithi057 ceph-mon[29805]: osdmap e744: 8 total, 8 up, 8 in 2024-06-11T13:45:06.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:06 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-11T13:45:06.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:06 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-11T13:45:06.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:06 smithi057 ceph-mon[40588]: pgmap v1165: 300 pgs: 300 active+clean; 4.4 MiB data, 852 MiB used, 714 GiB / 715 GiB avail 2024-06-11T13:45:06.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:06 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-06-11T13:45:06.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:06 smithi057 ceph-mon[40588]: osdmap e744: 8 total, 8 up, 8 in 2024-06-11T13:45:06.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:06 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-11T13:45:06.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:06 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-06-11T13:45:06.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:06 smithi107 ceph-mon[33178]: pgmap v1165: 300 pgs: 300 active+clean; 4.4 MiB data, 852 MiB used, 714 GiB / 715 GiB avail 2024-06-11T13:45:06.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:06 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-06-11T13:45:06.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:06 smithi107 ceph-mon[33178]: osdmap e744: 8 total, 8 up, 8 in 2024-06-11T13:45:07.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:07 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-146","var": "target_max_objects","val": "1"}]: dispatch 2024-06-11T13:45:07.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:07 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-146","var": "target_max_objects","val": "1"}]: dispatch 2024-06-11T13:45:07.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:07 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-146","var": "target_max_objects","val": "1"}]': finished 2024-06-11T13:45:07.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:07 smithi057 ceph-mon[29805]: osdmap e745: 8 total, 8 up, 8 in 2024-06-11T13:45:07.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:07 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-146","var": "target_max_objects","val": "1"}]: dispatch 2024-06-11T13:45:07.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:07 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-146","var": "target_max_objects","val": "1"}]: dispatch 2024-06-11T13:45:07.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:07 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-146","var": "target_max_objects","val": "1"}]': finished 2024-06-11T13:45:07.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:07 smithi057 ceph-mon[40588]: osdmap e745: 8 total, 8 up, 8 in 2024-06-11T13:45:07.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:07 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-146","var": "target_max_objects","val": "1"}]: dispatch 2024-06-11T13:45:07.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:07 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-146","var": "target_max_objects","val": "1"}]: dispatch 2024-06-11T13:45:07.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:07 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-146","var": "target_max_objects","val": "1"}]': finished 2024-06-11T13:45:07.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:07 smithi107 ceph-mon[33178]: osdmap e745: 8 total, 8 up, 8 in 2024-06-11T13:45:08.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:08 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:45:08.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:08 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:45:08.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:08 smithi057 ceph-mon[29805]: pgmap v1168: 300 pgs: 300 active+clean; 4.4 MiB data, 857 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-06-11T13:45:08.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:08 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:45:08.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:08 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:45:08.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:08 smithi057 ceph-mon[40588]: pgmap v1168: 300 pgs: 300 active+clean; 4.4 MiB data, 857 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-06-11T13:45:08.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:08 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:45:08.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:08 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:45:08.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:08 smithi107 ceph-mon[33178]: pgmap v1168: 300 pgs: 300 active+clean; 4.4 MiB data, 857 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-06-11T13:45:09.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:09 smithi057 ceph-mon[29805]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-06-11T13:45:09.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:09 smithi057 ceph-mon[40588]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-06-11T13:45:09.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:09 smithi107 ceph-mon[33178]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-06-11T13:45:10.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:10 smithi057 ceph-mon[29805]: pgmap v1169: 300 pgs: 300 active+clean; 4.4 MiB data, 857 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-06-11T13:45:10.785 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:10 smithi057 ceph-mon[40588]: pgmap v1169: 300 pgs: 300 active+clean; 4.4 MiB data, 857 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-06-11T13:45:10.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:10 smithi107 ceph-mon[33178]: pgmap v1169: 300 pgs: 300 active+clean; 4.4 MiB data, 857 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-06-11T13:45:10.855 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.855 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.855 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.855 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.855 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.856 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.856 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.856 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.856 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.856 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.856 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.856 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.856 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.856 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.856 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.857 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.857 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.857 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.857 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.857 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.857 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.857 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.857 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.857 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.857 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.857 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.857 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.857 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.857 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.858 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.858 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.858 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.858 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.858 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.858 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.858 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.859 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.859 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.859 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.859 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.859 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.859 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.859 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.860 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.860 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.860 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.860 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.860 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.860 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.861 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.861 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.861 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.861 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.861 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.861 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.861 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.862 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.862 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.862 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.862 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.862 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.862 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.862 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.863 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.863 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.863 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.863 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.863 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.863 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.863 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.864 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.864 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.864 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.864 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.864 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.864 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.864 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.865 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.865 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.865 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.865 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.865 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.865 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.865 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.866 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.866 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.866 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.866 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.866 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.866 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.866 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.867 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.867 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.867 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.867 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.867 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.867 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.867 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.868 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:10.868 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:45:12.298 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:45:11 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:45:13.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:12 smithi057 ceph-mon[29805]: pgmap v1170: 300 pgs: 300 active+clean; 4.4 MiB data, 857 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-06-11T13:45:13.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:12 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:45:13.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:12 smithi057 ceph-mon[40588]: pgmap v1170: 300 pgs: 300 active+clean; 4.4 MiB data, 857 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-06-11T13:45:13.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:12 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:45:13.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:12 smithi107 ceph-mon[33178]: pgmap v1170: 300 pgs: 300 active+clean; 4.4 MiB data, 857 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-06-11T13:45:13.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:12 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:45:14.035 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:45:13 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:45:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:45:15.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:14 smithi057 ceph-mon[29805]: pgmap v1171: 300 pgs: 300 active+clean; 4.4 MiB data, 857 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-06-11T13:45:15.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:14 smithi057 ceph-mon[40588]: pgmap v1171: 300 pgs: 300 active+clean; 4.4 MiB data, 857 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-06-11T13:45:15.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:14 smithi107 ceph-mon[33178]: pgmap v1171: 300 pgs: 300 active+clean; 4.4 MiB data, 857 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-06-11T13:45:17.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:16 smithi057 ceph-mon[29805]: pgmap v1172: 300 pgs: 300 active+clean; 4.4 MiB data, 857 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-06-11T13:45:17.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:16 smithi057 ceph-mon[40588]: pgmap v1172: 300 pgs: 300 active+clean; 4.4 MiB data, 857 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-06-11T13:45:17.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:16 smithi107 ceph-mon[33178]: pgmap v1172: 300 pgs: 300 active+clean; 4.4 MiB data, 857 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-06-11T13:45:18.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:17 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:18.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:17 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:18.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:17 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:18.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:17 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:18.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:17 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:18.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:17 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:19.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:18 smithi057 ceph-mon[40588]: pgmap v1173: 300 pgs: 300 active+clean; 4.4 MiB data, 857 MiB used, 714 GiB / 715 GiB avail; 979 B/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:45:19.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:18 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:45:19.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:18 smithi057 ceph-mon[40588]: osdmap e746: 8 total, 8 up, 8 in 2024-06-11T13:45:19.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:18 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-146"}]: dispatch 2024-06-11T13:45:19.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:18 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-146"}]: dispatch 2024-06-11T13:45:19.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:18 smithi057 ceph-mon[29805]: pgmap v1173: 300 pgs: 300 active+clean; 4.4 MiB data, 857 MiB used, 714 GiB / 715 GiB avail; 979 B/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:45:19.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:18 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:45:19.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:18 smithi057 ceph-mon[29805]: osdmap e746: 8 total, 8 up, 8 in 2024-06-11T13:45:19.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:18 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-146"}]: dispatch 2024-06-11T13:45:19.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:18 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-146"}]: dispatch 2024-06-11T13:45:19.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:18 smithi107 ceph-mon[33178]: pgmap v1173: 300 pgs: 300 active+clean; 4.4 MiB data, 857 MiB used, 714 GiB / 715 GiB avail; 979 B/s rd, 0 B/s wr, 1 op/s 2024-06-11T13:45:19.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:18 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:45:19.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:18 smithi107 ceph-mon[33178]: osdmap e746: 8 total, 8 up, 8 in 2024-06-11T13:45:19.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:18 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-146"}]: dispatch 2024-06-11T13:45:19.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:18 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-146"}]: dispatch 2024-06-11T13:45:20.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:19 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-146"}]': finished 2024-06-11T13:45:20.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:19 smithi057 ceph-mon[29805]: osdmap e747: 8 total, 8 up, 8 in 2024-06-11T13:45:20.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:19 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:20.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:19 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:20.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:19 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-146"}]: dispatch 2024-06-11T13:45:20.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:19 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-146"}]: dispatch 2024-06-11T13:45:20.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:19 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-146"}]': finished 2024-06-11T13:45:20.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:19 smithi057 ceph-mon[40588]: osdmap e747: 8 total, 8 up, 8 in 2024-06-11T13:45:20.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:19 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:20.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:19 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:20.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:19 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-146"}]: dispatch 2024-06-11T13:45:20.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:19 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-146"}]: dispatch 2024-06-11T13:45:20.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:19 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-146"}]': finished 2024-06-11T13:45:20.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:19 smithi107 ceph-mon[33178]: osdmap e747: 8 total, 8 up, 8 in 2024-06-11T13:45:20.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:19 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:20.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:19 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:20.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:19 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-146"}]: dispatch 2024-06-11T13:45:20.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:19 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-146"}]: dispatch 2024-06-11T13:45:21.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:20 smithi057 ceph-mon[29805]: pgmap v1176: 300 pgs: 300 active+clean; 4.4 MiB data, 857 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-06-11T13:45:21.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:20 smithi057 ceph-mon[29805]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:45:21.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:20 smithi057 ceph-mon[29805]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-06-11T13:45:21.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:20 smithi057 ceph-mon[29805]: osdmap e748: 8 total, 8 up, 8 in 2024-06-11T13:45:21.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:20 smithi057 ceph-mon[40588]: pgmap v1176: 300 pgs: 300 active+clean; 4.4 MiB data, 857 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-06-11T13:45:21.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:20 smithi057 ceph-mon[40588]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:45:21.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:20 smithi057 ceph-mon[40588]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-06-11T13:45:21.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:20 smithi057 ceph-mon[40588]: osdmap e748: 8 total, 8 up, 8 in 2024-06-11T13:45:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:20 smithi107 ceph-mon[33178]: pgmap v1176: 300 pgs: 300 active+clean; 4.4 MiB data, 857 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-06-11T13:45:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:20 smithi107 ceph-mon[33178]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:45:21.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:20 smithi107 ceph-mon[33178]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-06-11T13:45:21.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:20 smithi107 ceph-mon[33178]: osdmap e748: 8 total, 8 up, 8 in 2024-06-11T13:45:22.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:21 smithi057 ceph-mon[29805]: osdmap e749: 8 total, 8 up, 8 in 2024-06-11T13:45:22.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:21 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:45:22.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:21 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:45:22.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:21 smithi057 ceph-mon[40588]: osdmap e749: 8 total, 8 up, 8 in 2024-06-11T13:45:22.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:21 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:45:22.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:21 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:45:22.049 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:45:21 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:45:22.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:21 smithi107 ceph-mon[33178]: osdmap e749: 8 total, 8 up, 8 in 2024-06-11T13:45:22.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:21 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:45:22.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:21 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:45:23.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:22 smithi057 ceph-mon[40588]: pgmap v1179: 300 pgs: 32 creating+peering, 268 active+clean; 4.4 MiB data, 858 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-06-11T13:45:23.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:22 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:45:23.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:22 smithi057 ceph-mon[40588]: osdmap e750: 8 total, 8 up, 8 in 2024-06-11T13:45:23.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:22 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:45:23.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:22 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:45:23.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:22 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:45:23.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:22 smithi057 ceph-mon[29805]: pgmap v1179: 300 pgs: 32 creating+peering, 268 active+clean; 4.4 MiB data, 858 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-06-11T13:45:23.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:22 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:45:23.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:22 smithi057 ceph-mon[29805]: osdmap e750: 8 total, 8 up, 8 in 2024-06-11T13:45:23.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:22 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:45:23.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:22 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:45:23.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:22 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:45:23.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:22 smithi107 ceph-mon[33178]: pgmap v1179: 300 pgs: 32 creating+peering, 268 active+clean; 4.4 MiB data, 858 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-06-11T13:45:23.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:22 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:45:23.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:22 smithi107 ceph-mon[33178]: osdmap e750: 8 total, 8 up, 8 in 2024-06-11T13:45:23.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:22 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:45:23.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:22 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-06-11T13:45:23.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:22 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:45:24.035 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:45:23 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:45:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:45:24.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:23 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:45:24.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:23 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-148", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:45:24.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:23 smithi057 ceph-mon[40588]: osdmap e751: 8 total, 8 up, 8 in 2024-06-11T13:45:24.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:23 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-148"}]: dispatch 2024-06-11T13:45:24.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:23 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-148"}]: dispatch 2024-06-11T13:45:24.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:23 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:45:24.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:23 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-148", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:45:24.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:23 smithi057 ceph-mon[29805]: osdmap e751: 8 total, 8 up, 8 in 2024-06-11T13:45:24.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:23 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-148"}]: dispatch 2024-06-11T13:45:24.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:23 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-148"}]: dispatch 2024-06-11T13:45:24.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:23 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:45:24.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:23 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-148", "force_nonempty": "--force-nonempty" }]': finished 2024-06-11T13:45:24.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:23 smithi107 ceph-mon[33178]: osdmap e751: 8 total, 8 up, 8 in 2024-06-11T13:45:24.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:23 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-148"}]: dispatch 2024-06-11T13:45:24.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:23 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-148"}]: dispatch 2024-06-11T13:45:25.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:24 smithi057 ceph-mon[29805]: pgmap v1182: 300 pgs: 32 creating+peering, 268 active+clean; 4.4 MiB data, 858 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-06-11T13:45:25.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:24 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-148"}]': finished 2024-06-11T13:45:25.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:24 smithi057 ceph-mon[29805]: osdmap e752: 8 total, 8 up, 8 in 2024-06-11T13:45:25.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:24 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:25.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:24 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:25.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:24 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-148"}]: dispatch 2024-06-11T13:45:25.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:24 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-148"}]: dispatch 2024-06-11T13:45:25.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:24 smithi057 ceph-mon[40588]: pgmap v1182: 300 pgs: 32 creating+peering, 268 active+clean; 4.4 MiB data, 858 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-06-11T13:45:25.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:24 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-148"}]': finished 2024-06-11T13:45:25.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:24 smithi057 ceph-mon[40588]: osdmap e752: 8 total, 8 up, 8 in 2024-06-11T13:45:25.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:24 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:25.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:24 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:25.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:24 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-148"}]: dispatch 2024-06-11T13:45:25.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:24 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-148"}]: dispatch 2024-06-11T13:45:25.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:24 smithi107 ceph-mon[33178]: pgmap v1182: 300 pgs: 32 creating+peering, 268 active+clean; 4.4 MiB data, 858 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-06-11T13:45:25.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:24 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-148"}]': finished 2024-06-11T13:45:25.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:24 smithi107 ceph-mon[33178]: osdmap e752: 8 total, 8 up, 8 in 2024-06-11T13:45:25.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:24 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:25.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:24 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:25.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:24 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-148"}]: dispatch 2024-06-11T13:45:25.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:24 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-148"}]: dispatch 2024-06-11T13:45:26.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:25 smithi057 ceph-mon[29805]: osdmap e753: 8 total, 8 up, 8 in 2024-06-11T13:45:26.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:25 smithi057 ceph-mon[40588]: osdmap e753: 8 total, 8 up, 8 in 2024-06-11T13:45:26.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:25 smithi107 ceph-mon[33178]: osdmap e753: 8 total, 8 up, 8 in 2024-06-11T13:45:27.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:26 smithi057 ceph-mon[29805]: pgmap v1185: 268 pgs: 268 active+clean; 4.4 MiB data, 858 MiB used, 714 GiB / 715 GiB avail 2024-06-11T13:45:27.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:26 smithi057 ceph-mon[29805]: osdmap e754: 8 total, 8 up, 8 in 2024-06-11T13:45:27.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:26 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:45:27.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:26 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:45:27.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:26 smithi057 ceph-mon[40588]: pgmap v1185: 268 pgs: 268 active+clean; 4.4 MiB data, 858 MiB used, 714 GiB / 715 GiB avail 2024-06-11T13:45:27.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:26 smithi057 ceph-mon[40588]: osdmap e754: 8 total, 8 up, 8 in 2024-06-11T13:45:27.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:26 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:45:27.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:26 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:45:27.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:26 smithi107 ceph-mon[33178]: pgmap v1185: 268 pgs: 268 active+clean; 4.4 MiB data, 858 MiB used, 714 GiB / 715 GiB avail 2024-06-11T13:45:27.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:26 smithi107 ceph-mon[33178]: osdmap e754: 8 total, 8 up, 8 in 2024-06-11T13:45:27.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:26 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:45:27.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:26 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:45:28.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:27 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:45:28.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:27 smithi057 ceph-mon[29805]: osdmap e755: 8 total, 8 up, 8 in 2024-06-11T13:45:28.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:27 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:28.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:27 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:28.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:27 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-150"}]: dispatch 2024-06-11T13:45:28.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:27 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-150"}]: dispatch 2024-06-11T13:45:28.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:27 smithi057 ceph-mon[29805]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:45:28.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:27 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:45:28.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:27 smithi057 ceph-mon[40588]: osdmap e755: 8 total, 8 up, 8 in 2024-06-11T13:45:28.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:27 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:28.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:27 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:28.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:27 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-150"}]: dispatch 2024-06-11T13:45:28.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:27 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-150"}]: dispatch 2024-06-11T13:45:28.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:27 smithi057 ceph-mon[40588]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:45:28.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:27 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:45:28.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:27 smithi107 ceph-mon[33178]: osdmap e755: 8 total, 8 up, 8 in 2024-06-11T13:45:28.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:27 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:28.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:27 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:28.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:27 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-150"}]: dispatch 2024-06-11T13:45:28.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:27 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-150"}]: dispatch 2024-06-11T13:45:28.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:27 smithi107 ceph-mon[33178]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:45:29.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:28 smithi057 ceph-mon[29805]: pgmap v1188: 300 pgs: 32 creating+peering, 268 active+clean; 4.4 MiB data, 866 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-11T13:45:29.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:28 smithi057 ceph-mon[29805]: osdmap e756: 8 total, 8 up, 8 in 2024-06-11T13:45:29.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:28 smithi057 ceph-mon[40588]: pgmap v1188: 300 pgs: 32 creating+peering, 268 active+clean; 4.4 MiB data, 866 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-11T13:45:29.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:28 smithi057 ceph-mon[40588]: osdmap e756: 8 total, 8 up, 8 in 2024-06-11T13:45:29.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:28 smithi107 ceph-mon[33178]: pgmap v1188: 300 pgs: 32 creating+peering, 268 active+clean; 4.4 MiB data, 866 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-11T13:45:29.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:28 smithi107 ceph-mon[33178]: osdmap e756: 8 total, 8 up, 8 in 2024-06-11T13:45:30.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:29 smithi057 ceph-mon[29805]: osdmap e757: 8 total, 8 up, 8 in 2024-06-11T13:45:30.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:29 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:45:30.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:29 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:45:30.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:29 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:45:30.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:29 smithi057 ceph-mon[29805]: osdmap e758: 8 total, 8 up, 8 in 2024-06-11T13:45:30.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:29 smithi057 ceph-mon[40588]: osdmap e757: 8 total, 8 up, 8 in 2024-06-11T13:45:30.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:29 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:45:30.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:29 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:45:30.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:29 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:45:30.038 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:29 smithi057 ceph-mon[40588]: osdmap e758: 8 total, 8 up, 8 in 2024-06-11T13:45:30.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:29 smithi107 ceph-mon[33178]: osdmap e757: 8 total, 8 up, 8 in 2024-06-11T13:45:30.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:29 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:45:30.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:29 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:45:30.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:29 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:45:30.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:29 smithi107 ceph-mon[33178]: osdmap e758: 8 total, 8 up, 8 in 2024-06-11T13:45:31.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:31 smithi057 ceph-mon[29805]: pgmap v1191: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 866 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-11T13:45:31.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:31 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:31.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:31 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:31.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:31 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-152"}]: dispatch 2024-06-11T13:45:31.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:31 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-152"}]: dispatch 2024-06-11T13:45:31.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:31 smithi057 ceph-mon[29805]: osdmap e759: 8 total, 8 up, 8 in 2024-06-11T13:45:31.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:31 smithi057 ceph-mon[40588]: pgmap v1191: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 866 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-11T13:45:31.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:31 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:31.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:31 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:31.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:31 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-152"}]: dispatch 2024-06-11T13:45:31.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:31 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-152"}]: dispatch 2024-06-11T13:45:31.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:31 smithi057 ceph-mon[40588]: osdmap e759: 8 total, 8 up, 8 in 2024-06-11T13:45:31.548 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:31 smithi107 ceph-mon[33178]: pgmap v1191: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 866 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-06-11T13:45:31.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:31 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:31.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:31 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:31.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:31 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-152"}]: dispatch 2024-06-11T13:45:31.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:31 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-152"}]: dispatch 2024-06-11T13:45:31.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:31 smithi107 ceph-mon[33178]: osdmap e759: 8 total, 8 up, 8 in 2024-06-11T13:45:32.298 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:45:31 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:45:33.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:32 smithi057 ceph-mon[29805]: pgmap v1194: 268 pgs: 268 active+clean; 4.4 MiB data, 902 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-06-11T13:45:33.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:32 smithi057 ceph-mon[29805]: osdmap e760: 8 total, 8 up, 8 in 2024-06-11T13:45:33.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:32 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:45:33.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:32 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:45:33.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:32 smithi057 ceph-mon[40588]: pgmap v1194: 268 pgs: 268 active+clean; 4.4 MiB data, 902 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-06-11T13:45:33.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:32 smithi057 ceph-mon[40588]: osdmap e760: 8 total, 8 up, 8 in 2024-06-11T13:45:33.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:32 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:45:33.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:32 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:45:33.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:32 smithi107 ceph-mon[33178]: pgmap v1194: 268 pgs: 268 active+clean; 4.4 MiB data, 902 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-06-11T13:45:33.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:32 smithi107 ceph-mon[33178]: osdmap e760: 8 total, 8 up, 8 in 2024-06-11T13:45:33.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:32 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:45:33.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:32 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T13:45:34.035 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:45:33 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:45:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:45:34.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:33 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:45:34.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:33 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:45:34.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:33 smithi057 ceph-mon[29805]: osdmap e761: 8 total, 8 up, 8 in 2024-06-11T13:45:34.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:33 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-111","var": "dedup_tier","val": "test-rados-api-smithi057-77599-154"}]: dispatch 2024-06-11T13:45:34.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:33 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-111","var": "dedup_tier","val": "test-rados-api-smithi057-77599-154"}]: dispatch 2024-06-11T13:45:34.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:33 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:34.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:33 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:34.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:33 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-154"}]: dispatch 2024-06-11T13:45:34.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:33 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-154"}]: dispatch 2024-06-11T13:45:34.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:33 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:45:34.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:33 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:45:34.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:33 smithi057 ceph-mon[40588]: osdmap e761: 8 total, 8 up, 8 in 2024-06-11T13:45:34.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:33 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-111","var": "dedup_tier","val": "test-rados-api-smithi057-77599-154"}]: dispatch 2024-06-11T13:45:34.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:33 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-111","var": "dedup_tier","val": "test-rados-api-smithi057-77599-154"}]: dispatch 2024-06-11T13:45:34.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:33 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:34.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:33 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:34.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:33 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-154"}]: dispatch 2024-06-11T13:45:34.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:33 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-154"}]: dispatch 2024-06-11T13:45:34.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:33 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:45:34.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:33 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi057-77599-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T13:45:34.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:33 smithi107 ceph-mon[33178]: osdmap e761: 8 total, 8 up, 8 in 2024-06-11T13:45:34.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:33 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-111","var": "dedup_tier","val": "test-rados-api-smithi057-77599-154"}]: dispatch 2024-06-11T13:45:34.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:33 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi057-77599-111","var": "dedup_tier","val": "test-rados-api-smithi057-77599-154"}]: dispatch 2024-06-11T13:45:34.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:33 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:34.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:33 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:34.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:33 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-154"}]: dispatch 2024-06-11T13:45:34.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:33 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi057-77599-111", "tierpool": "test-rados-api-smithi057-77599-154"}]: dispatch 2024-06-11T13:45:35.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:34 smithi107 ceph-mon[33178]: pgmap v1197: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 902 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-06-11T13:45:35.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:34 smithi107 ceph-mon[33178]: osdmap e762: 8 total, 8 up, 8 in 2024-06-11T13:45:35.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:34 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:45:35.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:34 smithi057 ceph-mon[29805]: pgmap v1197: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 902 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-06-11T13:45:35.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:34 smithi057 ceph-mon[29805]: osdmap e762: 8 total, 8 up, 8 in 2024-06-11T13:45:35.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:34 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:45:35.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:34 smithi057 ceph-mon[40588]: pgmap v1197: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 902 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-06-11T13:45:35.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:34 smithi057 ceph-mon[40588]: osdmap e762: 8 total, 8 up, 8 in 2024-06-11T13:45:35.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:34 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:45:36.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:35 smithi057 ceph-mon[29805]: osdmap e763: 8 total, 8 up, 8 in 2024-06-11T13:45:36.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:35 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:36.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:35 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:36.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:35 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:45:36.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:35 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:45:36.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:35 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:45:36.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:35 smithi057 ceph-mon[40588]: osdmap e763: 8 total, 8 up, 8 in 2024-06-11T13:45:36.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:35 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:36.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:35 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:36.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:35 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:45:36.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:35 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:45:36.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:35 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:45:36.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:35 smithi107 ceph-mon[33178]: osdmap e763: 8 total, 8 up, 8 in 2024-06-11T13:45:36.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:35 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:36.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:35 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:36.301 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:35 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:45:36.301 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:35 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:45:36.301 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:35 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:45:36.826 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: wait ] LibRadosTwoPoolsECPP.TryFlush (7751 ms) 2024-06-11T13:45:36.826 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FailedFlush 2024-06-11T13:45:36.827 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FailedFlush (12439 ms) 2024-06-11T13:45:36.827 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Flush 2024-06-11T13:45:36.827 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Flush (8145 ms) 2024-06-11T13:45:36.827 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushSnap 2024-06-11T13:45:36.827 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushSnap (13170 ms) 2024-06-11T13:45:36.827 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2024-06-11T13:45:36.827 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushTryFlushRaces (7288 ms) 2024-06-11T13:45:36.827 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlushReadRace 2024-06-11T13:45:36.827 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlushReadRace (8125 ms) 2024-06-11T13:45:36.827 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetRead 2024-06-11T13:45:36.827 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: hmm, no HitSet yet 2024-06-11T13:45:36.827 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: ok, hit_set contains 336:602f83fe:::foo:head 2024-06-11T13:45:36.828 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetRead (9100 ms) 2024-06-11T13:45:36.828 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetTrim 2024-06-11T13:45:36.828 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: got ls 1718113451,0 2024-06-11T13:45:36.828 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: first is 1718113451 2024-06-11T13:45:36.828 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: got ls 1718113451,0 2024-06-11T13:45:36.828 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: got ls 1718113451,0 2024-06-11T13:45:36.828 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: got ls 1718113451,1718113453,1718113454,0 2024-06-11T13:45:36.828 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: got ls 1718113451,1718113453,1718113454,0 2024-06-11T13:45:36.828 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: got ls 1718113451,1718113453,1718113454,0 2024-06-11T13:45:36.828 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: got ls 1718113451,1718113453,1718113454,1718113456,1718113457,0 2024-06-11T13:45:36.829 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: got ls 1718113451,1718113453,1718113454,1718113456,1718113457,0 2024-06-11T13:45:36.829 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: got ls 1718113451,1718113453,1718113454,1718113456,1718113457,0 2024-06-11T13:45:36.829 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: got ls 1718113451,1718113453,1718113454,1718113456,1718113457,1718113459,1718113460,0 2024-06-11T13:45:36.829 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: got ls 1718113451,1718113453,1718113454,1718113456,1718113457,1718113459,1718113460,0 2024-06-11T13:45:36.829 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: got ls 1718113451,1718113453,1718113454,1718113456,1718113457,1718113459,1718113460,0 2024-06-11T13:45:36.829 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: got ls 1718113454,1718113456,1718113457,1718113459,1718113460,1718113462,1718113463,0 2024-06-11T13:45:36.829 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: first now 1718113454, trimmed 2024-06-11T13:45:36.829 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetTrim (20195 ms) 2024-06-11T13:45:36.829 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2024-06-11T13:45:36.829 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: foo0 2024-06-11T13:45:36.829 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-06-11T13:45:36.829 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteOn2ndRead (14168 ms) 2024-06-11T13:45:36.829 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ProxyRead 2024-06-11T13:45:36.829 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ProxyRead (19178 ms) 2024-06-11T13:45:36.830 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.CachePin 2024-06-11T13:45:36.830 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.CachePin (22689 ms) 2024-06-11T13:45:36.830 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetRedirectRead 2024-06-11T13:45:36.830 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetRedirectRead (5069 ms) 2024-06-11T13:45:36.830 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetChunkRead 2024-06-11T13:45:36.830 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetChunkRead (3049 ms) 2024-06-11T13:45:36.830 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2024-06-11T13:45:36.830 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ManifestPromoteRead (3009 ms) 2024-06-11T13:45:36.830 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TrySetDedupTier 2024-06-11T13:45:36.830 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TrySetDedupTier (3036 ms) 2024-06-11T13:45:36.830 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP (232332 ms total) 2024-06-11T13:45:36.830 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: 2024-06-11T13:45:36.830 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [----------] Global test environment tear-down 2024-06-11T13:45:36.830 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [==========] 77 tests from 4 test suites ran. (893136 ms total) 2024-06-11T13:45:36.831 INFO:tasks.workunit.client.0.smithi057.stdout: api_tier_pp: [ PASSED ] 77 tests. 2024-06-11T13:45:36.831 INFO:tasks.workunit.client.0.smithi057.stderr:+ for t in "${!pids[@]}" 2024-06-11T13:45:36.832 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77316 2024-06-11T13:45:36.832 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 77316 2024-06-11T13:45:36.832 INFO:tasks.workunit.client.0.smithi057.stderr:+ for t in "${!pids[@]}" 2024-06-11T13:45:36.832 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77584 2024-06-11T13:45:36.832 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 77584 2024-06-11T13:45:36.832 INFO:tasks.workunit.client.0.smithi057.stderr:+ for t in "${!pids[@]}" 2024-06-11T13:45:36.832 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=78054 2024-06-11T13:45:36.832 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 78054 2024-06-11T13:45:36.832 INFO:tasks.workunit.client.0.smithi057.stderr:+ for t in "${!pids[@]}" 2024-06-11T13:45:36.832 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77847 2024-06-11T13:45:36.832 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 77847 2024-06-11T13:45:36.832 INFO:tasks.workunit.client.0.smithi057.stderr:+ for t in "${!pids[@]}" 2024-06-11T13:45:36.832 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77658 2024-06-11T13:45:36.833 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 77658 2024-06-11T13:45:36.833 INFO:tasks.workunit.client.0.smithi057.stderr:+ for t in "${!pids[@]}" 2024-06-11T13:45:36.833 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77399 2024-06-11T13:45:36.833 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 77399 2024-06-11T13:45:37.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:36 smithi057 ceph-mon[29805]: pgmap v1200: 260 pgs: 260 active+clean; 583 KiB data, 902 MiB used, 714 GiB / 715 GiB avail 2024-06-11T13:45:37.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:36 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:45:37.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:36 smithi057 ceph-mon[29805]: osdmap e764: 8 total, 8 up, 8 in 2024-06-11T13:45:37.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:36 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:37.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:36 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:37.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:36 smithi057 ceph-mon[29805]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:45:37.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:36 smithi057 ceph-mon[29805]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:45:37.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:36 smithi057 ceph-mon[29805]: osdmap e765: 8 total, 8 up, 8 in 2024-06-11T13:45:37.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:36 smithi057 ceph-mon[40588]: pgmap v1200: 260 pgs: 260 active+clean; 583 KiB data, 902 MiB used, 714 GiB / 715 GiB avail 2024-06-11T13:45:37.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:36 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:45:37.288 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:36 smithi057 ceph-mon[40588]: osdmap e764: 8 total, 8 up, 8 in 2024-06-11T13:45:37.288 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:36 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:37.288 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:36 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:37.288 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:36 smithi057 ceph-mon[40588]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:45:37.288 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:36 smithi057 ceph-mon[40588]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:45:37.288 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:36 smithi057 ceph-mon[40588]: osdmap e765: 8 total, 8 up, 8 in 2024-06-11T13:45:37.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:36 smithi107 ceph-mon[33178]: pgmap v1200: 260 pgs: 260 active+clean; 583 KiB data, 902 MiB used, 714 GiB / 715 GiB avail 2024-06-11T13:45:37.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:36 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:45:37.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:36 smithi107 ceph-mon[33178]: osdmap e764: 8 total, 8 up, 8 in 2024-06-11T13:45:37.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:36 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1314076303' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:37.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:36 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi057-77599-111"}]: dispatch 2024-06-11T13:45:37.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:36 smithi107 ceph-mon[33178]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T13:45:37.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:36 smithi107 ceph-mon[33178]: from='client.36431 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi057-77599-111"}]': finished 2024-06-11T13:45:37.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:36 smithi107 ceph-mon[33178]: osdmap e765: 8 total, 8 up, 8 in 2024-06-11T13:45:38.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:38 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:45:38.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:38 smithi057 ceph-mon[29805]: pgmap v1203: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail 2024-06-11T13:45:38.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:38 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:45:38.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:38 smithi057 ceph-mon[40588]: pgmap v1203: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail 2024-06-11T13:45:38.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:38 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:45:38.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:38 smithi107 ceph-mon[33178]: pgmap v1203: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail 2024-06-11T13:45:41.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:40 smithi057 ceph-mon[29805]: pgmap v1204: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail 2024-06-11T13:45:41.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:40 smithi057 ceph-mon[40588]: pgmap v1204: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail 2024-06-11T13:45:41.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:40 smithi107 ceph-mon[33178]: pgmap v1204: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail 2024-06-11T13:45:42.299 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:45:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:45:43.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:42 smithi057 ceph-mon[29805]: pgmap v1205: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-06-11T13:45:43.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:42 smithi057 ceph-mon[40588]: pgmap v1205: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-06-11T13:45:43.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:42 smithi107 ceph-mon[33178]: pgmap v1205: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-06-11T13:45:44.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:43 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:45:44.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:43 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:45:44.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:45:43 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:45:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:45:44.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:43 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:45:45.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:44 smithi057 ceph-mon[29805]: pgmap v1206: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:45:45.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:44 smithi057 ceph-mon[40588]: pgmap v1206: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:45:45.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:44 smithi107 ceph-mon[33178]: pgmap v1206: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:45:47.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:46 smithi057 ceph-mon[29805]: pgmap v1207: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-06-11T13:45:47.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:46 smithi057 ceph-mon[40588]: pgmap v1207: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-06-11T13:45:47.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:46 smithi107 ceph-mon[33178]: pgmap v1207: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-06-11T13:45:49.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:48 smithi057 ceph-mon[29805]: pgmap v1208: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 956 B/s rd, 0 op/s 2024-06-11T13:45:49.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:48 smithi057 ceph-mon[40588]: pgmap v1208: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 956 B/s rd, 0 op/s 2024-06-11T13:45:49.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:48 smithi107 ceph-mon[33178]: pgmap v1208: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 956 B/s rd, 0 op/s 2024-06-11T13:45:51.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:50 smithi057 ceph-mon[29805]: pgmap v1209: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:45:51.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:50 smithi057 ceph-mon[40588]: pgmap v1209: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:45:51.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:50 smithi107 ceph-mon[33178]: pgmap v1209: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:45:52.298 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:45:51 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:45:53.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:52 smithi057 ceph-mon[29805]: pgmap v1210: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:45:53.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:52 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:45:53.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:52 smithi057 ceph-mon[40588]: pgmap v1210: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:45:53.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:52 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:45:53.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:52 smithi107 ceph-mon[33178]: pgmap v1210: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:45:53.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:52 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:45:54.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:53 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:45:54.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:53 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:45:54.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:45:53 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:45:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:45:54.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:53 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:45:54.936 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:54 smithi057 ceph-mon[40588]: pgmap v1211: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:45:54.937 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:54 smithi057 ceph-mon[29805]: pgmap v1211: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:45:55.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:54 smithi107 ceph-mon[33178]: pgmap v1211: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:45:57.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:56 smithi057 ceph-mon[29805]: pgmap v1212: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:45:57.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:56 smithi057 ceph-mon[40588]: pgmap v1212: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:45:57.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:56 smithi107 ceph-mon[33178]: pgmap v1212: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:45:59.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:45:58 smithi057 ceph-mon[29805]: pgmap v1213: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:45:59.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:45:58 smithi057 ceph-mon[40588]: pgmap v1213: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:45:59.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:45:58 smithi107 ceph-mon[33178]: pgmap v1213: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:46:01.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:46:00 smithi057 ceph-mon[29805]: pgmap v1214: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:46:01.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:46:00 smithi057 ceph-mon[40588]: pgmap v1214: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:46:01.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:46:00 smithi107 ceph-mon[33178]: pgmap v1214: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:46:02.302 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:46:01 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:46:03.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:46:02 smithi057 ceph-mon[29805]: pgmap v1215: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:46:03.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:46:02 smithi057 ceph-mon[40588]: pgmap v1215: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:46:03.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:46:02 smithi107 ceph-mon[33178]: pgmap v1215: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:46:04.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:46:03 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:46:04.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:46:03 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:46:04.037 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:46:03 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:46:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:46:04.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:46:03 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:46:05.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:46:04 smithi057 ceph-mon[40588]: pgmap v1216: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:46:05.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:46:04 smithi057 ceph-mon[29805]: pgmap v1216: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:46:05.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:46:04 smithi107 ceph-mon[33178]: pgmap v1216: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:46:07.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:46:06 smithi057 ceph-mon[29805]: pgmap v1217: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:46:07.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:46:06 smithi057 ceph-mon[40588]: pgmap v1217: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:46:07.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:46:06 smithi107 ceph-mon[33178]: pgmap v1217: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:46:08.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:46:07 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:46:08.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:46:07 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:46:08.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:46:07 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:46:09.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:46:08 smithi057 ceph-mon[29805]: pgmap v1218: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:46:09.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:46:08 smithi057 ceph-mon[40588]: pgmap v1218: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:46:09.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:46:08 smithi107 ceph-mon[33178]: pgmap v1218: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:46:11.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:46:10 smithi057 ceph-mon[29805]: pgmap v1219: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:46:11.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:46:10 smithi057 ceph-mon[40588]: pgmap v1219: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:46:11.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:46:10 smithi107 ceph-mon[33178]: pgmap v1219: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:46:12.299 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:46:11 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:46:13.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:46:12 smithi057 ceph-mon[40588]: pgmap v1220: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:46:13.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:46:12 smithi057 ceph-mon[29805]: pgmap v1220: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:46:13.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:46:12 smithi107 ceph-mon[33178]: pgmap v1220: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:46:14.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:46:13 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:46:14.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:46:13 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:46:14.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:46:13 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:46:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:46:14.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:46:13 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:46:15.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:46:14 smithi107 ceph-mon[33178]: pgmap v1221: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:46:15.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:46:14 smithi057 ceph-mon[29805]: pgmap v1221: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:46:15.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:46:14 smithi057 ceph-mon[40588]: pgmap v1221: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:46:17.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:46:16 smithi057 ceph-mon[29805]: pgmap v1222: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:46:17.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:46:16 smithi057 ceph-mon[40588]: pgmap v1222: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:46:17.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:46:16 smithi107 ceph-mon[33178]: pgmap v1222: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:46:19.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:46:18 smithi057 ceph-mon[29805]: pgmap v1223: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:46:19.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:46:18 smithi057 ceph-mon[40588]: pgmap v1223: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:46:19.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:46:18 smithi107 ceph-mon[33178]: pgmap v1223: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:46:21.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:46:20 smithi057 ceph-mon[29805]: pgmap v1224: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:46:21.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:46:20 smithi057 ceph-mon[40588]: pgmap v1224: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:46:21.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:46:20 smithi107 ceph-mon[33178]: pgmap v1224: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:46:22.299 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:46:21 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:46:23.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:46:22 smithi057 ceph-mon[29805]: pgmap v1225: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:46:23.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:46:22 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:46:23.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:46:22 smithi057 ceph-mon[40588]: pgmap v1225: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:46:23.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:46:22 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:46:23.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:46:22 smithi107 ceph-mon[33178]: pgmap v1225: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:46:23.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:46:22 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:46:24.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:46:23 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:46:24.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:46:23 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:46:24.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:46:23 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:46:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:46:24.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:46:23 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:46:25.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:46:24 smithi057 ceph-mon[29805]: pgmap v1226: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:46:25.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:46:24 smithi057 ceph-mon[40588]: pgmap v1226: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:46:25.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:46:24 smithi107 ceph-mon[33178]: pgmap v1226: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:46:27.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:46:26 smithi057 ceph-mon[29805]: pgmap v1227: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:46:27.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:46:26 smithi057 ceph-mon[40588]: pgmap v1227: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:46:27.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:46:26 smithi107 ceph-mon[33178]: pgmap v1227: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:46:29.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:46:28 smithi057 ceph-mon[29805]: pgmap v1228: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:46:29.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:46:28 smithi057 ceph-mon[40588]: pgmap v1228: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:46:29.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:46:28 smithi107 ceph-mon[33178]: pgmap v1228: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:46:31.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:46:30 smithi057 ceph-mon[29805]: pgmap v1229: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:46:31.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:46:30 smithi057 ceph-mon[40588]: pgmap v1229: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:46:31.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:46:30 smithi107 ceph-mon[33178]: pgmap v1229: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:46:32.298 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:46:31 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:46:33.200 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:46:32 smithi107 ceph-mon[33178]: pgmap v1230: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:46:33.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:46:32 smithi057 ceph-mon[29805]: pgmap v1230: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:46:33.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:46:32 smithi057 ceph-mon[40588]: pgmap v1230: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:46:34.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:46:33 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:46:34.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:46:33 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:46:34.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:46:33 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:46:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:46:34.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:46:33 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:46:35.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:46:34 smithi057 ceph-mon[29805]: pgmap v1231: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:46:35.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:46:34 smithi057 ceph-mon[40588]: pgmap v1231: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:46:35.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:46:34 smithi107 ceph-mon[33178]: pgmap v1231: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:46:36.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:46:35 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:46:36.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:46:35 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:46:36.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:46:35 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:46:36.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:46:35 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:46:36.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:46:35 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:46:36.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:46:35 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:46:36.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:46:35 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:46:36.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:46:35 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:46:36.301 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:46:35 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:46:36.301 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:46:35 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:46:36.301 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:46:35 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:46:36.302 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:46:35 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:46:37.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:46:36 smithi057 ceph-mon[29805]: pgmap v1232: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:46:37.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:46:36 smithi057 ceph-mon[40588]: pgmap v1232: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:46:37.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:46:36 smithi107 ceph-mon[33178]: pgmap v1232: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:46:38.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:46:37 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:46:38.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:46:37 smithi057 ceph-mon[29805]: pgmap v1233: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:46:38.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:46:37 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:46:38.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:46:37 smithi057 ceph-mon[40588]: pgmap v1233: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:46:38.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:46:37 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:46:38.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:46:37 smithi107 ceph-mon[33178]: pgmap v1233: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:46:41.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:46:40 smithi057 ceph-mon[29805]: pgmap v1234: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:46:41.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:46:40 smithi057 ceph-mon[40588]: pgmap v1234: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:46:41.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:46:40 smithi107 ceph-mon[33178]: pgmap v1234: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:46:42.298 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:46:41 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:46:43.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:46:42 smithi057 ceph-mon[29805]: pgmap v1235: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:46:43.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:46:42 smithi057 ceph-mon[40588]: pgmap v1235: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:46:43.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:46:42 smithi107 ceph-mon[33178]: pgmap v1235: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:46:44.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:46:43 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:46:44.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:46:43 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:46:44.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:46:43 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:46:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:46:44.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:46:43 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:46:45.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:46:44 smithi057 ceph-mon[29805]: pgmap v1236: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:46:45.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:46:44 smithi057 ceph-mon[40588]: pgmap v1236: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:46:45.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:46:44 smithi107 ceph-mon[33178]: pgmap v1236: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:46:47.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:46:46 smithi057 ceph-mon[29805]: pgmap v1237: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:46:47.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:46:46 smithi057 ceph-mon[40588]: pgmap v1237: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:46:47.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:46:46 smithi107 ceph-mon[33178]: pgmap v1237: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:46:49.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:46:48 smithi057 ceph-mon[29805]: pgmap v1238: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:46:49.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:46:48 smithi057 ceph-mon[40588]: pgmap v1238: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:46:49.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:46:48 smithi107 ceph-mon[33178]: pgmap v1238: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:46:50.867 INFO:tasks.workunit.client.0.smithi057.stdout:ing 1 sec 2024-06-11T13:46:50.868 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.868 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.868 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.868 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.868 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.868 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.868 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.868 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.868 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.869 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.869 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.869 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.869 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.869 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.869 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.869 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.869 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.869 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.869 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.869 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.869 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.870 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.870 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.870 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.870 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.870 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.870 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.870 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.870 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.870 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.870 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.870 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.870 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.870 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.871 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.871 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.871 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.871 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.871 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.871 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.871 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.871 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.871 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.871 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.871 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.871 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.871 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.872 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.872 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.872 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.872 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.872 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.872 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.872 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.872 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.873 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.873 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.873 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.873 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.873 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.873 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.873 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.873 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.873 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.873 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.873 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.873 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.873 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.874 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.874 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.874 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.874 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.874 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.874 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.874 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.874 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.874 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.874 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.874 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.874 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.874 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.875 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.875 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.875 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.875 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.875 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.875 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.875 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.875 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.875 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.876 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.876 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.876 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.876 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.876 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.876 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.876 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.876 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:50.876 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:46:51.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:46:50 smithi057 ceph-mon[29805]: pgmap v1239: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:46:51.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:46:50 smithi057 ceph-mon[40588]: pgmap v1239: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:46:51.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:46:50 smithi107 ceph-mon[33178]: pgmap v1239: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:46:52.299 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:46:51 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:46:53.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:46:52 smithi057 ceph-mon[29805]: pgmap v1240: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:46:53.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:46:52 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:46:53.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:46:52 smithi057 ceph-mon[40588]: pgmap v1240: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:46:53.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:46:52 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:46:53.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:46:52 smithi107 ceph-mon[33178]: pgmap v1240: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:46:53.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:46:52 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:46:54.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:46:53 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:46:54.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:46:53 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:46:54.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:46:53 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:46:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:46:54.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:46:53 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:46:55.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:46:54 smithi057 ceph-mon[29805]: pgmap v1241: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:46:55.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:46:54 smithi057 ceph-mon[40588]: pgmap v1241: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:46:55.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:46:54 smithi107 ceph-mon[33178]: pgmap v1241: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:46:57.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:46:56 smithi057 ceph-mon[29805]: pgmap v1242: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:46:57.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:46:56 smithi057 ceph-mon[40588]: pgmap v1242: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:46:57.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:46:56 smithi107 ceph-mon[33178]: pgmap v1242: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:46:58.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:46:58 smithi057 ceph-mon[29805]: pgmap v1243: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:46:58.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:46:58 smithi057 ceph-mon[40588]: pgmap v1243: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:46:58.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:46:58 smithi107 ceph-mon[33178]: pgmap v1243: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:47:01.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:47:00 smithi057 ceph-mon[29805]: pgmap v1244: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:47:01.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:47:00 smithi057 ceph-mon[40588]: pgmap v1244: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:47:01.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:47:00 smithi107 ceph-mon[33178]: pgmap v1244: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:47:02.298 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:47:01 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:47:03.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:47:02 smithi057 ceph-mon[29805]: pgmap v1245: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:47:03.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:47:02 smithi057 ceph-mon[40588]: pgmap v1245: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:47:03.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:47:02 smithi107 ceph-mon[33178]: pgmap v1245: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:47:04.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:47:03 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:47:04.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:47:03 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:47:04.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:47:03 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:47:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:47:04.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:47:03 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:47:05.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:47:04 smithi057 ceph-mon[29805]: pgmap v1246: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:47:05.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:47:04 smithi057 ceph-mon[40588]: pgmap v1246: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:47:05.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:47:04 smithi107 ceph-mon[33178]: pgmap v1246: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:47:07.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:47:06 smithi057 ceph-mon[29805]: pgmap v1247: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:47:07.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:47:06 smithi057 ceph-mon[40588]: pgmap v1247: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:47:07.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:47:06 smithi107 ceph-mon[33178]: pgmap v1247: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:47:08.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:47:07 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:47:08.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:47:07 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:47:08.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:47:07 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:47:09.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:47:08 smithi057 ceph-mon[29805]: pgmap v1248: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:47:09.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:47:08 smithi057 ceph-mon[40588]: pgmap v1248: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:47:09.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:47:08 smithi107 ceph-mon[33178]: pgmap v1248: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:47:11.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:47:10 smithi057 ceph-mon[29805]: pgmap v1249: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:47:11.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:47:10 smithi057 ceph-mon[40588]: pgmap v1249: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:47:11.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:47:10 smithi107 ceph-mon[33178]: pgmap v1249: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:47:12.298 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:47:12 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:47:13.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:47:12 smithi057 ceph-mon[29805]: pgmap v1250: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:47:13.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:47:12 smithi057 ceph-mon[40588]: pgmap v1250: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:47:13.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:47:12 smithi107 ceph-mon[33178]: pgmap v1250: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:47:14.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:47:13 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:47:14.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:47:13 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:47:14.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:47:13 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:47:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:47:14.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:47:13 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:47:15.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:47:14 smithi057 ceph-mon[29805]: pgmap v1251: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:47:15.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:47:14 smithi057 ceph-mon[40588]: pgmap v1251: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:47:15.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:47:14 smithi107 ceph-mon[33178]: pgmap v1251: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:47:17.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:47:16 smithi057 ceph-mon[29805]: pgmap v1252: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:47:17.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:47:16 smithi057 ceph-mon[40588]: pgmap v1252: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:47:17.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:47:16 smithi107 ceph-mon[33178]: pgmap v1252: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:47:19.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:47:18 smithi057 ceph-mon[29805]: pgmap v1253: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:47:19.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:47:18 smithi057 ceph-mon[40588]: pgmap v1253: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:47:19.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:47:18 smithi107 ceph-mon[33178]: pgmap v1253: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:47:21.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:47:20 smithi057 ceph-mon[29805]: pgmap v1254: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:47:21.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:47:20 smithi057 ceph-mon[40588]: pgmap v1254: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:47:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:47:20 smithi107 ceph-mon[33178]: pgmap v1254: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:47:22.298 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:47:22 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:47:23.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:47:22 smithi057 ceph-mon[29805]: pgmap v1255: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:47:23.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:47:22 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:47:23.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:47:22 smithi057 ceph-mon[40588]: pgmap v1255: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:47:23.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:47:22 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:47:23.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:47:22 smithi107 ceph-mon[33178]: pgmap v1255: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:47:23.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:47:22 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:47:24.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:47:23 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:47:24.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:47:23 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:47:24.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:47:23 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:47:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:47:24.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:47:23 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:47:25.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:47:24 smithi057 ceph-mon[29805]: pgmap v1256: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:47:25.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:47:24 smithi057 ceph-mon[40588]: pgmap v1256: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:47:25.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:47:24 smithi107 ceph-mon[33178]: pgmap v1256: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:47:27.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:47:26 smithi057 ceph-mon[29805]: pgmap v1257: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:47:27.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:47:26 smithi057 ceph-mon[40588]: pgmap v1257: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:47:27.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:47:26 smithi107 ceph-mon[33178]: pgmap v1257: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:47:29.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:47:28 smithi057 ceph-mon[29805]: pgmap v1258: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:47:29.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:47:28 smithi057 ceph-mon[40588]: pgmap v1258: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:47:29.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:47:28 smithi107 ceph-mon[33178]: pgmap v1258: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:47:31.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:47:30 smithi107 ceph-mon[33178]: pgmap v1259: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:47:31.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:47:30 smithi057 ceph-mon[29805]: pgmap v1259: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:47:31.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:47:30 smithi057 ceph-mon[40588]: pgmap v1259: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:47:32.298 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:47:32 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:47:33.200 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:47:32 smithi107 ceph-mon[33178]: pgmap v1260: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:47:33.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:47:32 smithi057 ceph-mon[29805]: pgmap v1260: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:47:33.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:47:32 smithi057 ceph-mon[40588]: pgmap v1260: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:47:34.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:47:33 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:47:34.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:47:33 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:47:34.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:47:33 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:47:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:47:34.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:47:33 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:47:35.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:47:34 smithi057 ceph-mon[29805]: pgmap v1261: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:47:35.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:47:34 smithi057 ceph-mon[40588]: pgmap v1261: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:47:35.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:47:34 smithi107 ceph-mon[33178]: pgmap v1261: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:47:36.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:47:35 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:47:36.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:47:35 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:47:36.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:47:35 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:47:37.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:47:36 smithi057 ceph-mon[29805]: pgmap v1262: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:47:37.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:47:36 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:47:37.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:47:36 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:47:37.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:47:36 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:47:37.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:47:36 smithi057 ceph-mon[40588]: pgmap v1262: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:47:37.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:47:36 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:47:37.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:47:36 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:47:37.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:47:36 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:47:37.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:47:36 smithi107 ceph-mon[33178]: pgmap v1262: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:47:37.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:47:36 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:47:37.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:47:36 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:47:37.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:47:36 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:47:38.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:47:37 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:47:38.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:47:37 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:47:38.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:47:37 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:47:39.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:47:38 smithi057 ceph-mon[29805]: pgmap v1263: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:47:39.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:47:38 smithi057 ceph-mon[40588]: pgmap v1263: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:47:39.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:47:38 smithi107 ceph-mon[33178]: pgmap v1263: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:47:41.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:47:40 smithi057 ceph-mon[29805]: pgmap v1264: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:47:41.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:47:40 smithi057 ceph-mon[40588]: pgmap v1264: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:47:41.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:47:40 smithi107 ceph-mon[33178]: pgmap v1264: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:47:42.299 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:47:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:47:43.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:47:42 smithi057 ceph-mon[29805]: pgmap v1265: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:47:43.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:47:42 smithi057 ceph-mon[40588]: pgmap v1265: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:47:43.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:47:42 smithi107 ceph-mon[33178]: pgmap v1265: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:47:44.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:47:43 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:47:44.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:47:43 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:47:44.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:47:43 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:47:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:47:44.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:47:43 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:47:45.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:47:44 smithi057 ceph-mon[40588]: pgmap v1266: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:47:45.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:47:44 smithi057 ceph-mon[29805]: pgmap v1266: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:47:45.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:47:44 smithi107 ceph-mon[33178]: pgmap v1266: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:47:47.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:47:46 smithi057 ceph-mon[29805]: pgmap v1267: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:47:47.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:47:46 smithi057 ceph-mon[40588]: pgmap v1267: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:47:47.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:47:46 smithi107 ceph-mon[33178]: pgmap v1267: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:47:49.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:47:48 smithi057 ceph-mon[29805]: pgmap v1268: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:47:49.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:47:48 smithi057 ceph-mon[40588]: pgmap v1268: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:47:49.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:47:48 smithi107 ceph-mon[33178]: pgmap v1268: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:47:51.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:47:50 smithi057 ceph-mon[29805]: pgmap v1269: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:47:51.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:47:50 smithi057 ceph-mon[40588]: pgmap v1269: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:47:51.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:47:50 smithi107 ceph-mon[33178]: pgmap v1269: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:47:52.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:47:51 smithi057 ceph-mon[29805]: pgmap v1270: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:47:52.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:47:51 smithi057 ceph-mon[40588]: pgmap v1270: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:47:52.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:47:51 smithi107 ceph-mon[33178]: pgmap v1270: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:47:52.299 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:47:52 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:47:53.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:47:52 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:47:53.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:47:52 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:47:53.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:47:52 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:47:53.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:47:52 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:47:53.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:47:52 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:47:53.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:47:52 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:47:53.989 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:47:53 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:47:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:47:54.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:47:53 smithi057 ceph-mon[29805]: pgmap v1271: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:47:54.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:47:53 smithi057 ceph-mon[40588]: pgmap v1271: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:47:54.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:47:53 smithi107 ceph-mon[33178]: pgmap v1271: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:47:57.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:47:56 smithi057 ceph-mon[29805]: pgmap v1272: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:47:57.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:47:56 smithi057 ceph-mon[40588]: pgmap v1272: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:47:57.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:47:56 smithi107 ceph-mon[33178]: pgmap v1272: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:47:59.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:47:58 smithi057 ceph-mon[29805]: pgmap v1273: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:47:59.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:47:58 smithi057 ceph-mon[40588]: pgmap v1273: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:47:59.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:47:58 smithi107 ceph-mon[33178]: pgmap v1273: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:48:01.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:48:00 smithi057 ceph-mon[29805]: pgmap v1274: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:48:01.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:48:00 smithi057 ceph-mon[40588]: pgmap v1274: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:48:01.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:48:00 smithi107 ceph-mon[33178]: pgmap v1274: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:48:02.548 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:48:02 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:48:03.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:48:02 smithi057 ceph-mon[29805]: pgmap v1275: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:48:03.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:48:02 smithi057 ceph-mon[40588]: pgmap v1275: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:48:03.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:48:02 smithi107 ceph-mon[33178]: pgmap v1275: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:48:04.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:48:03 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:48:04.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:48:03 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:48:04.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:48:03 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:48:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:48:04.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:48:03 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:48:05.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:48:04 smithi057 ceph-mon[29805]: pgmap v1276: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:48:05.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:48:04 smithi057 ceph-mon[40588]: pgmap v1276: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:48:05.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:48:04 smithi107 ceph-mon[33178]: pgmap v1276: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:48:07.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:48:06 smithi057 ceph-mon[40588]: pgmap v1277: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:48:07.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:48:06 smithi057 ceph-mon[29805]: pgmap v1277: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:48:07.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:48:06 smithi107 ceph-mon[33178]: pgmap v1277: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:48:08.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:48:07 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:48:08.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:48:07 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:48:08.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:48:07 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:48:09.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:48:08 smithi057 ceph-mon[40588]: pgmap v1278: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:48:09.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:48:08 smithi057 ceph-mon[29805]: pgmap v1278: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:48:09.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:48:08 smithi107 ceph-mon[33178]: pgmap v1278: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:48:11.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:48:10 smithi057 ceph-mon[40588]: pgmap v1279: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:48:11.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:48:10 smithi057 ceph-mon[29805]: pgmap v1279: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:48:11.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:48:10 smithi107 ceph-mon[33178]: pgmap v1279: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:48:12.549 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:48:12 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:48:13.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:48:12 smithi057 ceph-mon[40588]: pgmap v1280: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:48:13.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:48:12 smithi057 ceph-mon[29805]: pgmap v1280: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:48:13.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:48:12 smithi107 ceph-mon[33178]: pgmap v1280: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:48:14.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:48:13 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:48:14.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:48:13 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:48:14.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:48:13 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:48:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:48:14.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:48:13 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:48:15.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:48:14 smithi057 ceph-mon[40588]: pgmap v1281: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:48:15.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:48:14 smithi057 ceph-mon[29805]: pgmap v1281: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:48:15.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:48:14 smithi107 ceph-mon[33178]: pgmap v1281: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:48:17.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:48:16 smithi057 ceph-mon[29805]: pgmap v1282: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:48:17.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:48:16 smithi057 ceph-mon[40588]: pgmap v1282: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:48:17.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:48:16 smithi107 ceph-mon[33178]: pgmap v1282: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:48:19.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:48:18 smithi057 ceph-mon[40588]: pgmap v1283: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:48:19.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:48:18 smithi057 ceph-mon[29805]: pgmap v1283: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:48:19.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:48:18 smithi107 ceph-mon[33178]: pgmap v1283: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:48:21.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:48:20 smithi057 ceph-mon[29805]: pgmap v1284: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:48:21.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:48:20 smithi057 ceph-mon[40588]: pgmap v1284: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:48:21.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:48:20 smithi107 ceph-mon[33178]: pgmap v1284: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:48:22.549 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:48:22 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:48:23.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:48:22 smithi057 ceph-mon[40588]: pgmap v1285: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-06-11T13:48:23.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:48:22 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:48:23.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:48:22 smithi057 ceph-mon[29805]: pgmap v1285: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-06-11T13:48:23.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:48:22 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:48:23.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:48:22 smithi107 ceph-mon[33178]: pgmap v1285: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-06-11T13:48:23.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:48:22 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:48:24.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:48:23 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:48:24.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:48:23 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:48:24.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:48:23 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:48:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:48:24.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:48:23 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:48:25.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:48:24 smithi107 ceph-mon[33178]: pgmap v1286: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 0 B/s wr, 7 op/s 2024-06-11T13:48:25.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:48:24 smithi057 ceph-mon[29805]: pgmap v1286: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 0 B/s wr, 7 op/s 2024-06-11T13:48:25.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:48:24 smithi057 ceph-mon[40588]: pgmap v1286: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 0 B/s wr, 7 op/s 2024-06-11T13:48:27.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:48:26 smithi057 ceph-mon[29805]: pgmap v1287: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 0 B/s wr, 7 op/s 2024-06-11T13:48:27.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:48:26 smithi057 ceph-mon[40588]: pgmap v1287: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 0 B/s wr, 7 op/s 2024-06-11T13:48:27.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:48:26 smithi107 ceph-mon[33178]: pgmap v1287: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 0 B/s wr, 7 op/s 2024-06-11T13:48:29.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:48:28 smithi057 ceph-mon[29805]: pgmap v1288: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-11T13:48:29.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:48:28 smithi057 ceph-mon[40588]: pgmap v1288: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-11T13:48:29.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:48:28 smithi107 ceph-mon[33178]: pgmap v1288: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-11T13:48:30.881 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.881 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.881 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.881 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.881 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.881 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.881 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.881 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.881 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.881 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.881 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.882 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.882 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.882 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.882 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.882 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.882 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.882 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.882 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.882 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.882 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.882 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.883 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.883 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.883 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.883 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.883 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.883 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.883 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.883 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.883 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.883 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.883 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.883 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.884 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.884 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.884 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.884 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.884 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.884 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.884 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.884 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.884 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.884 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.884 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.884 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.884 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.884 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.885 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.885 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.885 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.885 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.885 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.885 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.885 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.885 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.885 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.885 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.885 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.885 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.885 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.885 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.886 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.886 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.886 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.886 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.886 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.886 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.886 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.886 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.886 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.886 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.886 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.886 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.886 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.886 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.887 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.887 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.887 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.887 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.887 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.887 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.887 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.887 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.887 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.887 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.887 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.887 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.887 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.888 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.888 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.888 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.888 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.888 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.888 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.888 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.888 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.888 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.888 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:30.888 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:48:31.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:48:30 smithi057 ceph-mon[40588]: pgmap v1289: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-11T13:48:31.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:48:30 smithi057 ceph-mon[29805]: pgmap v1289: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-11T13:48:31.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:48:30 smithi107 ceph-mon[33178]: pgmap v1289: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-11T13:48:32.548 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:48:32 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:48:33.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:48:32 smithi057 ceph-mon[29805]: pgmap v1290: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-11T13:48:33.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:48:32 smithi057 ceph-mon[40588]: pgmap v1290: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-11T13:48:33.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:48:32 smithi107 ceph-mon[33178]: pgmap v1290: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-11T13:48:34.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:48:33 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:48:34.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:48:33 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:48:34.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:48:33 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:48:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:48:34.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:48:33 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:48:35.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:48:34 smithi057 ceph-mon[29805]: pgmap v1291: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 53 op/s 2024-06-11T13:48:35.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:48:34 smithi057 ceph-mon[40588]: pgmap v1291: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 53 op/s 2024-06-11T13:48:35.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:48:34 smithi107 ceph-mon[33178]: pgmap v1291: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 53 op/s 2024-06-11T13:48:37.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:48:36 smithi057 ceph-mon[29805]: pgmap v1292: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 53 op/s 2024-06-11T13:48:37.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:48:36 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:48:37.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:48:36 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:48:37.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:48:36 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:48:37.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:48:36 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:48:37.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:48:36 smithi057 ceph-mon[40588]: pgmap v1292: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 53 op/s 2024-06-11T13:48:37.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:48:36 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:48:37.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:48:36 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:48:37.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:48:36 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:48:37.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:48:36 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:48:37.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:48:36 smithi107 ceph-mon[33178]: pgmap v1292: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 53 op/s 2024-06-11T13:48:37.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:48:36 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:48:37.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:48:36 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:48:37.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:48:36 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:48:37.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:48:36 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:48:38.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:48:37 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:48:38.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:48:37 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:48:38.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:48:37 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:48:39.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:48:38 smithi057 ceph-mon[29805]: pgmap v1293: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 54 op/s 2024-06-11T13:48:39.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:48:38 smithi057 ceph-mon[40588]: pgmap v1293: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 54 op/s 2024-06-11T13:48:39.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:48:38 smithi107 ceph-mon[33178]: pgmap v1293: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 54 op/s 2024-06-11T13:48:41.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:48:40 smithi057 ceph-mon[40588]: pgmap v1294: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:48:41.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:48:40 smithi057 ceph-mon[29805]: pgmap v1294: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:48:41.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:48:40 smithi107 ceph-mon[33178]: pgmap v1294: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:48:42.549 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:48:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:48:43.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:48:42 smithi057 ceph-mon[29805]: pgmap v1295: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:48:43.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:48:42 smithi057 ceph-mon[40588]: pgmap v1295: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:48:43.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:48:42 smithi107 ceph-mon[33178]: pgmap v1295: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:48:44.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:48:43 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:48:44.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:48:43 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:48:44.037 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:48:43 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:48:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:48:44.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:48:43 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:48:45.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:48:44 smithi057 ceph-mon[29805]: pgmap v1296: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:48:45.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:48:44 smithi057 ceph-mon[40588]: pgmap v1296: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:48:45.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:48:44 smithi107 ceph-mon[33178]: pgmap v1296: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:48:46.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:48:45 smithi057 ceph-mon[29805]: pgmap v1297: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:48:46.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:48:45 smithi057 ceph-mon[40588]: pgmap v1297: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:48:46.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:48:45 smithi107 ceph-mon[33178]: pgmap v1297: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:48:49.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:48:48 smithi057 ceph-mon[29805]: pgmap v1298: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:48:49.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:48:48 smithi057 ceph-mon[40588]: pgmap v1298: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:48:49.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:48:48 smithi107 ceph-mon[33178]: pgmap v1298: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:48:51.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:48:50 smithi057 ceph-mon[29805]: pgmap v1299: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:48:51.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:48:50 smithi057 ceph-mon[40588]: pgmap v1299: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:48:51.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:48:50 smithi107 ceph-mon[33178]: pgmap v1299: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:48:52.549 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:48:52 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:48:53.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:48:52 smithi057 ceph-mon[29805]: pgmap v1300: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:48:53.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:48:52 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:48:53.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:48:52 smithi057 ceph-mon[40588]: pgmap v1300: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:48:53.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:48:52 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:48:53.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:48:52 smithi107 ceph-mon[33178]: pgmap v1300: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:48:53.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:48:52 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:48:54.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:48:53 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:48:54.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:48:53 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:48:54.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:48:53 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:48:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:48:54.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:48:53 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:48:55.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:48:54 smithi057 ceph-mon[29805]: pgmap v1301: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:48:55.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:48:54 smithi057 ceph-mon[40588]: pgmap v1301: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:48:55.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:48:54 smithi107 ceph-mon[33178]: pgmap v1301: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:48:57.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:48:56 smithi057 ceph-mon[29805]: pgmap v1302: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:48:57.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:48:56 smithi057 ceph-mon[40588]: pgmap v1302: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:48:57.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:48:56 smithi107 ceph-mon[33178]: pgmap v1302: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:48:59.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:48:58 smithi057 ceph-mon[29805]: pgmap v1303: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:48:59.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:48:58 smithi057 ceph-mon[40588]: pgmap v1303: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:48:59.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:48:58 smithi107 ceph-mon[33178]: pgmap v1303: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:49:01.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:49:00 smithi057 ceph-mon[29805]: pgmap v1304: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:49:01.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:49:00 smithi057 ceph-mon[40588]: pgmap v1304: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:49:01.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:49:00 smithi107 ceph-mon[33178]: pgmap v1304: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:49:02.549 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:49:02 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:49:03.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:49:02 smithi057 ceph-mon[29805]: pgmap v1305: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:49:03.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:49:02 smithi057 ceph-mon[40588]: pgmap v1305: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:49:03.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:49:02 smithi107 ceph-mon[33178]: pgmap v1305: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:49:04.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:49:03 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:49:04.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:49:03 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:49:04.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:49:03 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:49:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:49:04.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:49:03 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:49:05.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:49:04 smithi057 ceph-mon[40588]: pgmap v1306: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:49:05.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:49:04 smithi057 ceph-mon[29805]: pgmap v1306: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:49:05.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:49:04 smithi107 ceph-mon[33178]: pgmap v1306: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:49:07.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:49:06 smithi057 ceph-mon[40588]: pgmap v1307: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:49:07.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:49:06 smithi057 ceph-mon[29805]: pgmap v1307: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:49:07.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:49:06 smithi107 ceph-mon[33178]: pgmap v1307: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:49:08.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:49:07 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:49:08.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:49:07 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:49:08.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:49:07 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:49:09.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:49:08 smithi107 ceph-mon[33178]: pgmap v1308: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:49:09.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:49:08 smithi057 ceph-mon[29805]: pgmap v1308: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:49:09.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:49:08 smithi057 ceph-mon[40588]: pgmap v1308: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:49:11.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:49:10 smithi057 ceph-mon[29805]: pgmap v1309: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:49:11.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:49:10 smithi057 ceph-mon[40588]: pgmap v1309: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:49:11.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:49:10 smithi107 ceph-mon[33178]: pgmap v1309: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:49:12.549 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:49:12 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:49:13.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:49:12 smithi057 ceph-mon[29805]: pgmap v1310: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:49:13.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:49:12 smithi057 ceph-mon[40588]: pgmap v1310: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:49:13.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:49:12 smithi107 ceph-mon[33178]: pgmap v1310: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:49:14.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:49:13 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:49:14.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:49:13 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:49:14.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:49:13 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:49:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:49:14.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:49:13 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:49:15.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:49:14 smithi057 ceph-mon[29805]: pgmap v1311: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:49:15.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:49:14 smithi057 ceph-mon[40588]: pgmap v1311: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:49:15.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:49:14 smithi107 ceph-mon[33178]: pgmap v1311: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:49:17.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:49:16 smithi057 ceph-mon[29805]: pgmap v1312: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:49:17.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:49:16 smithi057 ceph-mon[40588]: pgmap v1312: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:49:17.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:49:16 smithi107 ceph-mon[33178]: pgmap v1312: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:49:19.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:49:18 smithi057 ceph-mon[29805]: pgmap v1313: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:49:19.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:49:18 smithi057 ceph-mon[40588]: pgmap v1313: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:49:19.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:49:18 smithi107 ceph-mon[33178]: pgmap v1313: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:49:21.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:49:20 smithi057 ceph-mon[29805]: pgmap v1314: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:49:21.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:49:20 smithi057 ceph-mon[40588]: pgmap v1314: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:49:21.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:49:20 smithi107 ceph-mon[33178]: pgmap v1314: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:49:22.549 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:49:22 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:49:23.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:49:22 smithi057 ceph-mon[29805]: pgmap v1315: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:49:23.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:49:22 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:49:23.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:49:22 smithi057 ceph-mon[40588]: pgmap v1315: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:49:23.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:49:22 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:49:23.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:49:22 smithi107 ceph-mon[33178]: pgmap v1315: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:49:23.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:49:22 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:49:24.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:49:23 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:49:24.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:49:23 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:49:24.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:49:23 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:49:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:49:24.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:49:23 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:49:25.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:49:24 smithi057 ceph-mon[29805]: pgmap v1316: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:49:25.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:49:24 smithi057 ceph-mon[40588]: pgmap v1316: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:49:25.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:49:24 smithi107 ceph-mon[33178]: pgmap v1316: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:49:27.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:49:26 smithi057 ceph-mon[29805]: pgmap v1317: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:49:27.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:49:26 smithi057 ceph-mon[40588]: pgmap v1317: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:49:27.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:49:26 smithi107 ceph-mon[33178]: pgmap v1317: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:49:29.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:49:28 smithi057 ceph-mon[29805]: pgmap v1318: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:49:29.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:49:28 smithi057 ceph-mon[40588]: pgmap v1318: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:49:29.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:49:28 smithi107 ceph-mon[33178]: pgmap v1318: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:49:31.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:49:30 smithi057 ceph-mon[40588]: pgmap v1319: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:49:31.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:49:30 smithi057 ceph-mon[29805]: pgmap v1319: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:49:31.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:49:30 smithi107 ceph-mon[33178]: pgmap v1319: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:49:32.549 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:49:32 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:49:33.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:49:32 smithi057 ceph-mon[29805]: pgmap v1320: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:49:33.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:49:32 smithi057 ceph-mon[40588]: pgmap v1320: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:49:33.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:49:32 smithi107 ceph-mon[33178]: pgmap v1320: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:49:34.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:49:33 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:49:34.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:49:33 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:49:34.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:49:33 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:49:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:49:34.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:49:33 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:49:35.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:49:34 smithi057 ceph-mon[29805]: pgmap v1321: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:49:35.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:49:34 smithi057 ceph-mon[40588]: pgmap v1321: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:49:35.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:49:34 smithi107 ceph-mon[33178]: pgmap v1321: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:49:37.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:49:36 smithi057 ceph-mon[40588]: pgmap v1322: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:49:37.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:49:36 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:49:37.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:49:36 smithi057 ceph-mon[29805]: pgmap v1322: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:49:37.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:49:36 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:49:37.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:49:36 smithi107 ceph-mon[33178]: pgmap v1322: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:49:37.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:49:36 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:49:38.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:49:37 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:49:38.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:49:37 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:49:38.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:49:37 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:49:38.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:49:37 smithi057 ceph-mon[29805]: pgmap v1323: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:49:38.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:49:37 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:49:38.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:49:37 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:49:38.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:49:37 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:49:38.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:49:37 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:49:38.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:49:37 smithi057 ceph-mon[40588]: pgmap v1323: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:49:38.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:49:37 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:49:38.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:49:37 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:49:38.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:49:37 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:49:38.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:49:37 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:49:38.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:49:37 smithi107 ceph-mon[33178]: pgmap v1323: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:49:38.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:49:37 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:49:41.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:49:40 smithi057 ceph-mon[40588]: pgmap v1324: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:49:41.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:49:40 smithi057 ceph-mon[29805]: pgmap v1324: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:49:41.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:49:40 smithi107 ceph-mon[33178]: pgmap v1324: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:49:42.549 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:49:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:49:43.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:49:42 smithi057 ceph-mon[29805]: pgmap v1325: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:49:43.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:49:42 smithi057 ceph-mon[40588]: pgmap v1325: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:49:43.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:49:42 smithi107 ceph-mon[33178]: pgmap v1325: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:49:43.049 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:49:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=cleanup t=2024-06-11T13:49:42.717338609Z level=info msg="Completed cleanup jobs" duration=3.323532ms 2024-06-11T13:49:44.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:49:43 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:49:44.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:49:43 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:49:44.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:49:43 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:49:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:49:44.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:49:43 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:49:45.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:49:44 smithi057 ceph-mon[29805]: pgmap v1326: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:49:45.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:49:44 smithi057 ceph-mon[40588]: pgmap v1326: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:49:45.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:49:44 smithi107 ceph-mon[33178]: pgmap v1326: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:49:47.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:49:46 smithi057 ceph-mon[29805]: pgmap v1327: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:49:47.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:49:46 smithi057 ceph-mon[40588]: pgmap v1327: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:49:47.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:49:46 smithi107 ceph-mon[33178]: pgmap v1327: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:49:49.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:49:48 smithi057 ceph-mon[29805]: pgmap v1328: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:49:49.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:49:48 smithi057 ceph-mon[40588]: pgmap v1328: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:49:49.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:49:48 smithi107 ceph-mon[33178]: pgmap v1328: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:49:51.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:49:50 smithi057 ceph-mon[29805]: pgmap v1329: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:49:51.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:49:50 smithi057 ceph-mon[40588]: pgmap v1329: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:49:51.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:49:50 smithi107 ceph-mon[33178]: pgmap v1329: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:49:52.549 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:49:52 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:49:53.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:49:52 smithi057 ceph-mon[29805]: pgmap v1330: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:49:53.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:49:52 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:49:53.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:49:52 smithi057 ceph-mon[40588]: pgmap v1330: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:49:53.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:49:52 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:49:53.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:49:52 smithi107 ceph-mon[33178]: pgmap v1330: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:49:53.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:49:52 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:49:54.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:49:53 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:49:54.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:49:53 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:49:54.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:49:53 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:49:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:49:54.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:49:53 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:49:55.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:49:54 smithi057 ceph-mon[40588]: pgmap v1331: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-11T13:49:55.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:49:54 smithi057 ceph-mon[29805]: pgmap v1331: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-11T13:49:55.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:49:54 smithi107 ceph-mon[33178]: pgmap v1331: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-11T13:49:57.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:49:56 smithi057 ceph-mon[29805]: pgmap v1332: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-11T13:49:57.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:49:56 smithi057 ceph-mon[40588]: pgmap v1332: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-11T13:49:57.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:49:56 smithi107 ceph-mon[33178]: pgmap v1332: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-11T13:49:59.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:49:58 smithi057 ceph-mon[29805]: pgmap v1333: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:49:59.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:49:58 smithi057 ceph-mon[40588]: pgmap v1333: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:49:59.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:49:58 smithi107 ceph-mon[33178]: pgmap v1333: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:50:01.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:50:00 smithi107 ceph-mon[33178]: pgmap v1334: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-11T13:50:01.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:50:00 smithi107 ceph-mon[33178]: overall HEALTH_WARN 4 pool(s) do not have an application enabled 2024-06-11T13:50:01.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:50:00 smithi057 ceph-mon[29805]: pgmap v1334: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-11T13:50:01.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:50:00 smithi057 ceph-mon[29805]: overall HEALTH_WARN 4 pool(s) do not have an application enabled 2024-06-11T13:50:01.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:50:00 smithi057 ceph-mon[40588]: pgmap v1334: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-11T13:50:01.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:50:00 smithi057 ceph-mon[40588]: overall HEALTH_WARN 4 pool(s) do not have an application enabled 2024-06-11T13:50:02.549 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:50:02 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:50:03.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:50:02 smithi057 ceph-mon[29805]: pgmap v1335: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:50:03.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:50:02 smithi057 ceph-mon[40588]: pgmap v1335: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:50:03.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:50:02 smithi107 ceph-mon[33178]: pgmap v1335: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:50:04.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:50:03 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:50:04.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:50:03 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:50:04.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:50:03 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:50:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:50:04.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:50:03 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:50:05.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:50:04 smithi057 ceph-mon[29805]: pgmap v1336: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:50:05.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:50:04 smithi057 ceph-mon[40588]: pgmap v1336: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:50:05.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:50:04 smithi107 ceph-mon[33178]: pgmap v1336: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:50:07.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:50:06 smithi057 ceph-mon[29805]: pgmap v1337: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:50:07.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:50:06 smithi057 ceph-mon[40588]: pgmap v1337: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:50:07.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:50:06 smithi107 ceph-mon[33178]: pgmap v1337: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:50:08.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:50:07 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:50:08.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:50:07 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:50:08.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:50:07 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:50:09.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:50:08 smithi057 ceph-mon[29805]: pgmap v1338: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:50:09.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:50:08 smithi057 ceph-mon[40588]: pgmap v1338: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:50:09.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:50:08 smithi107 ceph-mon[33178]: pgmap v1338: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:50:10.894 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.894 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.894 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.894 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.894 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.894 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.894 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.894 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.895 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.895 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.895 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.895 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.895 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.895 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.895 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.895 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.895 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.895 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.895 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.896 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.896 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.896 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.896 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.896 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.896 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.896 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.896 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.896 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.896 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.896 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.896 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.897 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.897 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.897 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.897 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.897 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.897 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.897 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.897 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.897 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.897 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.897 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.897 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.898 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.898 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.898 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.898 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.898 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.898 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.898 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.898 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.898 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.898 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.898 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.898 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.898 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.899 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.899 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.899 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.899 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.899 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.899 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.899 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.899 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.899 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.899 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.899 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.899 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.899 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.900 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.900 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.900 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.900 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.900 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.900 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.900 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.900 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.900 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.900 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.900 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.900 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.900 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.900 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.900 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.901 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.901 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.901 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.901 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.901 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.901 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.901 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.901 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.901 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.901 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.901 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.901 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.901 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.902 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.902 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:10.902 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:50:11.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:50:10 smithi057 ceph-mon[40588]: pgmap v1339: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:50:11.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:50:10 smithi057 ceph-mon[29805]: pgmap v1339: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:50:11.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:50:10 smithi107 ceph-mon[33178]: pgmap v1339: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:50:12.549 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:50:12 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:50:13.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:50:12 smithi057 ceph-mon[29805]: pgmap v1340: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:50:13.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:50:12 smithi057 ceph-mon[40588]: pgmap v1340: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:50:13.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:50:12 smithi107 ceph-mon[33178]: pgmap v1340: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:50:14.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:50:13 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:50:14.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:50:13 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:50:14.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:50:13 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:50:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:50:14.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:50:13 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:50:15.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:50:14 smithi057 ceph-mon[40588]: pgmap v1341: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:50:15.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:50:14 smithi057 ceph-mon[29805]: pgmap v1341: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:50:15.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:50:14 smithi107 ceph-mon[33178]: pgmap v1341: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:50:17.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:50:16 smithi057 ceph-mon[40588]: pgmap v1342: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:50:17.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:50:16 smithi057 ceph-mon[29805]: pgmap v1342: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:50:17.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:50:16 smithi107 ceph-mon[33178]: pgmap v1342: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:50:19.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:50:18 smithi057 ceph-mon[40588]: pgmap v1343: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:50:19.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:50:18 smithi057 ceph-mon[29805]: pgmap v1343: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:50:19.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:50:18 smithi107 ceph-mon[33178]: pgmap v1343: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:50:21.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:50:20 smithi057 ceph-mon[40588]: pgmap v1344: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:50:21.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:50:20 smithi057 ceph-mon[29805]: pgmap v1344: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:50:21.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:50:20 smithi107 ceph-mon[33178]: pgmap v1344: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:50:22.549 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:50:22 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:50:23.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:50:22 smithi057 ceph-mon[40588]: pgmap v1345: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:50:23.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:50:22 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:50:23.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:50:22 smithi057 ceph-mon[29805]: pgmap v1345: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:50:23.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:50:22 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:50:23.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:50:22 smithi107 ceph-mon[33178]: pgmap v1345: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:50:23.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:50:22 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:50:24.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:50:23 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:50:24.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:50:23 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:50:24.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:50:23 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:50:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:50:24.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:50:23 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:50:25.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:50:24 smithi057 ceph-mon[29805]: pgmap v1346: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:50:25.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:50:24 smithi057 ceph-mon[40588]: pgmap v1346: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:50:25.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:50:24 smithi107 ceph-mon[33178]: pgmap v1346: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:50:27.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:50:26 smithi057 ceph-mon[29805]: pgmap v1347: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:50:27.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:50:26 smithi057 ceph-mon[40588]: pgmap v1347: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:50:27.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:50:26 smithi107 ceph-mon[33178]: pgmap v1347: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:50:29.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:50:28 smithi057 ceph-mon[29805]: pgmap v1348: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:50:29.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:50:28 smithi057 ceph-mon[40588]: pgmap v1348: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:50:29.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:50:28 smithi107 ceph-mon[33178]: pgmap v1348: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:50:30.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:50:29 smithi057 ceph-mon[29805]: pgmap v1349: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:50:30.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:50:29 smithi057 ceph-mon[40588]: pgmap v1349: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:50:30.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:50:29 smithi107 ceph-mon[33178]: pgmap v1349: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:50:32.549 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:50:32 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:50:33.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:50:32 smithi057 ceph-mon[29805]: pgmap v1350: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:50:33.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:50:32 smithi057 ceph-mon[40588]: pgmap v1350: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:50:33.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:50:32 smithi107 ceph-mon[33178]: pgmap v1350: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:50:34.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:50:33 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:50:34.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:50:33 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:50:34.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:50:33 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:50:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:50:34.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:50:33 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:50:35.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:50:34 smithi057 ceph-mon[29805]: pgmap v1351: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:50:35.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:50:34 smithi057 ceph-mon[40588]: pgmap v1351: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:50:35.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:50:34 smithi107 ceph-mon[33178]: pgmap v1351: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:50:37.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:50:36 smithi057 ceph-mon[29805]: pgmap v1352: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:50:37.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:50:36 smithi057 ceph-mon[40588]: pgmap v1352: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:50:37.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:50:36 smithi107 ceph-mon[33178]: pgmap v1352: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:50:38.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:50:37 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:50:38.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:50:37 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:50:38.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:50:37 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:50:38.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:50:37 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:50:38.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:50:37 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:50:38.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:50:37 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:50:38.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:50:37 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:50:38.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:50:37 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:50:38.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:50:37 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:50:38.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:50:37 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:50:38.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:50:37 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:50:38.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:50:37 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:50:38.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:50:37 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:50:38.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:50:37 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:50:38.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:50:37 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:50:39.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:50:38 smithi057 ceph-mon[29805]: pgmap v1353: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:50:39.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:50:38 smithi057 ceph-mon[40588]: pgmap v1353: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:50:39.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:50:38 smithi107 ceph-mon[33178]: pgmap v1353: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:50:41.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:50:40 smithi057 ceph-mon[29805]: pgmap v1354: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:50:41.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:50:40 smithi057 ceph-mon[40588]: pgmap v1354: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:50:41.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:50:40 smithi107 ceph-mon[33178]: pgmap v1354: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:50:42.549 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:50:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:50:43.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:50:42 smithi057 ceph-mon[29805]: pgmap v1355: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:50:43.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:50:42 smithi057 ceph-mon[40588]: pgmap v1355: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:50:43.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:50:42 smithi107 ceph-mon[33178]: pgmap v1355: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:50:44.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:50:43 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:50:44.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:50:43 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:50:44.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:50:43 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:50:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:50:44.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:50:43 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:50:45.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:50:44 smithi057 ceph-mon[40588]: pgmap v1356: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:50:45.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:50:44 smithi057 ceph-mon[29805]: pgmap v1356: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:50:45.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:50:44 smithi107 ceph-mon[33178]: pgmap v1356: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:50:47.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:50:46 smithi057 ceph-mon[29805]: pgmap v1357: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:50:47.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:50:46 smithi057 ceph-mon[40588]: pgmap v1357: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:50:47.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:50:46 smithi107 ceph-mon[33178]: pgmap v1357: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:50:49.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:50:48 smithi107 ceph-mon[33178]: pgmap v1358: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:50:49.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:50:48 smithi057 ceph-mon[40588]: pgmap v1358: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:50:49.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:50:48 smithi057 ceph-mon[29805]: pgmap v1358: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:50:51.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:50:50 smithi057 ceph-mon[29805]: pgmap v1359: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:50:51.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:50:50 smithi057 ceph-mon[40588]: pgmap v1359: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:50:51.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:50:50 smithi107 ceph-mon[33178]: pgmap v1359: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:50:52.549 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:50:52 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:50:53.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:50:52 smithi057 ceph-mon[29805]: pgmap v1360: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:50:53.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:50:52 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:50:53.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:50:52 smithi057 ceph-mon[40588]: pgmap v1360: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:50:53.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:50:52 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:50:53.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:50:52 smithi107 ceph-mon[33178]: pgmap v1360: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:50:53.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:50:52 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:50:54.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:50:53 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:50:54.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:50:53 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:50:54.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:50:53 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:50:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:50:54.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:50:53 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:50:55.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:50:54 smithi057 ceph-mon[29805]: pgmap v1361: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:50:55.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:50:54 smithi057 ceph-mon[40588]: pgmap v1361: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:50:55.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:50:54 smithi107 ceph-mon[33178]: pgmap v1361: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:50:57.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:50:56 smithi057 ceph-mon[29805]: pgmap v1362: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:50:57.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:50:56 smithi057 ceph-mon[40588]: pgmap v1362: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:50:57.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:50:56 smithi107 ceph-mon[33178]: pgmap v1362: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:50:58.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:50:58 smithi057 ceph-mon[29805]: pgmap v1363: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:50:58.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:50:58 smithi057 ceph-mon[40588]: pgmap v1363: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:50:58.798 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:50:58 smithi107 ceph-mon[33178]: pgmap v1363: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:51:01.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:00 smithi057 ceph-mon[29805]: pgmap v1364: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:51:01.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:00 smithi057 ceph-mon[40588]: pgmap v1364: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:51:01.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:00 smithi107 ceph-mon[33178]: pgmap v1364: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:51:02.549 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:51:02 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:51:03.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:02 smithi057 ceph-mon[29805]: pgmap v1365: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:51:03.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:02 smithi057 ceph-mon[40588]: pgmap v1365: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:51:03.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:02 smithi107 ceph-mon[33178]: pgmap v1365: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:51:04.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:03 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:51:04.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:03 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:51:04.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:51:03 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:51:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:51:04.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:03 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:51:05.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:04 smithi057 ceph-mon[29805]: pgmap v1366: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:51:05.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:04 smithi057 ceph-mon[40588]: pgmap v1366: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:51:05.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:04 smithi107 ceph-mon[33178]: pgmap v1366: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:51:07.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:06 smithi057 ceph-mon[29805]: pgmap v1367: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:51:07.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:06 smithi057 ceph-mon[40588]: pgmap v1367: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:51:07.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:06 smithi107 ceph-mon[33178]: pgmap v1367: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:51:08.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:07 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:51:08.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:07 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:51:08.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:07 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:51:09.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:08 smithi057 ceph-mon[29805]: pgmap v1368: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:51:09.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:08 smithi057 ceph-mon[40588]: pgmap v1368: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:51:09.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:08 smithi107 ceph-mon[33178]: pgmap v1368: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:51:11.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:10 smithi057 ceph-mon[29805]: pgmap v1369: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:51:11.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:10 smithi057 ceph-mon[40588]: pgmap v1369: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:51:11.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:10 smithi107 ceph-mon[33178]: pgmap v1369: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:51:12.549 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:51:12 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:51:13.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:12 smithi057 ceph-mon[29805]: pgmap v1370: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:51:13.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:12 smithi057 ceph-mon[40588]: pgmap v1370: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:51:13.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:12 smithi107 ceph-mon[33178]: pgmap v1370: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:51:14.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:13 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:51:14.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:13 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:51:14.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:51:13 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:51:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:51:14.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:13 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:51:15.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:14 smithi057 ceph-mon[29805]: pgmap v1371: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:51:15.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:14 smithi057 ceph-mon[40588]: pgmap v1371: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:51:15.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:14 smithi107 ceph-mon[33178]: pgmap v1371: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:51:17.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:16 smithi057 ceph-mon[29805]: pgmap v1372: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:51:17.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:16 smithi057 ceph-mon[40588]: pgmap v1372: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:51:17.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:16 smithi107 ceph-mon[33178]: pgmap v1372: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:51:19.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:18 smithi057 ceph-mon[29805]: pgmap v1373: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:51:19.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:18 smithi057 ceph-mon[40588]: pgmap v1373: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:51:19.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:18 smithi107 ceph-mon[33178]: pgmap v1373: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:51:21.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:20 smithi057 ceph-mon[29805]: pgmap v1374: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:51:21.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:20 smithi057 ceph-mon[40588]: pgmap v1374: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:51:21.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:20 smithi107 ceph-mon[33178]: pgmap v1374: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:51:22.549 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:51:22 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:51:23.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:22 smithi057 ceph-mon[40588]: pgmap v1375: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:51:23.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:22 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:51:23.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:22 smithi057 ceph-mon[29805]: pgmap v1375: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:51:23.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:22 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:51:23.302 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:22 smithi107 ceph-mon[33178]: pgmap v1375: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:51:23.302 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:22 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:51:24.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:23 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:51:24.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:23 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:51:24.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:51:23 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:51:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:51:24.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:23 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:51:25.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:24 smithi057 ceph-mon[29805]: pgmap v1376: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:51:25.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:24 smithi057 ceph-mon[40588]: pgmap v1376: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:51:25.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:24 smithi107 ceph-mon[33178]: pgmap v1376: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:51:27.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:26 smithi057 ceph-mon[40588]: pgmap v1377: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:51:27.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:26 smithi057 ceph-mon[29805]: pgmap v1377: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:51:27.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:26 smithi107 ceph-mon[33178]: pgmap v1377: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:51:29.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:28 smithi057 ceph-mon[40588]: pgmap v1378: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:51:29.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:28 smithi057 ceph-mon[29805]: pgmap v1378: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:51:29.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:28 smithi107 ceph-mon[33178]: pgmap v1378: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:51:31.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:30 smithi057 ceph-mon[29805]: pgmap v1379: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:51:31.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:30 smithi057 ceph-mon[40588]: pgmap v1379: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:51:31.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:30 smithi107 ceph-mon[33178]: pgmap v1379: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:51:32.549 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:51:32 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:51:33.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:32 smithi057 ceph-mon[29805]: pgmap v1380: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:51:33.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:32 smithi057 ceph-mon[40588]: pgmap v1380: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:51:33.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:32 smithi107 ceph-mon[33178]: pgmap v1380: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:51:34.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:33 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:51:34.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:33 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:51:34.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:51:33 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:51:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:51:34.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:33 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:51:35.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:34 smithi057 ceph-mon[29805]: pgmap v1381: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:51:35.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:34 smithi057 ceph-mon[40588]: pgmap v1381: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:51:35.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:34 smithi107 ceph-mon[33178]: pgmap v1381: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:51:37.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:36 smithi057 ceph-mon[29805]: pgmap v1382: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:51:37.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:36 smithi057 ceph-mon[40588]: pgmap v1382: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:51:37.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:36 smithi107 ceph-mon[33178]: pgmap v1382: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:51:38.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:37 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:51:38.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:37 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:51:38.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:37 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:51:38.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:37 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:51:38.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:37 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:51:38.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:37 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:51:39.262 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:38 smithi107 ceph-mon[33178]: pgmap v1383: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:51:39.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:38 smithi057 ceph-mon[29805]: pgmap v1383: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:51:39.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:38 smithi057 ceph-mon[40588]: pgmap v1383: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:51:40.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:39 smithi057 ceph-mon[29805]: pgmap v1384: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:51:40.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:39 smithi057 ceph-mon[40588]: pgmap v1384: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:51:40.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:39 smithi107 ceph-mon[33178]: pgmap v1384: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:51:42.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:41 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:51:42.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:41 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:51:42.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:41 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:51:42.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:41 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:51:42.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:41 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:51:42.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:41 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:51:42.549 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:51:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:51:43.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:42 smithi107 ceph-mon[33178]: pgmap v1385: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:51:43.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:42 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:51:43.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:42 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:51:43.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:42 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:51:43.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:42 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:51:43.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:42 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:51:43.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:42 smithi057 ceph-mon[29805]: pgmap v1385: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:51:43.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:42 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:51:43.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:42 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:51:43.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:42 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:51:43.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:42 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:51:43.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:42 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:51:43.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:42 smithi057 ceph-mon[40588]: pgmap v1385: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:51:43.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:42 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:51:43.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:42 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:51:43.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:42 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:51:43.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:42 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:51:43.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:42 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:51:44.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:43 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:51:44.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:51:43 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:51:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:51:44.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:43 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:51:44.048 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:43 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:51:45.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:44 smithi057 ceph-mon[29805]: pgmap v1386: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:51:45.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:44 smithi057 ceph-mon[40588]: pgmap v1386: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:51:45.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:44 smithi107 ceph-mon[33178]: pgmap v1386: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:51:47.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:46 smithi057 ceph-mon[29805]: pgmap v1387: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:51:47.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:46 smithi057 ceph-mon[40588]: pgmap v1387: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:51:47.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:46 smithi107 ceph-mon[33178]: pgmap v1387: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:51:49.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:48 smithi057 ceph-mon[29805]: pgmap v1388: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:51:49.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:48 smithi057 ceph-mon[40588]: pgmap v1388: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:51:49.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:48 smithi107 ceph-mon[33178]: pgmap v1388: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:51:50.907 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.907 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.907 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.907 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.907 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.908 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.908 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.908 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.908 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.908 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.908 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.908 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.908 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.908 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.908 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.908 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.908 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.909 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.909 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.909 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.909 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.909 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.909 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.909 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.909 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.909 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.909 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.909 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.909 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.909 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.909 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.910 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.910 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.910 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.910 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.910 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.910 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.910 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.910 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.910 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.910 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.910 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.910 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.911 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.911 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.911 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.911 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.911 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.911 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.911 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.911 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.911 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.911 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.911 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.911 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.911 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.912 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.912 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.912 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.912 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.912 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.912 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.912 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.912 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.912 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.912 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.912 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.912 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.912 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.913 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.913 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.913 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.913 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.913 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.913 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.913 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.913 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.913 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.913 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.913 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.913 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.913 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.913 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.914 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.914 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.914 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.914 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.914 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.914 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.914 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.914 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.914 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.914 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.914 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.914 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.915 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.915 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.915 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.915 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:50.915 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:51:51.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:50 smithi057 ceph-mon[29805]: pgmap v1389: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:51:51.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:50 smithi057 ceph-mon[40588]: pgmap v1389: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:51:51.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:50 smithi107 ceph-mon[33178]: pgmap v1389: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:51:52.549 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:51:52 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:51:53.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:52 smithi057 ceph-mon[29805]: pgmap v1390: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:51:53.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:52 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:51:53.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:52 smithi057 ceph-mon[40588]: pgmap v1390: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:51:53.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:52 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:51:53.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:52 smithi107 ceph-mon[33178]: pgmap v1390: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:51:53.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:52 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:51:54.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:53 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:51:54.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:53 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:51:54.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:51:53 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:51:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:51:54.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:53 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:51:55.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:54 smithi057 ceph-mon[29805]: pgmap v1391: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:51:55.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:54 smithi057 ceph-mon[40588]: pgmap v1391: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:51:55.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:54 smithi107 ceph-mon[33178]: pgmap v1391: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:51:57.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:56 smithi057 ceph-mon[29805]: pgmap v1392: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:51:57.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:56 smithi057 ceph-mon[40588]: pgmap v1392: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:51:57.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:56 smithi107 ceph-mon[33178]: pgmap v1392: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:51:59.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:51:58 smithi057 ceph-mon[29805]: pgmap v1393: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:51:59.345 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:51:58 smithi057 ceph-mon[40588]: pgmap v1393: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:51:59.345 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:51:58 smithi107 ceph-mon[33178]: pgmap v1393: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:52:01.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:52:00 smithi057 ceph-mon[29805]: pgmap v1394: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:52:01.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:52:00 smithi057 ceph-mon[40588]: pgmap v1394: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:52:01.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:52:00 smithi107 ceph-mon[33178]: pgmap v1394: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:52:02.549 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:52:02 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:52:03.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:52:02 smithi057 ceph-mon[29805]: pgmap v1395: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:52:03.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:52:02 smithi057 ceph-mon[40588]: pgmap v1395: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:52:03.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:52:02 smithi107 ceph-mon[33178]: pgmap v1395: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:52:04.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:52:03 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:52:04.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:52:03 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:52:04.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:52:03 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:52:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:52:04.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:52:03 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:52:05.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:52:04 smithi057 ceph-mon[29805]: pgmap v1396: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:52:05.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:52:04 smithi057 ceph-mon[40588]: pgmap v1396: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:52:05.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:52:04 smithi107 ceph-mon[33178]: pgmap v1396: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:52:07.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:52:06 smithi057 ceph-mon[29805]: pgmap v1397: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:52:07.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:52:06 smithi057 ceph-mon[40588]: pgmap v1397: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:52:07.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:52:06 smithi107 ceph-mon[33178]: pgmap v1397: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:52:08.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:52:07 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:52:08.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:52:07 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:52:08.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:52:07 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:52:09.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:52:08 smithi057 ceph-mon[29805]: pgmap v1398: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:52:09.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:52:08 smithi057 ceph-mon[40588]: pgmap v1398: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:52:09.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:52:08 smithi107 ceph-mon[33178]: pgmap v1398: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:52:11.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:52:10 smithi057 ceph-mon[29805]: pgmap v1399: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:52:11.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:52:10 smithi057 ceph-mon[40588]: pgmap v1399: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:52:11.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:52:10 smithi107 ceph-mon[33178]: pgmap v1399: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:52:12.799 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:52:12 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:52:13.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:52:12 smithi057 ceph-mon[29805]: pgmap v1400: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:52:13.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:52:12 smithi057 ceph-mon[40588]: pgmap v1400: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:52:13.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:52:12 smithi107 ceph-mon[33178]: pgmap v1400: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:52:13.998 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:52:13 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:52:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:52:14.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:52:14 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:52:14.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:52:14 smithi057 ceph-mon[29805]: pgmap v1401: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:52:14.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:52:14 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:52:14.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:52:14 smithi057 ceph-mon[40588]: pgmap v1401: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:52:14.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:52:14 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:52:14.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:52:14 smithi107 ceph-mon[33178]: pgmap v1401: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:52:17.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:52:16 smithi057 ceph-mon[40588]: pgmap v1402: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:52:17.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:52:16 smithi057 ceph-mon[29805]: pgmap v1402: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:52:17.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:52:16 smithi107 ceph-mon[33178]: pgmap v1402: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:52:19.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:52:18 smithi057 ceph-mon[29805]: pgmap v1403: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:52:19.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:52:18 smithi057 ceph-mon[40588]: pgmap v1403: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:52:19.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:52:18 smithi107 ceph-mon[33178]: pgmap v1403: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:52:21.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:52:20 smithi057 ceph-mon[40588]: pgmap v1404: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:52:21.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:52:20 smithi057 ceph-mon[29805]: pgmap v1404: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:52:21.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:52:20 smithi107 ceph-mon[33178]: pgmap v1404: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:52:22.761 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:52:22 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:52:23.035 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:52:22 smithi057 ceph-mon[40588]: pgmap v1405: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:52:23.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:52:22 smithi057 ceph-mon[29805]: pgmap v1405: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:52:23.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:52:22 smithi107 ceph-mon[33178]: pgmap v1405: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:52:24.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:52:23 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:52:24.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:52:23 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:52:24.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:52:23 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:52:24.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:52:23 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:52:24.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:52:23 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:52:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:52:24.051 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:52:23 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:52:24.051 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:52:23 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:52:25.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:52:24 smithi057 ceph-mon[29805]: pgmap v1406: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:52:25.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:52:24 smithi057 ceph-mon[40588]: pgmap v1406: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:52:25.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:52:24 smithi107 ceph-mon[33178]: pgmap v1406: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:52:27.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:52:26 smithi057 ceph-mon[29805]: pgmap v1407: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:52:27.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:52:26 smithi057 ceph-mon[40588]: pgmap v1407: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:52:27.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:52:26 smithi107 ceph-mon[33178]: pgmap v1407: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:52:29.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:52:28 smithi107 ceph-mon[33178]: pgmap v1408: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:52:29.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:52:28 smithi057 ceph-mon[29805]: pgmap v1408: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:52:29.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:52:28 smithi057 ceph-mon[40588]: pgmap v1408: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:52:31.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:52:30 smithi107 ceph-mon[33178]: pgmap v1409: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:52:31.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:52:30 smithi057 ceph-mon[29805]: pgmap v1409: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:52:31.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:52:30 smithi057 ceph-mon[40588]: pgmap v1409: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:52:32.799 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:52:32 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:52:33.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:52:32 smithi057 ceph-mon[29805]: pgmap v1410: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:52:33.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:52:32 smithi057 ceph-mon[40588]: pgmap v1410: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:52:33.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:52:32 smithi107 ceph-mon[33178]: pgmap v1410: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:52:34.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:52:33 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:52:34.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:52:33 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:52:34.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:52:33 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:52:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:52:34.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:52:33 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:52:35.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:52:34 smithi057 ceph-mon[29805]: pgmap v1411: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:52:35.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:52:34 smithi057 ceph-mon[40588]: pgmap v1411: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:52:35.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:52:34 smithi107 ceph-mon[33178]: pgmap v1411: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:52:37.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:52:36 smithi057 ceph-mon[29805]: pgmap v1412: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:52:37.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:52:36 smithi057 ceph-mon[40588]: pgmap v1412: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:52:37.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:52:36 smithi107 ceph-mon[33178]: pgmap v1412: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:52:38.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:52:37 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:52:38.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:52:37 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:52:38.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:52:37 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:52:39.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:52:38 smithi057 ceph-mon[29805]: pgmap v1413: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:52:39.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:52:38 smithi057 ceph-mon[40588]: pgmap v1413: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:52:39.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:52:38 smithi107 ceph-mon[33178]: pgmap v1413: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:52:41.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:52:40 smithi057 ceph-mon[29805]: pgmap v1414: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:52:41.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:52:40 smithi057 ceph-mon[40588]: pgmap v1414: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:52:41.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:52:40 smithi107 ceph-mon[33178]: pgmap v1414: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:52:42.799 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:52:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:52:43.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:52:42 smithi057 ceph-mon[29805]: pgmap v1415: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:52:43.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:52:42 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:52:43.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:52:42 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:52:43.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:52:42 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:52:43.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:52:42 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:52:43.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:52:42 smithi057 ceph-mon[40588]: pgmap v1415: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:52:43.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:52:42 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:52:43.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:52:42 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:52:43.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:52:42 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:52:43.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:52:42 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:52:43.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:52:42 smithi107 ceph-mon[33178]: pgmap v1415: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:52:43.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:52:42 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:52:43.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:52:42 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:52:43.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:52:42 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:52:43.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:52:42 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:52:44.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:52:43 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:52:44.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:52:43 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:52:44.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:52:43 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:52:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:52:44.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:52:43 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:52:45.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:52:44 smithi057 ceph-mon[40588]: pgmap v1416: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:52:45.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:52:44 smithi057 ceph-mon[29805]: pgmap v1416: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:52:45.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:52:44 smithi107 ceph-mon[33178]: pgmap v1416: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:52:47.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:52:46 smithi057 ceph-mon[29805]: pgmap v1417: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:52:47.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:52:46 smithi057 ceph-mon[40588]: pgmap v1417: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:52:47.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:52:46 smithi107 ceph-mon[33178]: pgmap v1417: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:52:49.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:52:48 smithi057 ceph-mon[29805]: pgmap v1418: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:52:49.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:52:48 smithi057 ceph-mon[40588]: pgmap v1418: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:52:49.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:52:48 smithi107 ceph-mon[33178]: pgmap v1418: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:52:51.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:52:50 smithi057 ceph-mon[29805]: pgmap v1419: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:52:51.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:52:50 smithi057 ceph-mon[40588]: pgmap v1419: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:52:51.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:52:50 smithi107 ceph-mon[33178]: pgmap v1419: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:52:52.799 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:52:52 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:52:53.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:52:52 smithi057 ceph-mon[29805]: pgmap v1420: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:52:53.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:52:52 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:52:53.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:52:52 smithi057 ceph-mon[40588]: pgmap v1420: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:52:53.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:52:52 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:52:53.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:52:52 smithi107 ceph-mon[33178]: pgmap v1420: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:52:53.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:52:52 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:52:54.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:52:53 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:52:54.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:52:53 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:52:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:52:54.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:52:53 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:52:54.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:52:53 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:52:55.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:52:54 smithi057 ceph-mon[40588]: pgmap v1421: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:52:55.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:52:54 smithi057 ceph-mon[29805]: pgmap v1421: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:52:55.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:52:54 smithi107 ceph-mon[33178]: pgmap v1421: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:52:57.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:52:56 smithi057 ceph-mon[29805]: pgmap v1422: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:52:57.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:52:56 smithi057 ceph-mon[40588]: pgmap v1422: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:52:57.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:52:56 smithi107 ceph-mon[33178]: pgmap v1422: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:52:59.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:52:58 smithi057 ceph-mon[29805]: pgmap v1423: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:52:59.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:52:58 smithi057 ceph-mon[40588]: pgmap v1423: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:52:59.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:52:58 smithi107 ceph-mon[33178]: pgmap v1423: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:53:01.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:53:00 smithi057 ceph-mon[40588]: pgmap v1424: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:53:01.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:53:00 smithi057 ceph-mon[29805]: pgmap v1424: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:53:01.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:53:00 smithi107 ceph-mon[33178]: pgmap v1424: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:53:02.799 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:53:02 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:53:03.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:53:02 smithi057 ceph-mon[29805]: pgmap v1425: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:53:03.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:53:02 smithi057 ceph-mon[40588]: pgmap v1425: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:53:03.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:53:02 smithi107 ceph-mon[33178]: pgmap v1425: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:53:03.988 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:53:03 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:53:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:53:04.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:53:03 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:53:04.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:53:03 smithi057 ceph-mon[29805]: pgmap v1426: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:53:04.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:53:03 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:53:04.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:53:03 smithi057 ceph-mon[40588]: pgmap v1426: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:53:04.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:53:03 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:53:04.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:53:03 smithi107 ceph-mon[33178]: pgmap v1426: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:53:07.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:53:06 smithi057 ceph-mon[29805]: pgmap v1427: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:53:07.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:53:06 smithi057 ceph-mon[40588]: pgmap v1427: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:53:07.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:53:06 smithi107 ceph-mon[33178]: pgmap v1427: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:53:09.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:53:08 smithi057 ceph-mon[40588]: pgmap v1428: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:53:09.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:53:08 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:53:09.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:53:08 smithi057 ceph-mon[29805]: pgmap v1428: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:53:09.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:53:08 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:53:09.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:53:08 smithi107 ceph-mon[33178]: pgmap v1428: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:53:09.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:53:08 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:53:11.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:53:10 smithi057 ceph-mon[29805]: pgmap v1429: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:53:11.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:53:10 smithi057 ceph-mon[40588]: pgmap v1429: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:53:11.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:53:10 smithi107 ceph-mon[33178]: pgmap v1429: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:53:12.781 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:53:12 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:53:13.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:53:12 smithi057 ceph-mon[29805]: pgmap v1430: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:53:13.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:53:12 smithi057 ceph-mon[40588]: pgmap v1430: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:53:13.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:53:12 smithi107 ceph-mon[33178]: pgmap v1430: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:53:14.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:53:13 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:53:14.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:53:13 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:53:14.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:53:13 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:53:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:53:14.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:53:13 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:53:15.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:53:14 smithi057 ceph-mon[29805]: pgmap v1431: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:53:15.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:53:14 smithi057 ceph-mon[40588]: pgmap v1431: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:53:15.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:53:14 smithi107 ceph-mon[33178]: pgmap v1431: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:53:17.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:53:16 smithi057 ceph-mon[29805]: pgmap v1432: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:53:17.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:53:16 smithi057 ceph-mon[40588]: pgmap v1432: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:53:17.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:53:16 smithi107 ceph-mon[33178]: pgmap v1432: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:53:19.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:53:18 smithi057 ceph-mon[29805]: pgmap v1433: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:53:19.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:53:18 smithi057 ceph-mon[40588]: pgmap v1433: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:53:19.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:53:18 smithi107 ceph-mon[33178]: pgmap v1433: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:53:21.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:53:20 smithi057 ceph-mon[29805]: pgmap v1434: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:53:21.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:53:20 smithi057 ceph-mon[40588]: pgmap v1434: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:53:21.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:53:20 smithi107 ceph-mon[33178]: pgmap v1434: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:53:22.799 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:53:22 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:53:23.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:53:22 smithi057 ceph-mon[29805]: pgmap v1435: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:53:23.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:53:22 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:53:23.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:53:22 smithi057 ceph-mon[40588]: pgmap v1435: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:53:23.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:53:22 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:53:23.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:53:22 smithi107 ceph-mon[33178]: pgmap v1435: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:53:23.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:53:22 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:53:24.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:53:23 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:53:24.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:53:23 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:53:24.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:53:23 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:53:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:53:24.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:53:23 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:53:25.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:53:24 smithi057 ceph-mon[29805]: pgmap v1436: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:53:25.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:53:24 smithi057 ceph-mon[40588]: pgmap v1436: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:53:25.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:53:24 smithi107 ceph-mon[33178]: pgmap v1436: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:53:27.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:53:26 smithi057 ceph-mon[29805]: pgmap v1437: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:53:27.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:53:26 smithi057 ceph-mon[40588]: pgmap v1437: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:53:27.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:53:26 smithi107 ceph-mon[33178]: pgmap v1437: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:53:29.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:53:28 smithi057 ceph-mon[29805]: pgmap v1438: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:53:29.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:53:28 smithi057 ceph-mon[40588]: pgmap v1438: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:53:29.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:53:28 smithi107 ceph-mon[33178]: pgmap v1438: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:53:30.921 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.921 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.922 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.922 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.922 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.922 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.922 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.922 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.922 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.922 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.922 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.922 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.922 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.922 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.923 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.923 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.923 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.923 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.923 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.923 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.923 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.923 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.923 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.923 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.923 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.923 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.924 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.924 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.924 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.924 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.924 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.924 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.924 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.924 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.924 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.924 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.924 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.924 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.924 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.925 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.925 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.925 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.925 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.925 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.925 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.925 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.925 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.925 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.925 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.925 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.925 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.925 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.925 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.926 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.926 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.926 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.926 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.926 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.926 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.926 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.926 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.926 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.926 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.926 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.926 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.926 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.927 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.927 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.927 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.927 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.927 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.927 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.927 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.927 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.927 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.927 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.927 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.927 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.927 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.927 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.927 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.928 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.928 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.928 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.928 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.928 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.928 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.928 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.928 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.928 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.928 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.928 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.928 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.928 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.928 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.929 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.929 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.929 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.929 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:30.929 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:53:31.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:53:30 smithi057 ceph-mon[40588]: pgmap v1439: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:53:31.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:53:30 smithi057 ceph-mon[29805]: pgmap v1439: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:53:31.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:53:30 smithi107 ceph-mon[33178]: pgmap v1439: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:53:32.799 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:53:32 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:53:33.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:53:32 smithi057 ceph-mon[29805]: pgmap v1440: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:53:33.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:53:32 smithi057 ceph-mon[40588]: pgmap v1440: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:53:33.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:53:32 smithi107 ceph-mon[33178]: pgmap v1440: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:53:34.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:53:33 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:53:34.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:53:33 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:53:34.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:53:33 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:53:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:53:34.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:53:33 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:53:35.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:53:34 smithi057 ceph-mon[29805]: pgmap v1441: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:53:35.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:53:34 smithi057 ceph-mon[40588]: pgmap v1441: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:53:35.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:53:34 smithi107 ceph-mon[33178]: pgmap v1441: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:53:37.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:53:36 smithi057 ceph-mon[29805]: pgmap v1442: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:53:37.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:53:36 smithi057 ceph-mon[40588]: pgmap v1442: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:53:37.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:53:36 smithi107 ceph-mon[33178]: pgmap v1442: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:53:38.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:53:37 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:53:38.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:53:37 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:53:38.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:53:37 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:53:39.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:53:38 smithi057 ceph-mon[29805]: pgmap v1443: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:53:39.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:53:38 smithi057 ceph-mon[40588]: pgmap v1443: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:53:39.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:53:38 smithi107 ceph-mon[33178]: pgmap v1443: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:53:41.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:53:40 smithi057 ceph-mon[29805]: pgmap v1444: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:53:41.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:53:40 smithi057 ceph-mon[40588]: pgmap v1444: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:53:41.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:53:40 smithi107 ceph-mon[33178]: pgmap v1444: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:53:42.799 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:53:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:53:43.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:53:42 smithi057 ceph-mon[29805]: pgmap v1445: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:53:43.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:53:42 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:53:43.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:53:42 smithi057 ceph-mon[40588]: pgmap v1445: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:53:43.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:53:42 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:53:43.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:53:42 smithi107 ceph-mon[33178]: pgmap v1445: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:53:43.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:53:42 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:53:43.985 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:53:43 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:53:43.985 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:53:43 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:53:43.986 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:53:43 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:53:43.986 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:53:43 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:53:43.986 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:53:43 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:53:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:53:44.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:53:43 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:53:44.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:53:43 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:53:44.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:53:43 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:53:44.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:53:43 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:53:44.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:53:43 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:53:44.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:53:43 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:53:44.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:53:43 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:53:44.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:53:43 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:53:45.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:53:44 smithi057 ceph-mon[29805]: pgmap v1446: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:53:45.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:53:44 smithi057 ceph-mon[40588]: pgmap v1446: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:53:45.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:53:44 smithi107 ceph-mon[33178]: pgmap v1446: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:53:46.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:53:46 smithi057 ceph-mon[29805]: pgmap v1447: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:53:46.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:53:46 smithi057 ceph-mon[40588]: pgmap v1447: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:53:46.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:53:46 smithi107 ceph-mon[33178]: pgmap v1447: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:53:49.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:53:48 smithi057 ceph-mon[40588]: pgmap v1448: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:53:49.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:53:48 smithi057 ceph-mon[29805]: pgmap v1448: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:53:49.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:53:48 smithi107 ceph-mon[33178]: pgmap v1448: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:53:51.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:53:50 smithi107 ceph-mon[33178]: pgmap v1449: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:53:51.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:53:50 smithi057 ceph-mon[40588]: pgmap v1449: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:53:51.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:53:50 smithi057 ceph-mon[29805]: pgmap v1449: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:53:52.799 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:53:52 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:53:53.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:53:52 smithi057 ceph-mon[40588]: pgmap v1450: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:53:53.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:53:52 smithi057 ceph-mon[29805]: pgmap v1450: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:53:53.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:53:52 smithi107 ceph-mon[33178]: pgmap v1450: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:53:54.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:53:53 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:53:54.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:53:53 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:53:54.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:53:53 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:53:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:53:54.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:53:53 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:53:54.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:53:53 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:53:54.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:53:53 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:53:54.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:53:53 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:53:55.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:53:54 smithi057 ceph-mon[29805]: pgmap v1451: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:53:55.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:53:54 smithi057 ceph-mon[40588]: pgmap v1451: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:53:55.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:53:54 smithi107 ceph-mon[33178]: pgmap v1451: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:53:57.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:53:56 smithi057 ceph-mon[29805]: pgmap v1452: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:53:57.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:53:56 smithi057 ceph-mon[40588]: pgmap v1452: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:53:57.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:53:56 smithi107 ceph-mon[33178]: pgmap v1452: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:53:59.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:53:58 smithi057 ceph-mon[29805]: pgmap v1453: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:53:59.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:53:58 smithi057 ceph-mon[40588]: pgmap v1453: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:53:59.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:53:58 smithi107 ceph-mon[33178]: pgmap v1453: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:54:01.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:00 smithi057 ceph-mon[29805]: pgmap v1454: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:54:01.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:00 smithi057 ceph-mon[40588]: pgmap v1454: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:54:01.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:00 smithi107 ceph-mon[33178]: pgmap v1454: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:54:02.799 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:54:02 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:54:03.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:02 smithi057 ceph-mon[40588]: pgmap v1455: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:54:03.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:02 smithi057 ceph-mon[29805]: pgmap v1455: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:54:03.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:02 smithi107 ceph-mon[33178]: pgmap v1455: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:54:04.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:03 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:54:04.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:03 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:54:04.037 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:54:03 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:54:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:54:04.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:03 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:54:05.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:04 smithi057 ceph-mon[29805]: pgmap v1456: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:54:05.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:04 smithi057 ceph-mon[40588]: pgmap v1456: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:54:05.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:04 smithi107 ceph-mon[33178]: pgmap v1456: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:54:07.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:06 smithi057 ceph-mon[29805]: pgmap v1457: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:54:07.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:06 smithi057 ceph-mon[40588]: pgmap v1457: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:54:07.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:06 smithi107 ceph-mon[33178]: pgmap v1457: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:54:08.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:07 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:54:08.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:07 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:54:08.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:07 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:54:09.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:08 smithi057 ceph-mon[29805]: pgmap v1458: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:54:09.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:08 smithi057 ceph-mon[40588]: pgmap v1458: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:54:09.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:08 smithi107 ceph-mon[33178]: pgmap v1458: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:54:11.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:10 smithi057 ceph-mon[29805]: pgmap v1459: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:54:11.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:10 smithi057 ceph-mon[40588]: pgmap v1459: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:54:11.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:10 smithi107 ceph-mon[33178]: pgmap v1459: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:54:12.799 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:54:12 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:54:13.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:12 smithi057 ceph-mon[29805]: pgmap v1460: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:54:13.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:12 smithi057 ceph-mon[40588]: pgmap v1460: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:54:13.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:12 smithi107 ceph-mon[33178]: pgmap v1460: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:54:14.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:13 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:54:14.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:54:13 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:54:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:54:14.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:13 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:54:14.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:13 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:54:15.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:14 smithi057 ceph-mon[29805]: pgmap v1461: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:54:15.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:14 smithi057 ceph-mon[40588]: pgmap v1461: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:54:15.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:14 smithi107 ceph-mon[33178]: pgmap v1461: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:54:17.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:16 smithi057 ceph-mon[29805]: pgmap v1462: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:54:17.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:16 smithi057 ceph-mon[40588]: pgmap v1462: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:54:17.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:16 smithi107 ceph-mon[33178]: pgmap v1462: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:54:19.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:18 smithi057 ceph-mon[29805]: pgmap v1463: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:54:19.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:18 smithi057 ceph-mon[40588]: pgmap v1463: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:54:19.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:18 smithi107 ceph-mon[33178]: pgmap v1463: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:54:21.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:20 smithi057 ceph-mon[29805]: pgmap v1464: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:54:21.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:20 smithi057 ceph-mon[40588]: pgmap v1464: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:54:21.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:20 smithi107 ceph-mon[33178]: pgmap v1464: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:54:22.799 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:54:22 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:54:23.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:22 smithi057 ceph-mon[29805]: pgmap v1465: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:54:23.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:22 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:54:23.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:22 smithi057 ceph-mon[40588]: pgmap v1465: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:54:23.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:22 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:54:23.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:22 smithi107 ceph-mon[33178]: pgmap v1465: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:54:23.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:22 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:54:23.998 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:54:23 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:54:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:54:24.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:24 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:54:24.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:24 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:54:24.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:24 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:54:25.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:25 smithi057 ceph-mon[29805]: pgmap v1466: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:54:25.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:25 smithi057 ceph-mon[40588]: pgmap v1466: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:54:25.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:25 smithi107 ceph-mon[33178]: pgmap v1466: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:54:26.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:26 smithi057 ceph-mon[29805]: pgmap v1467: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:54:26.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:26 smithi057 ceph-mon[40588]: pgmap v1467: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:54:26.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:26 smithi107 ceph-mon[33178]: pgmap v1467: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:54:29.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:28 smithi107 ceph-mon[33178]: pgmap v1468: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:54:29.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:28 smithi057 ceph-mon[29805]: pgmap v1468: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:54:29.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:28 smithi057 ceph-mon[40588]: pgmap v1468: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:54:31.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:30 smithi107 ceph-mon[33178]: pgmap v1469: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:54:31.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:30 smithi057 ceph-mon[29805]: pgmap v1469: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:54:31.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:30 smithi057 ceph-mon[40588]: pgmap v1469: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:54:32.802 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:54:32 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:54:33.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:32 smithi057 ceph-mon[29805]: pgmap v1470: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:54:33.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:32 smithi057 ceph-mon[40588]: pgmap v1470: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:54:33.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:32 smithi107 ceph-mon[33178]: pgmap v1470: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:54:34.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:33 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:54:34.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:33 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:54:34.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:54:33 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:54:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:54:34.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:33 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:54:35.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:34 smithi057 ceph-mon[29805]: pgmap v1471: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:54:35.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:34 smithi057 ceph-mon[40588]: pgmap v1471: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:54:35.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:34 smithi107 ceph-mon[33178]: pgmap v1471: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:54:37.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:36 smithi057 ceph-mon[29805]: pgmap v1472: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:54:37.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:36 smithi057 ceph-mon[40588]: pgmap v1472: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:54:37.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:36 smithi107 ceph-mon[33178]: pgmap v1472: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:54:39.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:38 smithi057 ceph-mon[29805]: pgmap v1473: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:54:39.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:38 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:54:39.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:38 smithi057 ceph-mon[40588]: pgmap v1473: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:54:39.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:38 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:54:39.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:38 smithi107 ceph-mon[33178]: pgmap v1473: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:54:39.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:38 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:54:41.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:40 smithi057 ceph-mon[40588]: pgmap v1474: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:54:41.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:40 smithi057 ceph-mon[29805]: pgmap v1474: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:54:41.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:40 smithi107 ceph-mon[33178]: pgmap v1474: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:54:42.799 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:54:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:54:43.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:42 smithi057 ceph-mon[40588]: pgmap v1475: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:54:43.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:42 smithi057 ceph-mon[29805]: pgmap v1475: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:54:43.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:42 smithi107 ceph-mon[33178]: pgmap v1475: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:54:44.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:54:43 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:54:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:54:44.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:43 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:54:44.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:43 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:54:44.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:43 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:54:44.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:43 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:54:44.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:43 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:54:44.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:43 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:54:44.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:43 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:54:44.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:43 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:54:44.038 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:43 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:54:44.038 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:43 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:54:44.038 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:43 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:54:44.038 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:43 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:54:44.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:43 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:54:44.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:43 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:54:44.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:43 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:54:44.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:43 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:54:44.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:43 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:54:44.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:43 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:54:45.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:44 smithi057 ceph-mon[40588]: pgmap v1476: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:54:45.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:44 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:54:45.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:44 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:54:45.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:44 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:54:45.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:44 smithi057 ceph-mon[29805]: pgmap v1476: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:54:45.289 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:44 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:54:45.289 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:44 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:54:45.290 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:44 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:54:45.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:44 smithi107 ceph-mon[33178]: pgmap v1476: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:54:45.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:44 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:54:45.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:44 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:54:45.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:44 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:54:47.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:46 smithi057 ceph-mon[29805]: pgmap v1477: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:54:47.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:46 smithi057 ceph-mon[40588]: pgmap v1477: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:54:47.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:46 smithi107 ceph-mon[33178]: pgmap v1477: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:54:49.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:48 smithi057 ceph-mon[29805]: pgmap v1478: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:54:49.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:48 smithi057 ceph-mon[40588]: pgmap v1478: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:54:49.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:48 smithi107 ceph-mon[33178]: pgmap v1478: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:54:51.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:50 smithi057 ceph-mon[29805]: pgmap v1479: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:54:51.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:50 smithi057 ceph-mon[40588]: pgmap v1479: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:54:51.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:50 smithi107 ceph-mon[33178]: pgmap v1479: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:54:52.799 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:54:52 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:54:53.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:52 smithi057 ceph-mon[29805]: pgmap v1480: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:54:53.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:52 smithi057 ceph-mon[40588]: pgmap v1480: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:54:53.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:52 smithi107 ceph-mon[33178]: pgmap v1480: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:54:54.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:53 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:54:54.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:53 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:54:54.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:53 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:54:54.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:53 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:54:54.037 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:54:53 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:54:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:54:54.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:53 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:54:54.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:53 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:54:55.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:54 smithi057 ceph-mon[29805]: pgmap v1481: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:54:55.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:54 smithi057 ceph-mon[40588]: pgmap v1481: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:54:55.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:54 smithi107 ceph-mon[33178]: pgmap v1481: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:54:57.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:56 smithi057 ceph-mon[29805]: pgmap v1482: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:54:57.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:56 smithi057 ceph-mon[40588]: pgmap v1482: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:54:57.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:56 smithi107 ceph-mon[33178]: pgmap v1482: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:54:58.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:54:58 smithi057 ceph-mon[29805]: pgmap v1483: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:54:58.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:54:58 smithi057 ceph-mon[40588]: pgmap v1483: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:54:58.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:54:58 smithi107 ceph-mon[33178]: pgmap v1483: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:55:01.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:55:00 smithi057 ceph-mon[29805]: pgmap v1484: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:55:01.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:55:00 smithi057 ceph-mon[40588]: pgmap v1484: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:55:01.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:55:00 smithi107 ceph-mon[33178]: pgmap v1484: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:55:02.799 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:55:02 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:55:03.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:55:02 smithi057 ceph-mon[29805]: pgmap v1485: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:55:03.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:55:02 smithi057 ceph-mon[40588]: pgmap v1485: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:55:03.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:55:02 smithi107 ceph-mon[33178]: pgmap v1485: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:55:04.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:55:03 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:55:04.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:55:03 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:55:04.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:55:03 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:55:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:55:04.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:55:03 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:55:05.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:55:04 smithi057 ceph-mon[29805]: pgmap v1486: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:55:05.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:55:04 smithi057 ceph-mon[40588]: pgmap v1486: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:55:05.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:55:04 smithi107 ceph-mon[33178]: pgmap v1486: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:55:07.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:55:06 smithi057 ceph-mon[29805]: pgmap v1487: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:55:07.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:55:06 smithi057 ceph-mon[40588]: pgmap v1487: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:55:07.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:55:06 smithi107 ceph-mon[33178]: pgmap v1487: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:55:09.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:55:08 smithi057 ceph-mon[29805]: pgmap v1488: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:55:09.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:55:08 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:55:09.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:55:08 smithi057 ceph-mon[40588]: pgmap v1488: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:55:09.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:55:08 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:55:09.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:55:08 smithi107 ceph-mon[33178]: pgmap v1488: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:55:09.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:55:08 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:55:10.934 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.934 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.935 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.935 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.935 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.935 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.935 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.935 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.935 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.935 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.935 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.935 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.935 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.935 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.935 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.935 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.936 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.936 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.936 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.936 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.936 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.936 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.936 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.936 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.936 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.936 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.936 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.936 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.936 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.937 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.937 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.937 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.937 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.937 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.937 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.937 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.937 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.937 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.937 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.937 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.937 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.937 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.937 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.937 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.938 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.938 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.938 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.938 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.938 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.938 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.938 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.938 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.938 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.938 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.938 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.938 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.938 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.938 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.939 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.939 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.939 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.939 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.939 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.939 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.939 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.939 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.939 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.939 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.939 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.939 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.939 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.939 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.939 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.940 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.940 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.940 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.940 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.940 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.940 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.940 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.940 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.940 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.940 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.940 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.940 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.940 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.940 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.940 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.941 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.941 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.941 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.941 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.941 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.941 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.941 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.941 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.941 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.941 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.941 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:10.941 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:55:11.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:55:10 smithi057 ceph-mon[29805]: pgmap v1489: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:55:11.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:55:10 smithi057 ceph-mon[40588]: pgmap v1489: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:55:11.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:55:10 smithi107 ceph-mon[33178]: pgmap v1489: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:55:12.799 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:55:12 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:55:13.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:55:12 smithi057 ceph-mon[29805]: pgmap v1490: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:55:13.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:55:12 smithi057 ceph-mon[40588]: pgmap v1490: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:55:13.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:55:12 smithi107 ceph-mon[33178]: pgmap v1490: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:55:14.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:55:13 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:55:14.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:55:13 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:55:14.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:55:13 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:55:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:55:14.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:55:13 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:55:15.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:55:14 smithi057 ceph-mon[29805]: pgmap v1491: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:55:15.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:55:14 smithi057 ceph-mon[40588]: pgmap v1491: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:55:15.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:55:14 smithi107 ceph-mon[33178]: pgmap v1491: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:55:17.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:55:16 smithi057 ceph-mon[29805]: pgmap v1492: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:55:17.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:55:16 smithi057 ceph-mon[40588]: pgmap v1492: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:55:17.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:55:16 smithi107 ceph-mon[33178]: pgmap v1492: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:55:19.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:55:18 smithi057 ceph-mon[29805]: pgmap v1493: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:55:19.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:55:18 smithi057 ceph-mon[40588]: pgmap v1493: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:55:19.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:55:18 smithi107 ceph-mon[33178]: pgmap v1493: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:55:21.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:55:20 smithi057 ceph-mon[29805]: pgmap v1494: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:55:21.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:55:20 smithi057 ceph-mon[40588]: pgmap v1494: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:55:21.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:55:20 smithi107 ceph-mon[33178]: pgmap v1494: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:55:22.799 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:55:22 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:55:23.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:55:22 smithi057 ceph-mon[29805]: pgmap v1495: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:55:23.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:55:22 smithi057 ceph-mon[40588]: pgmap v1495: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:55:23.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:55:22 smithi107 ceph-mon[33178]: pgmap v1495: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:55:24.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:55:23 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:55:24.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:55:23 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:55:24.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:55:23 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:55:24.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:55:23 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:55:24.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:55:23 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:55:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:55:24.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:55:23 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:55:24.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:55:23 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:55:25.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:55:24 smithi057 ceph-mon[29805]: pgmap v1496: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:55:25.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:55:24 smithi057 ceph-mon[40588]: pgmap v1496: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:55:25.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:55:24 smithi107 ceph-mon[33178]: pgmap v1496: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:55:27.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:55:26 smithi057 ceph-mon[29805]: pgmap v1497: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:55:27.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:55:26 smithi057 ceph-mon[40588]: pgmap v1497: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:55:27.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:55:26 smithi107 ceph-mon[33178]: pgmap v1497: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:55:29.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:55:28 smithi057 ceph-mon[29805]: pgmap v1498: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:55:29.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:55:28 smithi057 ceph-mon[40588]: pgmap v1498: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:55:29.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:55:28 smithi107 ceph-mon[33178]: pgmap v1498: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:55:31.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:55:30 smithi057 ceph-mon[29805]: pgmap v1499: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:55:31.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:55:30 smithi057 ceph-mon[40588]: pgmap v1499: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:55:31.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:55:30 smithi107 ceph-mon[33178]: pgmap v1499: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:55:32.799 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:55:32 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:55:33.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:55:32 smithi057 ceph-mon[29805]: pgmap v1500: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:55:33.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:55:32 smithi057 ceph-mon[40588]: pgmap v1500: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:55:33.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:55:32 smithi107 ceph-mon[33178]: pgmap v1500: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:55:34.007 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:55:33 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:55:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:55:34.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:55:34 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:55:34.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:55:34 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:55:34.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:55:34 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:55:35.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:55:35 smithi057 ceph-mon[40588]: pgmap v1501: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:55:35.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:55:35 smithi057 ceph-mon[29805]: pgmap v1501: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:55:35.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:55:35 smithi107 ceph-mon[33178]: pgmap v1501: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:55:36.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:55:36 smithi057 ceph-mon[29805]: pgmap v1502: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:55:36.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:55:36 smithi057 ceph-mon[40588]: pgmap v1502: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:55:36.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:55:36 smithi107 ceph-mon[33178]: pgmap v1502: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:55:39.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:55:38 smithi057 ceph-mon[40588]: pgmap v1503: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:55:39.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:55:38 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:55:39.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:55:38 smithi057 ceph-mon[29805]: pgmap v1503: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:55:39.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:55:38 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:55:39.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:55:38 smithi107 ceph-mon[33178]: pgmap v1503: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:55:39.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:55:38 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:55:41.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:55:40 smithi057 ceph-mon[29805]: pgmap v1504: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:55:41.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:55:40 smithi057 ceph-mon[40588]: pgmap v1504: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:55:41.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:55:40 smithi107 ceph-mon[33178]: pgmap v1504: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:55:42.799 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:55:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:55:43.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:55:42 smithi057 ceph-mon[29805]: pgmap v1505: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:55:43.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:55:42 smithi057 ceph-mon[40588]: pgmap v1505: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:55:43.298 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:55:42 smithi107 ceph-mon[33178]: pgmap v1505: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:55:44.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:55:43 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:55:44.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:55:43 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:55:44.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:55:43 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:55:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:55:44.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:55:43 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:55:45.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:55:44 smithi057 ceph-mon[29805]: pgmap v1506: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:55:45.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:55:44 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:55:45.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:55:44 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:55:45.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:55:44 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:55:45.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:55:44 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:55:45.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:55:44 smithi057 ceph-mon[40588]: pgmap v1506: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:55:45.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:55:44 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:55:45.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:55:44 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:55:45.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:55:44 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:55:45.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:55:44 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:55:45.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:55:44 smithi107 ceph-mon[33178]: pgmap v1506: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:55:45.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:55:44 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:55:45.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:55:44 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:55:45.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:55:44 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:55:45.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:55:44 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:55:47.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:55:46 smithi057 ceph-mon[29805]: pgmap v1507: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:55:47.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:55:46 smithi057 ceph-mon[40588]: pgmap v1507: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:55:47.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:55:46 smithi107 ceph-mon[33178]: pgmap v1507: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:55:49.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:55:48 smithi057 ceph-mon[29805]: pgmap v1508: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:55:49.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:55:48 smithi057 ceph-mon[40588]: pgmap v1508: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:55:49.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:55:48 smithi107 ceph-mon[33178]: pgmap v1508: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:55:51.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:55:50 smithi057 ceph-mon[29805]: pgmap v1509: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:55:51.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:55:50 smithi057 ceph-mon[40588]: pgmap v1509: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:55:51.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:55:50 smithi107 ceph-mon[33178]: pgmap v1509: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:55:52.799 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:55:52 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:55:53.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:55:52 smithi057 ceph-mon[29805]: pgmap v1510: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:55:53.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:55:52 smithi057 ceph-mon[40588]: pgmap v1510: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:55:53.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:55:52 smithi107 ceph-mon[33178]: pgmap v1510: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:55:54.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:55:53 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:55:54.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:55:53 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:55:54.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:55:53 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:55:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:55:54.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:55:53 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:55:54.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:55:53 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:55:54.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:55:53 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:55:54.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:55:53 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:55:55.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:55:54 smithi057 ceph-mon[29805]: pgmap v1511: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:55:55.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:55:54 smithi057 ceph-mon[40588]: pgmap v1511: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:55:55.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:55:54 smithi107 ceph-mon[33178]: pgmap v1511: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:55:57.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:55:56 smithi057 ceph-mon[29805]: pgmap v1512: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:55:57.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:55:56 smithi057 ceph-mon[40588]: pgmap v1512: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:55:57.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:55:56 smithi107 ceph-mon[33178]: pgmap v1512: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:55:59.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:55:58 smithi057 ceph-mon[40588]: pgmap v1513: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:55:59.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:55:58 smithi057 ceph-mon[29805]: pgmap v1513: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:55:59.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:55:58 smithi107 ceph-mon[33178]: pgmap v1513: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:56:01.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:56:00 smithi057 ceph-mon[29805]: pgmap v1514: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:56:01.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:56:00 smithi057 ceph-mon[40588]: pgmap v1514: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:56:01.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:56:00 smithi107 ceph-mon[33178]: pgmap v1514: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:56:02.799 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:56:02 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:56:03.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:56:02 smithi057 ceph-mon[29805]: pgmap v1515: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:56:03.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:56:02 smithi057 ceph-mon[40588]: pgmap v1515: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:56:03.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:56:02 smithi107 ceph-mon[33178]: pgmap v1515: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:56:04.035 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:56:03 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:56:04.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:56:03 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:56:04.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:56:03 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:56:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:56:04.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:56:03 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:56:05.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:56:04 smithi057 ceph-mon[29805]: pgmap v1516: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:56:05.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:56:04 smithi057 ceph-mon[40588]: pgmap v1516: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:56:05.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:56:04 smithi107 ceph-mon[33178]: pgmap v1516: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:56:07.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:56:06 smithi057 ceph-mon[40588]: pgmap v1517: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:56:07.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:56:06 smithi057 ceph-mon[29805]: pgmap v1517: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:56:07.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:56:06 smithi107 ceph-mon[33178]: pgmap v1517: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:56:09.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:56:08 smithi057 ceph-mon[29805]: pgmap v1518: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:56:09.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:56:08 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:56:09.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:56:09 smithi057 ceph-mon[40588]: pgmap v1518: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:56:09.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:56:09 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:56:09.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:56:08 smithi107 ceph-mon[33178]: pgmap v1518: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:56:09.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:56:08 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:56:11.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:56:11 smithi057 ceph-mon[29805]: pgmap v1519: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:56:11.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:56:11 smithi057 ceph-mon[40588]: pgmap v1519: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:56:11.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:56:11 smithi107 ceph-mon[33178]: pgmap v1519: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:56:13.010 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:56:12 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:56:13.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:56:13 smithi057 ceph-mon[29805]: pgmap v1520: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:56:13.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:56:13 smithi057 ceph-mon[40588]: pgmap v1520: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:56:13.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:56:13 smithi107 ceph-mon[33178]: pgmap v1520: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:56:14.030 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:56:13 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:56:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:56:14.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:56:14 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:56:14.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:56:14 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:56:14.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:56:14 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:56:15.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:56:15 smithi057 ceph-mon[29805]: pgmap v1521: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:56:15.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:56:15 smithi057 ceph-mon[40588]: pgmap v1521: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:56:15.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:56:15 smithi107 ceph-mon[33178]: pgmap v1521: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:56:16.535 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:56:16 smithi057 ceph-mon[29805]: pgmap v1522: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:56:16.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:56:16 smithi057 ceph-mon[40588]: pgmap v1522: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:56:16.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:56:16 smithi107 ceph-mon[33178]: pgmap v1522: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:56:19.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:56:18 smithi057 ceph-mon[29805]: pgmap v1523: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:56:19.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:56:18 smithi057 ceph-mon[40588]: pgmap v1523: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:56:19.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:56:18 smithi107 ceph-mon[33178]: pgmap v1523: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:56:21.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:56:20 smithi057 ceph-mon[29805]: pgmap v1524: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:56:21.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:56:20 smithi057 ceph-mon[40588]: pgmap v1524: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:56:21.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:56:20 smithi107 ceph-mon[33178]: pgmap v1524: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:56:22.862 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:56:22 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:56:23.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:56:22 smithi057 ceph-mon[40588]: pgmap v1525: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:56:23.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:56:22 smithi057 ceph-mon[29805]: pgmap v1525: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:56:23.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:56:22 smithi107 ceph-mon[33178]: pgmap v1525: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:56:24.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:56:23 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:56:24.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:56:23 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:56:24.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:56:23 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:56:24.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:56:23 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:56:24.037 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:56:23 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:56:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:56:24.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:56:23 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:56:24.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:56:23 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:56:25.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:56:24 smithi057 ceph-mon[29805]: pgmap v1526: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:56:25.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:56:24 smithi057 ceph-mon[40588]: pgmap v1526: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:56:25.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:56:24 smithi107 ceph-mon[33178]: pgmap v1526: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:56:27.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:56:26 smithi057 ceph-mon[29805]: pgmap v1527: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:56:27.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:56:26 smithi057 ceph-mon[40588]: pgmap v1527: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:56:27.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:56:26 smithi107 ceph-mon[33178]: pgmap v1527: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:56:29.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:56:28 smithi057 ceph-mon[29805]: pgmap v1528: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:56:29.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:56:28 smithi057 ceph-mon[40588]: pgmap v1528: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:56:29.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:56:28 smithi107 ceph-mon[33178]: pgmap v1528: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:56:31.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:56:30 smithi057 ceph-mon[40588]: pgmap v1529: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:56:31.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:56:30 smithi057 ceph-mon[29805]: pgmap v1529: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:56:31.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:56:30 smithi107 ceph-mon[33178]: pgmap v1529: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:56:32.902 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:56:32 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:56:33.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:56:32 smithi057 ceph-mon[29805]: pgmap v1530: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:56:33.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:56:32 smithi057 ceph-mon[40588]: pgmap v1530: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:56:33.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:56:32 smithi107 ceph-mon[33178]: pgmap v1530: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:56:34.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:56:33 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:56:34.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:56:33 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:56:34.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:56:33 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:56:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:56:34.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:56:33 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:56:35.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:56:34 smithi057 ceph-mon[29805]: pgmap v1531: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:56:35.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:56:34 smithi057 ceph-mon[40588]: pgmap v1531: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:56:35.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:56:34 smithi107 ceph-mon[33178]: pgmap v1531: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:56:37.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:56:36 smithi057 ceph-mon[29805]: pgmap v1532: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:56:37.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:56:36 smithi057 ceph-mon[40588]: pgmap v1532: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:56:37.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:56:36 smithi107 ceph-mon[33178]: pgmap v1532: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:56:39.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:56:38 smithi057 ceph-mon[29805]: pgmap v1533: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:56:39.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:56:38 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:56:39.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:56:38 smithi057 ceph-mon[40588]: pgmap v1533: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:56:39.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:56:38 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:56:39.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:56:38 smithi107 ceph-mon[33178]: pgmap v1533: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:56:39.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:56:38 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:56:41.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:56:40 smithi057 ceph-mon[29805]: pgmap v1534: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:56:41.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:56:40 smithi057 ceph-mon[40588]: pgmap v1534: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:56:41.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:56:40 smithi107 ceph-mon[33178]: pgmap v1534: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:56:42.958 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:56:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:56:43.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:56:42 smithi057 ceph-mon[29805]: pgmap v1535: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:56:43.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:56:42 smithi057 ceph-mon[40588]: pgmap v1535: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:56:43.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:56:42 smithi107 ceph-mon[33178]: pgmap v1535: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:56:44.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:56:43 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:56:44.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:56:43 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:56:44.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:56:43 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:56:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:56:44.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:56:43 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:56:45.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:56:44 smithi057 ceph-mon[29805]: pgmap v1536: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:56:45.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:56:44 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:56:45.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:56:44 smithi057 ceph-mon[40588]: pgmap v1536: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:56:45.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:56:44 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:56:45.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:56:44 smithi107 ceph-mon[33178]: pgmap v1536: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:56:45.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:56:44 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:56:47.192 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:56:46 smithi057 ceph-mon[29805]: pgmap v1537: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:56:47.193 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:56:47 smithi057 ceph-mon[40588]: pgmap v1537: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:56:47.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:56:46 smithi107 ceph-mon[33178]: pgmap v1537: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:56:49.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:56:49 smithi057 ceph-mon[40588]: pgmap v1538: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:56:49.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:56:49 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:56:49.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:56:49 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:56:49.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:56:49 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:56:49.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:56:49 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:56:49.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:56:49 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:56:49.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:56:49 smithi057 ceph-mon[29805]: pgmap v1538: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:56:49.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:56:49 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:56:49.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:56:49 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:56:49.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:56:49 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:56:49.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:56:49 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:56:49.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:56:49 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:56:49.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:56:49 smithi107 ceph-mon[33178]: pgmap v1538: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:56:49.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:56:49 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:56:49.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:56:49 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:56:49.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:56:49 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:56:49.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:56:49 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:56:49.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:56:49 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:56:50.948 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.948 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.948 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.948 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.948 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.948 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.948 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.949 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.949 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.949 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.949 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.949 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.949 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.949 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.949 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.949 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.949 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.949 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.949 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.949 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.950 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.950 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.950 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.950 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.950 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.950 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.950 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.950 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.950 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.950 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.950 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.950 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.951 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.951 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.951 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.951 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.951 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.951 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.951 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.951 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.951 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.951 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.951 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.951 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.951 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.951 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.951 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.952 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.952 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.952 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.952 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.952 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.952 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.952 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.952 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.952 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.952 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.952 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.952 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.952 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.952 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.952 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.953 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.953 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.953 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.953 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.953 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.953 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.953 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.953 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.953 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.953 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.953 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.953 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.953 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.954 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.954 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.954 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.954 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.954 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.954 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.954 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.954 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.954 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.954 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.954 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.954 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.955 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.955 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.955 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.955 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.955 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.955 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.955 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.955 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.955 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.955 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.955 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.955 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:50.955 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:56:51.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:56:51 smithi057 ceph-mon[29805]: pgmap v1539: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:56:51.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:56:51 smithi057 ceph-mon[40588]: pgmap v1539: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:56:51.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:56:51 smithi107 ceph-mon[33178]: pgmap v1539: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:56:52.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:56:52 smithi107 ceph-mon[33178]: pgmap v1540: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:56:52.535 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:56:52 smithi057 ceph-mon[40588]: pgmap v1540: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:56:52.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:56:52 smithi057 ceph-mon[29805]: pgmap v1540: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:56:53.038 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:56:52 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:56:53.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:56:53 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:56:53.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:56:53 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:56:53.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:56:53 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:56:53.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:56:53 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:56:53.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:56:53 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:56:53.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:56:53 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:56:54.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:56:53 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:56:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:56:54.539 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:56:54 smithi057 ceph-mon[29805]: pgmap v1541: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:56:54.539 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:56:54 smithi057 ceph-mon[40588]: pgmap v1541: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:56:54.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:56:54 smithi107 ceph-mon[33178]: pgmap v1541: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:56:57.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:56:56 smithi057 ceph-mon[29805]: pgmap v1542: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:56:57.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:56:56 smithi057 ceph-mon[40588]: pgmap v1542: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:56:57.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:56:56 smithi107 ceph-mon[33178]: pgmap v1542: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:56:59.165 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:56:58 smithi057 ceph-mon[29805]: pgmap v1543: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:56:59.166 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:56:58 smithi057 ceph-mon[40588]: pgmap v1543: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:56:59.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:56:58 smithi107 ceph-mon[33178]: pgmap v1543: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:57:01.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:57:00 smithi057 ceph-mon[29805]: pgmap v1544: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:01.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:57:00 smithi057 ceph-mon[40588]: pgmap v1544: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:01.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:57:00 smithi107 ceph-mon[33178]: pgmap v1544: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:02.867 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:57:02 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:57:03.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:57:02 smithi057 ceph-mon[29805]: pgmap v1545: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:57:03.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:57:02 smithi057 ceph-mon[40588]: pgmap v1545: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:57:03.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:57:02 smithi107 ceph-mon[33178]: pgmap v1545: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:57:04.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:57:03 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:57:04.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:57:03 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:57:04.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:57:03 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:57:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:57:04.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:57:03 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:57:05.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:57:04 smithi057 ceph-mon[29805]: pgmap v1546: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:05.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:57:04 smithi057 ceph-mon[40588]: pgmap v1546: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:05.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:57:04 smithi107 ceph-mon[33178]: pgmap v1546: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:07.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:57:06 smithi057 ceph-mon[29805]: pgmap v1547: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:07.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:57:06 smithi057 ceph-mon[40588]: pgmap v1547: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:07.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:57:06 smithi107 ceph-mon[33178]: pgmap v1547: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:09.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:57:08 smithi057 ceph-mon[29805]: pgmap v1548: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:57:09.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:57:08 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:57:09.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:57:08 smithi057 ceph-mon[40588]: pgmap v1548: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:57:09.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:57:08 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:57:09.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:57:08 smithi107 ceph-mon[33178]: pgmap v1548: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:57:09.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:57:08 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:57:11.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:57:10 smithi057 ceph-mon[40588]: pgmap v1549: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:11.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:57:10 smithi057 ceph-mon[29805]: pgmap v1549: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:11.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:57:10 smithi107 ceph-mon[33178]: pgmap v1549: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:12.934 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:57:12 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:57:13.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:57:12 smithi057 ceph-mon[29805]: pgmap v1550: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:57:13.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:57:12 smithi057 ceph-mon[40588]: pgmap v1550: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:57:13.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:57:12 smithi107 ceph-mon[33178]: pgmap v1550: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:57:14.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:57:13 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:57:14.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:57:13 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:57:14.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:57:13 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:57:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:57:14.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:57:13 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:57:15.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:57:14 smithi057 ceph-mon[29805]: pgmap v1551: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:15.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:57:14 smithi057 ceph-mon[40588]: pgmap v1551: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:15.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:57:14 smithi107 ceph-mon[33178]: pgmap v1551: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:17.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:57:16 smithi057 ceph-mon[29805]: pgmap v1552: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:17.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:57:16 smithi057 ceph-mon[40588]: pgmap v1552: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:17.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:57:16 smithi107 ceph-mon[33178]: pgmap v1552: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:19.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:57:18 smithi057 ceph-mon[29805]: pgmap v1553: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:57:19.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:57:18 smithi057 ceph-mon[40588]: pgmap v1553: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:57:19.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:57:18 smithi107 ceph-mon[33178]: pgmap v1553: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:57:21.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:57:20 smithi057 ceph-mon[29805]: pgmap v1554: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:21.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:57:20 smithi057 ceph-mon[40588]: pgmap v1554: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:21.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:57:20 smithi107 ceph-mon[33178]: pgmap v1554: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:22.995 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:57:22 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:57:23.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:57:22 smithi057 ceph-mon[40588]: pgmap v1555: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:57:23.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:57:22 smithi057 ceph-mon[29805]: pgmap v1555: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:57:23.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:57:22 smithi107 ceph-mon[33178]: pgmap v1555: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:57:24.004 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:57:23 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:57:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:57:24.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:57:24 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:57:24.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:57:24 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:57:24.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:57:24 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:57:24.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:57:24 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:57:24.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:57:24 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:57:24.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:57:24 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:57:25.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:57:25 smithi057 ceph-mon[29805]: pgmap v1556: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:25.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:57:25 smithi057 ceph-mon[40588]: pgmap v1556: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:25.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:57:25 smithi107 ceph-mon[33178]: pgmap v1556: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:27.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:57:27 smithi057 ceph-mon[29805]: pgmap v1557: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:57:27.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:57:27 smithi057 ceph-mon[40588]: pgmap v1557: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:57:27.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:57:27 smithi107 ceph-mon[33178]: pgmap v1557: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:57:29.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:57:29 smithi057 ceph-mon[29805]: pgmap v1558: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:57:29.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:57:29 smithi057 ceph-mon[40588]: pgmap v1558: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:57:29.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:57:29 smithi107 ceph-mon[33178]: pgmap v1558: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:57:31.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:57:31 smithi057 ceph-mon[29805]: pgmap v1559: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:31.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:57:31 smithi057 ceph-mon[40588]: pgmap v1559: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:31.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:57:31 smithi107 ceph-mon[33178]: pgmap v1559: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:33.032 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:57:32 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:57:33.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:57:33 smithi057 ceph-mon[29805]: pgmap v1560: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:57:33.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:57:33 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:57:33.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:57:33 smithi057 ceph-mon[40588]: pgmap v1560: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:57:33.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:57:33 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:57:33.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:57:33 smithi107 ceph-mon[33178]: pgmap v1560: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:57:33.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:57:33 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:57:34.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:57:33 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:57:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:57:34.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:57:34 smithi107 ceph-mon[33178]: pgmap v1561: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:34.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:57:34 smithi057 ceph-mon[29805]: pgmap v1561: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:34.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:57:34 smithi057 ceph-mon[40588]: pgmap v1561: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:37.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:57:36 smithi057 ceph-mon[40588]: pgmap v1562: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:37.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:57:36 smithi057 ceph-mon[29805]: pgmap v1562: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:37.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:57:36 smithi107 ceph-mon[33178]: pgmap v1562: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:39.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:57:38 smithi057 ceph-mon[40588]: pgmap v1563: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:57:39.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:57:38 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:57:39.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:57:38 smithi057 ceph-mon[29805]: pgmap v1563: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:57:39.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:57:38 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:57:39.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:57:38 smithi107 ceph-mon[33178]: pgmap v1563: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:57:39.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:57:38 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:57:41.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:57:40 smithi057 ceph-mon[29805]: pgmap v1564: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:41.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:57:40 smithi057 ceph-mon[40588]: pgmap v1564: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:41.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:57:40 smithi107 ceph-mon[33178]: pgmap v1564: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:42.896 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:57:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:57:43.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:57:42 smithi057 ceph-mon[29805]: pgmap v1565: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:57:43.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:57:42 smithi057 ceph-mon[40588]: pgmap v1565: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:57:43.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:57:42 smithi107 ceph-mon[33178]: pgmap v1565: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:57:44.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:57:43 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:57:44.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:57:43 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:57:44.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:57:43 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:57:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:57:44.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:57:43 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:57:45.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:57:44 smithi057 ceph-mon[40588]: pgmap v1566: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:45.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:57:44 smithi057 ceph-mon[29805]: pgmap v1566: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:45.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:57:44 smithi107 ceph-mon[33178]: pgmap v1566: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:47.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:57:46 smithi057 ceph-mon[29805]: pgmap v1567: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:47.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:57:46 smithi057 ceph-mon[40588]: pgmap v1567: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:47.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:57:46 smithi107 ceph-mon[33178]: pgmap v1567: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:49.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:57:48 smithi057 ceph-mon[29805]: pgmap v1568: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:57:49.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:57:48 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:57:49.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:57:48 smithi057 ceph-mon[40588]: pgmap v1568: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:57:49.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:57:48 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:57:49.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:57:48 smithi107 ceph-mon[33178]: pgmap v1568: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:57:49.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:57:48 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:57:50.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:57:49 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:57:50.290 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:57:49 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:57:50.290 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:57:49 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:57:50.290 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:57:49 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:57:50.290 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:57:49 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:57:50.290 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:57:49 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:57:50.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:57:49 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:57:50.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:57:49 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:57:50.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:57:49 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:57:51.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:57:50 smithi057 ceph-mon[29805]: pgmap v1569: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:51.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:57:50 smithi057 ceph-mon[40588]: pgmap v1569: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:51.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:57:50 smithi107 ceph-mon[33178]: pgmap v1569: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:52.978 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:57:52 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:57:53.285 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:57:52 smithi057 ceph-mon[29805]: pgmap v1570: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:57:53.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:57:52 smithi057 ceph-mon[40588]: pgmap v1570: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:57:53.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:57:52 smithi107 ceph-mon[33178]: pgmap v1570: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:57:54.001 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:57:53 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:57:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:57:54.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:57:54 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:57:54.285 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:57:54 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:57:54.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:57:54 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:57:54.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:57:54 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:57:54.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:57:54 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:57:54.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:57:54 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:57:55.263 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:57:55 smithi057 ceph-mon[29805]: pgmap v1571: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:55.264 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:57:55 smithi057 ceph-mon[40588]: pgmap v1571: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:55.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:57:55 smithi107 ceph-mon[33178]: pgmap v1571: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:57:57.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:57:57 smithi057 ceph-mon[29805]: pgmap v1572: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:57:57.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:57:57 smithi057 ceph-mon[40588]: pgmap v1572: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:57:57.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:57:57 smithi107 ceph-mon[33178]: pgmap v1572: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:57:59.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:57:59 smithi057 ceph-mon[29805]: pgmap v1573: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:57:59.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:57:59 smithi057 ceph-mon[40588]: pgmap v1573: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:57:59.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:57:59 smithi107 ceph-mon[33178]: pgmap v1573: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:58:01.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:58:01 smithi057 ceph-mon[29805]: pgmap v1574: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:58:01.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:58:01 smithi057 ceph-mon[40588]: pgmap v1574: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:58:01.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:58:01 smithi107 ceph-mon[33178]: pgmap v1574: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:58:03.031 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:58:02 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:58:03.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:58:03 smithi057 ceph-mon[29805]: pgmap v1575: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:58:03.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:58:03 smithi057 ceph-mon[40588]: pgmap v1575: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:58:03.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:58:03 smithi107 ceph-mon[33178]: pgmap v1575: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:58:04.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:58:03 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:58:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:58:04.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:58:04 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:58:04.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:58:04 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:58:04.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:58:04 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:58:05.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:58:05 smithi107 ceph-mon[33178]: pgmap v1576: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:58:05.335 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:58:05 smithi057 ceph-mon[29805]: pgmap v1576: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:58:05.335 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:58:05 smithi057 ceph-mon[40588]: pgmap v1576: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:58:06.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:58:06 smithi057 ceph-mon[40588]: pgmap v1577: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:58:06.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:58:06 smithi057 ceph-mon[29805]: pgmap v1577: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:58:06.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:58:06 smithi107 ceph-mon[33178]: pgmap v1577: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:58:09.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:58:08 smithi057 ceph-mon[40588]: pgmap v1578: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:58:09.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:58:08 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:58:09.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:58:08 smithi057 ceph-mon[29805]: pgmap v1578: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:58:09.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:58:08 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:58:09.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:58:08 smithi107 ceph-mon[33178]: pgmap v1578: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:58:09.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:58:08 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:58:11.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:58:10 smithi057 ceph-mon[29805]: pgmap v1579: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:58:11.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:58:10 smithi057 ceph-mon[40588]: pgmap v1579: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:58:11.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:58:10 smithi107 ceph-mon[33178]: pgmap v1579: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:58:13.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:58:12 smithi107 ceph-mon[33178]: pgmap v1580: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:58:13.049 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:58:12 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:58:13.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:58:12 smithi057 ceph-mon[29805]: pgmap v1580: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:58:13.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:58:12 smithi057 ceph-mon[40588]: pgmap v1580: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:58:14.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:58:13 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:58:14.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:58:13 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:58:14.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:58:13 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:58:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:58:14.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:58:13 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:58:15.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:58:14 smithi057 ceph-mon[29805]: pgmap v1581: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:58:15.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:58:14 smithi057 ceph-mon[40588]: pgmap v1581: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:58:15.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:58:14 smithi107 ceph-mon[33178]: pgmap v1581: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:58:17.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:58:16 smithi057 ceph-mon[29805]: pgmap v1582: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:58:17.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:58:16 smithi057 ceph-mon[40588]: pgmap v1582: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:58:17.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:58:16 smithi107 ceph-mon[33178]: pgmap v1582: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:58:19.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:58:18 smithi057 ceph-mon[29805]: pgmap v1583: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:58:19.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:58:18 smithi057 ceph-mon[40588]: pgmap v1583: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:58:19.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:58:18 smithi107 ceph-mon[33178]: pgmap v1583: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:58:21.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:58:20 smithi057 ceph-mon[29805]: pgmap v1584: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:58:21.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:58:20 smithi057 ceph-mon[40588]: pgmap v1584: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:58:21.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:58:20 smithi107 ceph-mon[33178]: pgmap v1584: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:58:22.966 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:58:22 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:58:23.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:58:22 smithi057 ceph-mon[29805]: pgmap v1585: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-06-11T13:58:23.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:58:22 smithi057 ceph-mon[40588]: pgmap v1585: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-06-11T13:58:23.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:58:22 smithi107 ceph-mon[33178]: pgmap v1585: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-06-11T13:58:24.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:58:23 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:58:24.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:58:23 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:58:24.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:58:23 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:58:24.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:58:23 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:58:24.037 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:58:23 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:58:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:58:24.301 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:58:23 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:58:24.301 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:58:23 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:58:25.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:58:24 smithi057 ceph-mon[29805]: pgmap v1586: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 0 B/s wr, 7 op/s 2024-06-11T13:58:25.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:58:24 smithi057 ceph-mon[40588]: pgmap v1586: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 0 B/s wr, 7 op/s 2024-06-11T13:58:25.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:58:24 smithi107 ceph-mon[33178]: pgmap v1586: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 0 B/s wr, 7 op/s 2024-06-11T13:58:27.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:58:26 smithi057 ceph-mon[29805]: pgmap v1587: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 0 B/s wr, 7 op/s 2024-06-11T13:58:27.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:58:26 smithi057 ceph-mon[40588]: pgmap v1587: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 0 B/s wr, 7 op/s 2024-06-11T13:58:27.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:58:26 smithi107 ceph-mon[33178]: pgmap v1587: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 0 B/s wr, 7 op/s 2024-06-11T13:58:29.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:58:28 smithi057 ceph-mon[40588]: pgmap v1588: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-11T13:58:29.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:58:28 smithi057 ceph-mon[29805]: pgmap v1588: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-11T13:58:29.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:58:28 smithi107 ceph-mon[33178]: pgmap v1588: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-11T13:58:29.961 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.962 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.962 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.962 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.962 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.962 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.962 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.962 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.962 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.962 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.962 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.962 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.963 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.963 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.963 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.963 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.963 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.963 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.963 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.963 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.963 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.963 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.963 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.963 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.963 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.964 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.964 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.964 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.964 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.964 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.964 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.964 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.964 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.964 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.964 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.964 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.964 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.965 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.965 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.965 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.965 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.965 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.965 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.965 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.965 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.965 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.965 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.965 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.965 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.965 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.966 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.966 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.966 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.966 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.966 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.966 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.966 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.966 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.966 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.966 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.966 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.966 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.966 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.967 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.967 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.967 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.967 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.967 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.967 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.967 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.967 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.967 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.967 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.967 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.967 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.967 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.967 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.968 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.968 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.968 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.968 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.968 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.968 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.968 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.968 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.968 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.968 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.968 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.968 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.968 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.968 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.969 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.969 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.969 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.969 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.969 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.969 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.969 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:29.969 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T13:58:31.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:58:31 smithi057 ceph-mon[29805]: pgmap v1589: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-11T13:58:31.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:58:31 smithi057 ceph-mon[40588]: pgmap v1589: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-11T13:58:31.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:58:31 smithi107 ceph-mon[33178]: pgmap v1589: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-11T13:58:33.007 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:58:32 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:58:33.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:58:33 smithi057 ceph-mon[40588]: pgmap v1590: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-11T13:58:33.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:58:33 smithi057 ceph-mon[29805]: pgmap v1590: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-11T13:58:33.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:58:33 smithi107 ceph-mon[33178]: pgmap v1590: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-06-11T13:58:34.012 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:58:33 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:58:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:58:34.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:58:34 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:58:34.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:58:34 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:58:34.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:58:34 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:58:35.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:58:35 smithi057 ceph-mon[29805]: pgmap v1591: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 53 op/s 2024-06-11T13:58:35.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:58:35 smithi057 ceph-mon[40588]: pgmap v1591: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 53 op/s 2024-06-11T13:58:35.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:58:35 smithi107 ceph-mon[33178]: pgmap v1591: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 53 op/s 2024-06-11T13:58:37.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:58:37 smithi057 ceph-mon[29805]: pgmap v1592: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 53 op/s 2024-06-11T13:58:37.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:58:37 smithi057 ceph-mon[40588]: pgmap v1592: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 53 op/s 2024-06-11T13:58:37.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:58:37 smithi107 ceph-mon[33178]: pgmap v1592: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 53 op/s 2024-06-11T13:58:39.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:58:39 smithi057 ceph-mon[29805]: pgmap v1593: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 54 op/s 2024-06-11T13:58:39.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:58:39 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:58:39.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:58:39 smithi057 ceph-mon[40588]: pgmap v1593: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 54 op/s 2024-06-11T13:58:39.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:58:39 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:58:39.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:58:39 smithi107 ceph-mon[33178]: pgmap v1593: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 54 op/s 2024-06-11T13:58:39.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:58:39 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:58:41.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:58:41 smithi107 ceph-mon[33178]: pgmap v1594: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:58:41.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:58:41 smithi057 ceph-mon[29805]: pgmap v1594: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:58:41.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:58:41 smithi057 ceph-mon[40588]: pgmap v1594: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:58:42.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:58:42 smithi057 ceph-mon[29805]: pgmap v1595: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:58:42.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:58:42 smithi057 ceph-mon[40588]: pgmap v1595: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:58:42.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:58:42 smithi107 ceph-mon[33178]: pgmap v1595: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:58:43.049 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:58:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:58:43.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:58:43 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:58:43.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:58:43 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:58:43.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:58:43 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:58:44.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:58:43 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:58:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:58:44.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:58:44 smithi057 ceph-mon[29805]: pgmap v1596: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:58:44.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:58:44 smithi057 ceph-mon[40588]: pgmap v1596: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:58:44.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:58:44 smithi107 ceph-mon[33178]: pgmap v1596: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:58:47.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:58:46 smithi057 ceph-mon[29805]: pgmap v1597: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:58:47.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:58:46 smithi057 ceph-mon[40588]: pgmap v1597: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:58:47.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:58:46 smithi107 ceph-mon[33178]: pgmap v1597: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:58:49.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:58:48 smithi057 ceph-mon[40588]: pgmap v1598: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:58:49.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:58:48 smithi057 ceph-mon[29805]: pgmap v1598: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:58:49.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:58:48 smithi107 ceph-mon[33178]: pgmap v1598: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:58:50.186 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:58:49 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:58:50.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:58:49 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:58:50.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:58:49 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:58:51.258 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:58:50 smithi107 ceph-mon[33178]: pgmap v1599: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:58:51.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:58:50 smithi057 ceph-mon[29805]: pgmap v1599: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:58:51.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:58:50 smithi057 ceph-mon[40588]: pgmap v1599: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:58:52.945 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:58:52 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:58:53.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:58:52 smithi057 ceph-mon[29805]: pgmap v1600: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:58:53.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:58:52 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:58:53.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:58:52 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:58:53.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:58:52 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:58:53.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:58:52 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:58:53.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:58:52 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:58:53.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:58:52 smithi057 ceph-mon[40588]: pgmap v1600: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:58:53.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:58:52 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:58:53.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:58:52 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:58:53.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:58:52 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:58:53.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:58:52 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:58:53.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:58:52 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:58:53.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:58:52 smithi107 ceph-mon[33178]: pgmap v1600: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:58:53.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:58:52 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:58:53.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:58:52 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:58:53.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:58:52 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:58:53.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:58:52 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:58:53.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:58:52 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:58:54.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:58:53 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:58:54.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:58:53 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:58:54.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:58:53 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:58:54.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:58:53 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:58:54.037 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:58:53 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:58:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:58:54.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:58:53 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:58:54.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:58:53 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:58:55.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:58:54 smithi057 ceph-mon[29805]: pgmap v1601: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-11T13:58:55.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:58:54 smithi057 ceph-mon[40588]: pgmap v1601: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-11T13:58:55.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:58:54 smithi107 ceph-mon[33178]: pgmap v1601: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-11T13:58:57.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:58:56 smithi057 ceph-mon[29805]: pgmap v1602: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-11T13:58:57.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:58:56 smithi057 ceph-mon[40588]: pgmap v1602: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-11T13:58:57.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:58:56 smithi107 ceph-mon[33178]: pgmap v1602: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-11T13:58:59.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:58:58 smithi057 ceph-mon[40588]: pgmap v1603: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:58:59.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:58:58 smithi057 ceph-mon[29805]: pgmap v1603: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:58:59.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:58:58 smithi107 ceph-mon[33178]: pgmap v1603: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:59:01.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:00 smithi057 ceph-mon[29805]: pgmap v1604: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-11T13:59:01.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:00 smithi057 ceph-mon[40588]: pgmap v1604: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-11T13:59:01.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:00 smithi107 ceph-mon[33178]: pgmap v1604: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-11T13:59:03.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:02 smithi107 ceph-mon[33178]: pgmap v1605: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:59:03.049 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:59:02 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:59:03.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:02 smithi057 ceph-mon[29805]: pgmap v1605: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:59:03.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:02 smithi057 ceph-mon[40588]: pgmap v1605: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:59:04.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:03 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:59:04.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:03 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:59:04.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:59:03 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:59:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:59:04.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:03 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:59:05.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:04 smithi057 ceph-mon[29805]: pgmap v1606: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:05.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:04 smithi057 ceph-mon[40588]: pgmap v1606: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:05.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:04 smithi107 ceph-mon[33178]: pgmap v1606: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:07.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:06 smithi057 ceph-mon[29805]: pgmap v1607: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:07.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:06 smithi057 ceph-mon[40588]: pgmap v1607: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:07.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:06 smithi107 ceph-mon[33178]: pgmap v1607: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:09.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:08 smithi057 ceph-mon[29805]: pgmap v1608: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:59:09.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:08 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:59:09.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:08 smithi057 ceph-mon[40588]: pgmap v1608: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:59:09.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:08 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:59:09.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:08 smithi107 ceph-mon[33178]: pgmap v1608: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:59:09.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:08 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:59:11.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:10 smithi057 ceph-mon[29805]: pgmap v1609: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:11.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:10 smithi057 ceph-mon[40588]: pgmap v1609: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:11.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:10 smithi107 ceph-mon[33178]: pgmap v1609: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:13.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:12 smithi107 ceph-mon[33178]: pgmap v1610: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:59:13.049 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:59:12 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:59:13.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:12 smithi057 ceph-mon[29805]: pgmap v1610: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:59:13.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:12 smithi057 ceph-mon[40588]: pgmap v1610: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:59:13.986 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:59:13 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:59:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:59:14.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:13 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:59:14.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:13 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:59:14.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:13 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:59:15.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:14 smithi057 ceph-mon[29805]: pgmap v1611: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:15.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:14 smithi057 ceph-mon[40588]: pgmap v1611: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:15.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:14 smithi107 ceph-mon[33178]: pgmap v1611: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:17.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:17 smithi057 ceph-mon[29805]: pgmap v1612: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:17.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:17 smithi057 ceph-mon[40588]: pgmap v1612: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:17.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:17 smithi107 ceph-mon[33178]: pgmap v1612: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:19.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:19 smithi057 ceph-mon[29805]: pgmap v1613: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:59:19.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:19 smithi057 ceph-mon[40588]: pgmap v1613: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:59:19.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:19 smithi107 ceph-mon[33178]: pgmap v1613: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:59:21.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:21 smithi057 ceph-mon[29805]: pgmap v1614: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:21.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:21 smithi057 ceph-mon[40588]: pgmap v1614: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:21.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:21 smithi107 ceph-mon[33178]: pgmap v1614: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:23.023 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:59:22 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:59:23.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:23 smithi057 ceph-mon[29805]: pgmap v1615: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:59:23.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:23 smithi057 ceph-mon[40588]: pgmap v1615: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:59:23.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:23 smithi107 ceph-mon[33178]: pgmap v1615: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:59:24.031 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:59:23 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:59:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:59:24.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:24 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:59:24.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:24 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:59:24.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:24 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:59:24.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:24 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:59:24.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:24 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:59:24.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:24 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:59:25.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:25 smithi057 ceph-mon[40588]: pgmap v1616: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:25.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:25 smithi057 ceph-mon[29805]: pgmap v1616: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:25.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:25 smithi107 ceph-mon[33178]: pgmap v1616: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:27.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:27 smithi107 ceph-mon[33178]: pgmap v1617: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:59:27.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:27 smithi057 ceph-mon[29805]: pgmap v1617: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:59:27.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:27 smithi057 ceph-mon[40588]: pgmap v1617: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T13:59:29.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:29 smithi107 ceph-mon[33178]: pgmap v1618: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:59:29.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:29 smithi057 ceph-mon[29805]: pgmap v1618: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:59:29.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:29 smithi057 ceph-mon[40588]: pgmap v1618: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:59:31.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:31 smithi057 ceph-mon[29805]: pgmap v1619: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:31.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:31 smithi057 ceph-mon[40588]: pgmap v1619: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:31.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:31 smithi107 ceph-mon[33178]: pgmap v1619: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:32.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:32 smithi057 ceph-mon[40588]: pgmap v1620: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:59:32.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:32 smithi057 ceph-mon[29805]: pgmap v1620: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:59:32.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:32 smithi107 ceph-mon[33178]: pgmap v1620: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:59:33.049 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:59:32 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:59:33.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:33 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:59:33.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:33 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:59:33.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:33 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:59:34.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:59:33 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:59:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:59:34.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:34 smithi057 ceph-mon[29805]: pgmap v1621: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:34.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:34 smithi057 ceph-mon[40588]: pgmap v1621: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:34.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:34 smithi107 ceph-mon[33178]: pgmap v1621: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:37.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:36 smithi057 ceph-mon[40588]: pgmap v1622: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:37.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:36 smithi057 ceph-mon[29805]: pgmap v1622: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:37.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:36 smithi107 ceph-mon[33178]: pgmap v1622: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:39.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:38 smithi057 ceph-mon[40588]: pgmap v1623: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:59:39.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:38 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:59:39.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:38 smithi057 ceph-mon[29805]: pgmap v1623: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:59:39.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:38 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:59:39.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:38 smithi107 ceph-mon[33178]: pgmap v1623: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:59:39.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:38 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:59:41.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:40 smithi057 ceph-mon[40588]: pgmap v1624: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:41.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:40 smithi057 ceph-mon[29805]: pgmap v1624: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:41.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:40 smithi107 ceph-mon[33178]: pgmap v1624: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:43.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:42 smithi107 ceph-mon[33178]: pgmap v1625: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:59:43.050 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:59:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:59:43.050 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 13:59:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=cleanup t=2024-06-11T13:59:42.717703188Z level=info msg="Completed cleanup jobs" duration=4.562554ms 2024-06-11T13:59:43.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:42 smithi057 ceph-mon[29805]: pgmap v1625: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:59:43.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:42 smithi057 ceph-mon[40588]: pgmap v1625: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:59:44.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:43 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:59:44.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:43 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:59:44.037 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:59:43 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:59:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:59:44.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:43 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:59:45.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:44 smithi057 ceph-mon[29805]: pgmap v1626: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:45.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:44 smithi057 ceph-mon[40588]: pgmap v1626: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:45.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:44 smithi107 ceph-mon[33178]: pgmap v1626: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:47.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:46 smithi057 ceph-mon[29805]: pgmap v1627: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:47.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:46 smithi057 ceph-mon[40588]: pgmap v1627: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:47.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:46 smithi107 ceph-mon[33178]: pgmap v1627: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:49.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:49 smithi057 ceph-mon[29805]: pgmap v1628: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:59:49.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:49 smithi057 ceph-mon[40588]: pgmap v1628: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:59:49.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:49 smithi107 ceph-mon[33178]: pgmap v1628: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:59:51.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:51 smithi057 ceph-mon[29805]: pgmap v1629: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:51.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:51 smithi057 ceph-mon[40588]: pgmap v1629: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:51.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:51 smithi107 ceph-mon[33178]: pgmap v1629: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:53.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:53 smithi107 ceph-mon[33178]: pgmap v1630: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:59:53.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:53 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:59:53.050 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 13:59:52 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T13:59:53.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:53 smithi057 ceph-mon[29805]: pgmap v1630: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:59:53.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:53 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:59:53.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:53 smithi057 ceph-mon[40588]: pgmap v1630: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:59:53.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:53 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T13:59:54.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 13:59:53 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:13:59:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T13:59:54.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:54 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:59:54.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:54 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:59:54.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:54 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:59:54.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:54 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:59:54.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:54 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:59:54.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:54 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:59:54.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:54 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:59:54.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:54 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:59:54.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:54 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:59:54.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:54 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:59:54.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:54 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:59:54.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:54 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:59:54.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:54 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:59:54.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:54 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:59:54.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:54 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:59:54.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:54 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:59:54.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:54 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:59:54.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:54 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:59:54.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:54 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T13:59:54.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:54 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T13:59:54.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:54 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:59:54.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:54 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:59:54.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:54 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:59:54.538 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:54 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:59:54.538 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:54 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T13:59:54.538 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:54 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T13:59:54.538 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:54 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T13:59:55.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:55 smithi107 ceph-mon[33178]: pgmap v1631: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:55.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:55 smithi057 ceph-mon[29805]: pgmap v1631: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:55.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:55 smithi057 ceph-mon[40588]: pgmap v1631: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:57.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:57 smithi057 ceph-mon[29805]: pgmap v1632: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:57.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:57 smithi057 ceph-mon[40588]: pgmap v1632: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:57.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:57 smithi107 ceph-mon[33178]: pgmap v1632: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T13:59:59.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 13:59:59 smithi057 ceph-mon[29805]: pgmap v1633: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:59:59.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 13:59:59 smithi057 ceph-mon[40588]: pgmap v1633: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T13:59:59.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 13:59:59 smithi107 ceph-mon[33178]: pgmap v1633: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:00:00.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:00:00 smithi057 ceph-mon[29805]: pgmap v1634: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:00:00.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:00:00 smithi057 ceph-mon[29805]: overall HEALTH_WARN 4 pool(s) do not have an application enabled 2024-06-11T14:00:00.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:00:00 smithi057 ceph-mon[40588]: pgmap v1634: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:00:00.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:00:00 smithi057 ceph-mon[40588]: overall HEALTH_WARN 4 pool(s) do not have an application enabled 2024-06-11T14:00:00.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:00:00 smithi107 ceph-mon[33178]: pgmap v1634: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:00:00.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:00:00 smithi107 ceph-mon[33178]: overall HEALTH_WARN 4 pool(s) do not have an application enabled 2024-06-11T14:00:03.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:00:02 smithi107 ceph-mon[33178]: pgmap v1635: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:00:03.050 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 14:00:02 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T14:00:03.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:00:02 smithi057 ceph-mon[29805]: pgmap v1635: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:00:03.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:00:02 smithi057 ceph-mon[40588]: pgmap v1635: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:00:04.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:00:03 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:00:04.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:00:03 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:00:04.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 14:00:03 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:14:00:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T14:00:04.261 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:00:03 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:00:05.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:00:04 smithi057 ceph-mon[29805]: pgmap v1636: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:00:05.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:00:04 smithi057 ceph-mon[40588]: pgmap v1636: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:00:05.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:00:04 smithi107 ceph-mon[33178]: pgmap v1636: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:00:07.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:00:06 smithi057 ceph-mon[29805]: pgmap v1637: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:00:07.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:00:06 smithi057 ceph-mon[40588]: pgmap v1637: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:00:07.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:00:06 smithi107 ceph-mon[33178]: pgmap v1637: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:00:09.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:00:08 smithi057 ceph-mon[29805]: pgmap v1638: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:00:09.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:00:08 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:00:09.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:00:08 smithi057 ceph-mon[40588]: pgmap v1638: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:00:09.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:00:08 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:00:09.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:00:08 smithi107 ceph-mon[33178]: pgmap v1638: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:00:09.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:00:08 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:00:09.974 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.974 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.974 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.975 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.975 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.975 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.975 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.975 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.975 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.975 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.975 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.975 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.975 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.975 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.975 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.976 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.976 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.976 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.976 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.976 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.976 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.976 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.976 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.976 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.976 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.976 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.976 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.976 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.977 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.977 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.977 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.977 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.977 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.977 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.977 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.977 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.977 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.977 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.977 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.977 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.977 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.978 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.978 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.978 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.978 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.978 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.978 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.978 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.978 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.978 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.978 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.978 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.978 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.978 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.979 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.979 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.979 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.979 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.979 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.979 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.979 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.979 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.979 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.979 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.979 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.979 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.979 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.980 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.980 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.980 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.980 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.980 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.980 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.980 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.980 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.980 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.980 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.980 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.980 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.981 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.981 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.981 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.981 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.981 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.981 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.981 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.981 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.981 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.981 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.981 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.981 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.981 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.982 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.982 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.982 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.982 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.982 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.982 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.982 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:09.982 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:00:11.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:00:10 smithi057 ceph-mon[29805]: pgmap v1639: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:00:11.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:00:10 smithi057 ceph-mon[40588]: pgmap v1639: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:00:11.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:00:10 smithi107 ceph-mon[33178]: pgmap v1639: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:00:13.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:00:13 smithi107 ceph-mon[33178]: pgmap v1640: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:00:13.050 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 14:00:12 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T14:00:13.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:00:12 smithi057 ceph-mon[29805]: pgmap v1640: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:00:13.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:00:13 smithi057 ceph-mon[40588]: pgmap v1640: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:00:14.018 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 14:00:13 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:14:00:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T14:00:14.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:00:14 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:00:14.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:00:14 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:00:14.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:00:14 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:00:15.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:00:15 smithi057 ceph-mon[29805]: pgmap v1641: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:00:15.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:00:15 smithi057 ceph-mon[40588]: pgmap v1641: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:00:15.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:00:15 smithi107 ceph-mon[33178]: pgmap v1641: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:00:17.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:00:17 smithi107 ceph-mon[33178]: pgmap v1642: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:00:17.538 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:00:17 smithi057 ceph-mon[29805]: pgmap v1642: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:00:17.538 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:00:17 smithi057 ceph-mon[40588]: pgmap v1642: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:00:19.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:00:19 smithi107 ceph-mon[33178]: pgmap v1643: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:00:19.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:00:19 smithi057 ceph-mon[40588]: pgmap v1643: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:00:19.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:00:19 smithi057 ceph-mon[29805]: pgmap v1643: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:00:21.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:00:21 smithi057 ceph-mon[29805]: pgmap v1644: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:00:21.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:00:21 smithi057 ceph-mon[40588]: pgmap v1644: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:00:21.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:00:21 smithi107 ceph-mon[33178]: pgmap v1644: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:00:23.059 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 14:00:22 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T14:00:23.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:00:23 smithi057 ceph-mon[29805]: pgmap v1645: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:00:23.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:00:23 smithi057 ceph-mon[40588]: pgmap v1645: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:00:23.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:00:23 smithi107 ceph-mon[33178]: pgmap v1645: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:00:24.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 14:00:23 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:14:00:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T14:00:24.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:00:24 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:00:24.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:00:24 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:00:24.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:00:24 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:00:24.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:00:24 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:00:24.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:00:24 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:00:24.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:00:24 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:00:25.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:00:25 smithi057 ceph-mon[29805]: pgmap v1646: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:00:25.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:00:25 smithi057 ceph-mon[40588]: pgmap v1646: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:00:25.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:00:25 smithi107 ceph-mon[33178]: pgmap v1646: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:00:26.532 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:00:26 smithi057 ceph-mon[40588]: pgmap v1647: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:00:26.532 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:00:26 smithi057 ceph-mon[29805]: pgmap v1647: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:00:26.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:00:26 smithi107 ceph-mon[33178]: pgmap v1647: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:00:29.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:00:28 smithi057 ceph-mon[29805]: pgmap v1648: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:00:29.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:00:28 smithi057 ceph-mon[40588]: pgmap v1648: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:00:29.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:00:28 smithi107 ceph-mon[33178]: pgmap v1648: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:00:31.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:00:30 smithi057 ceph-mon[29805]: pgmap v1649: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:00:31.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:00:30 smithi057 ceph-mon[40588]: pgmap v1649: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:00:31.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:00:30 smithi107 ceph-mon[33178]: pgmap v1649: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:00:33.102 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:00:32 smithi107 ceph-mon[33178]: pgmap v1650: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:00:33.102 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 14:00:32 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T14:00:33.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:00:32 smithi057 ceph-mon[29805]: pgmap v1650: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:00:33.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:00:32 smithi057 ceph-mon[40588]: pgmap v1650: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:00:34.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:00:33 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:00:34.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:00:33 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:00:34.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 14:00:33 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:14:00:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T14:00:34.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:00:33 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:00:35.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:00:34 smithi057 ceph-mon[29805]: pgmap v1651: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:00:35.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:00:34 smithi057 ceph-mon[40588]: pgmap v1651: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:00:35.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:00:34 smithi107 ceph-mon[33178]: pgmap v1651: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:00:37.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:00:36 smithi057 ceph-mon[29805]: pgmap v1652: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:00:37.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:00:36 smithi057 ceph-mon[40588]: pgmap v1652: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:00:37.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:00:36 smithi107 ceph-mon[33178]: pgmap v1652: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:00:39.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:00:38 smithi057 ceph-mon[29805]: pgmap v1653: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:00:39.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:00:38 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:00:39.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:00:38 smithi057 ceph-mon[40588]: pgmap v1653: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:00:39.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:00:38 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:00:39.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:00:38 smithi107 ceph-mon[33178]: pgmap v1653: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:00:39.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:00:38 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:00:41.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:00:41 smithi057 ceph-mon[29805]: pgmap v1654: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:00:41.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:00:41 smithi057 ceph-mon[40588]: pgmap v1654: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:00:41.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:00:41 smithi107 ceph-mon[33178]: pgmap v1654: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:00:43.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:00:43 smithi057 ceph-mon[29805]: pgmap v1655: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:00:43.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:00:43 smithi057 ceph-mon[40588]: pgmap v1655: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:00:43.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:00:43 smithi107 ceph-mon[33178]: pgmap v1655: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:00:43.299 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 14:00:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T14:00:44.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 14:00:43 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:14:00:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T14:00:44.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:00:44 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:00:44.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:00:44 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:00:44.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:00:44 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:00:45.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:00:45 smithi107 ceph-mon[33178]: pgmap v1656: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:00:45.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:00:45 smithi057 ceph-mon[29805]: pgmap v1656: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:00:45.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:00:45 smithi057 ceph-mon[40588]: pgmap v1656: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:00:47.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:00:47 smithi057 ceph-mon[29805]: pgmap v1657: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:00:47.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:00:47 smithi057 ceph-mon[40588]: pgmap v1657: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:00:47.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:00:47 smithi107 ceph-mon[33178]: pgmap v1657: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:00:49.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:00:49 smithi057 ceph-mon[29805]: pgmap v1658: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:00:49.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:00:49 smithi057 ceph-mon[40588]: pgmap v1658: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:00:49.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:00:49 smithi107 ceph-mon[33178]: pgmap v1658: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:00:50.538 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:00:50 smithi057 ceph-mon[29805]: pgmap v1659: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:00:50.538 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:00:50 smithi057 ceph-mon[40588]: pgmap v1659: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:00:50.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:00:50 smithi107 ceph-mon[33178]: pgmap v1659: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:00:53.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:00:52 smithi057 ceph-mon[29805]: pgmap v1660: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:00:53.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:00:52 smithi057 ceph-mon[40588]: pgmap v1660: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:00:53.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:00:52 smithi107 ceph-mon[33178]: pgmap v1660: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:00:53.299 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 14:00:52 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T14:00:54.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 14:00:53 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:14:00:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T14:00:54.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:00:53 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:00:54.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:00:53 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:00:54.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:00:53 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T14:00:54.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:00:53 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:00:54.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:00:53 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:00:54.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:00:53 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T14:00:54.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:00:53 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:00:54.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:00:53 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:00:54.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:00:53 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T14:00:55.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:00:54 smithi057 ceph-mon[40588]: pgmap v1661: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:00:55.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:00:54 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T14:00:55.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:00:54 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T14:00:55.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:00:54 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T14:00:55.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:00:54 smithi057 ceph-mon[29805]: pgmap v1661: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:00:55.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:00:54 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T14:00:55.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:00:54 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T14:00:55.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:00:54 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T14:00:55.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:00:54 smithi107 ceph-mon[33178]: pgmap v1661: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:00:55.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:00:54 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T14:00:55.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:00:54 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T14:00:55.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:00:54 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T14:00:57.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:00:56 smithi057 ceph-mon[40588]: pgmap v1662: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:00:57.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:00:56 smithi057 ceph-mon[29805]: pgmap v1662: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:00:57.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:00:56 smithi107 ceph-mon[33178]: pgmap v1662: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:00:59.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:00:58 smithi057 ceph-mon[29805]: pgmap v1663: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:00:59.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:00:58 smithi057 ceph-mon[40588]: pgmap v1663: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:00:59.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:00:58 smithi107 ceph-mon[33178]: pgmap v1663: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:01:01.009 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 handler_error: Running main() from gmock_main.cc 2024-06-11T14:01:01.009 INFO:tasks.workunit.client.0.smithi057.stdout: handler_error: [==========] Running 1 test from 1 test suite. 2024-06-11T14:01:01.009 INFO:tasks.workunit.client.0.smithi057.stdout: handler_error: [----------] Global test environment set-up. 2024-06-11T14:01:01.009 INFO:tasks.workunit.client.0.smithi057.stdout: handler_error: [----------] 1 test from neocls_handler_error 2024-06-11T14:01:01.009 INFO:tasks.workunit.client.0.smithi057.stdout: handler_error: [ RUN ] neocls_handler_error.test_handler_error 2024-06-11T14:01:01.009 INFO:tasks.workunit.client.0.smithi057.stdout: handler_error: [ OK ] neocls_handler_error.test_handler_error (1816983 ms) 2024-06-11T14:01:01.009 INFO:tasks.workunit.client.0.smithi057.stdout: handler_error: [----------] 1 test from neocls_handler_error (1816983 ms total) 2024-06-11T14:01:01.010 INFO:tasks.workunit.client.0.smithi057.stdout: handler_error: 2024-06-11T14:01:01.010 INFO:tasks.workunit.client.0.smithi057.stdout: handler_error: [----------] Global test environment tear-down 2024-06-11T14:01:01.010 INFO:tasks.workunit.client.0.smithi057.stdout: handler_error: [==========] 1 test from 1 test suite ran. (1816984 ms total) 2024-06-11T14:01:01.010 INFO:tasks.workunit.client.0.smithi057.stdout: handler_error: [ PASSED ] 1 test. 2024-06-11T14:01:01.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:01 smithi057 ceph-mon[29805]: pgmap v1664: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:01:01.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:01 smithi057 ceph-mon[40588]: pgmap v1664: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:01:01.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:01 smithi107 ceph-mon[33178]: pgmap v1664: 260 pgs: 260 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:01:02.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:02 smithi057 ceph-mon[29805]: osdmap e766: 8 total, 8 up, 8 in 2024-06-11T14:01:02.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:02 smithi057 ceph-mon[40588]: osdmap e766: 8 total, 8 up, 8 in 2024-06-11T14:01:02.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:02 smithi107 ceph-mon[33178]: osdmap e766: 8 total, 8 up, 8 in 2024-06-11T14:01:02.983 INFO:tasks.workunit.client.0.smithi057.stdout: sec 2024-06-11T14:01:02.983 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.983 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.983 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.983 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.983 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.983 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.983 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.983 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.984 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.984 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.984 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.984 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.984 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.984 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.984 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.984 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.984 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.984 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.984 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.985 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.985 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.985 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.985 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.985 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.985 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.985 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.985 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.986 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.986 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.986 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.986 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.986 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.986 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.986 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.986 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.986 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.986 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.986 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.986 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.986 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.987 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.987 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.987 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.987 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.987 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.987 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.987 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.987 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.987 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.987 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.987 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.987 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 1 sec 2024-06-11T14:01:02.987 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: started 512 aios 2024-06-11T14:01:02.988 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x56407869d660 2024-06-11T14:01:02.988 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a64d70 2024-06-11T14:01:02.988 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a65790 2024-06-11T14:01:02.988 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6a5c0 2024-06-11T14:01:02.988 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6b050 2024-06-11T14:01:02.988 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6a750 2024-06-11T14:01:02.988 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6b5f0 2024-06-11T14:01:02.988 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6a8c0 2024-06-11T14:01:02.988 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6bb00 2024-06-11T14:01:02.988 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6aa30 2024-06-11T14:01:02.988 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6c0c0 2024-06-11T14:01:02.988 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6bc10 2024-06-11T14:01:02.988 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6b760 2024-06-11T14:01:02.989 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6b870 2024-06-11T14:01:02.989 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6b980 2024-06-11T14:01:02.989 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a65260 2024-06-11T14:01:02.989 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6c1d0 2024-06-11T14:01:02.989 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6c2e0 2024-06-11T14:01:02.989 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6c8a0 2024-06-11T14:01:02.989 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6c3f0 2024-06-11T14:01:02.989 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6bd20 2024-06-11T14:01:02.989 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x5640788c63b0 2024-06-11T14:01:02.989 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6be30 2024-06-11T14:01:02.989 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6bf40 2024-06-11T14:01:02.989 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6c9b0 2024-06-11T14:01:02.989 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6cea0 2024-06-11T14:01:02.989 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6cfb0 2024-06-11T14:01:02.990 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6cb30 2024-06-11T14:01:02.990 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6cc80 2024-06-11T14:01:02.990 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6d570 2024-06-11T14:01:02.990 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6d0c0 2024-06-11T14:01:02.990 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6c500 2024-06-11T14:01:02.990 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6c630 2024-06-11T14:01:02.990 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6c740 2024-06-11T14:01:02.990 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6db30 2024-06-11T14:01:02.990 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6d680 2024-06-11T14:01:02.990 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6d1d0 2024-06-11T14:01:02.990 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6d2e0 2024-06-11T14:01:02.991 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6d450 2024-06-11T14:01:02.991 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6e110 2024-06-11T14:01:02.991 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6dc60 2024-06-11T14:01:02.991 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6e220 2024-06-11T14:01:02.991 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6e3b0 2024-06-11T14:01:02.991 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6e970 2024-06-11T14:01:02.991 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6dd70 2024-06-11T14:01:02.991 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6de80 2024-06-11T14:01:02.991 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6dfd0 2024-06-11T14:01:02.991 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6eac0 2024-06-11T14:01:02.991 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6e4c0 2024-06-11T14:01:02.991 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6e5f0 2024-06-11T14:01:02.992 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6e740 2024-06-11T14:01:02.992 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6ebd0 2024-06-11T14:01:02.992 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6f080 2024-06-11T14:01:02.992 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6ece0 2024-06-11T14:01:02.992 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6edf0 2024-06-11T14:01:02.992 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6ef40 2024-06-11T14:01:02.992 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6f660 2024-06-11T14:01:02.992 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6f1b0 2024-06-11T14:01:02.992 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6d790 2024-06-11T14:01:02.992 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6d8e0 2024-06-11T14:01:02.992 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6f770 2024-06-11T14:01:02.992 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6fc20 2024-06-11T14:01:02.993 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6fd30 2024-06-11T14:01:02.993 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a70310 2024-06-11T14:01:02.993 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6f880 2024-06-11T14:01:02.993 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6f9d0 2024-06-11T14:01:02.993 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6fb00 2024-06-11T14:01:02.993 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a70440 2024-06-11T14:01:02.993 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6fe60 2024-06-11T14:01:02.993 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6ff90 2024-06-11T14:01:02.993 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a700e0 2024-06-11T14:01:02.993 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a701f0 2024-06-11T14:01:02.993 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a70a40 2024-06-11T14:01:02.993 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a70590 2024-06-11T14:01:02.993 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a706e0 2024-06-11T14:01:02.994 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a707f0 2024-06-11T14:01:02.994 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a70b50 2024-06-11T14:01:02.994 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a71020 2024-06-11T14:01:02.994 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a71150 2024-06-11T14:01:02.994 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a70920 2024-06-11T14:01:02.994 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a71260 2024-06-11T14:01:02.994 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6f2c0 2024-06-11T14:01:02.994 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6f410 2024-06-11T14:01:02.994 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a6f520 2024-06-11T14:01:02.994 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a71860 2024-06-11T14:01:02.994 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a713b0 2024-06-11T14:01:02.994 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a70c80 2024-06-11T14:01:02.994 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a70dd0 2024-06-11T14:01:02.994 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a70ee0 2024-06-11T14:01:02.994 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a71e40 2024-06-11T14:01:02.995 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a71990 2024-06-11T14:01:02.995 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a71ac0 2024-06-11T14:01:02.995 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a71c10 2024-06-11T14:01:02.995 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a71f50 2024-06-11T14:01:02.995 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a72440 2024-06-11T14:01:02.995 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a72550 2024-06-11T14:01:02.995 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a714c0 2024-06-11T14:01:02.995 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a715f0 2024-06-11T14:01:02.995 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a71720 2024-06-11T14:01:02.995 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a72680 2024-06-11T14:01:02.995 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a72b50 2024-06-11T14:01:02.995 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a72cc0 2024-06-11T14:01:02.995 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a732c0 2024-06-11T14:01:02.995 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a727b0 2024-06-11T14:01:02.996 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a72900 2024-06-11T14:01:02.996 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a72a30 2024-06-11T14:01:02.996 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a73410 2024-06-11T14:01:02.996 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a72e10 2024-06-11T14:01:02.996 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a72f60 2024-06-11T14:01:02.996 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a73070 2024-06-11T14:01:02.996 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a731a0 2024-06-11T14:01:02.996 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a739f0 2024-06-11T14:01:02.996 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a73540 2024-06-11T14:01:02.996 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a73690 2024-06-11T14:01:02.996 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a71d20 2024-06-11T14:01:02.996 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a737e0 2024-06-11T14:01:02.996 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a73fd0 2024-06-11T14:01:02.996 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a73b20 2024-06-11T14:01:02.996 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a73c70 2024-06-11T14:01:02.997 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a73dc0 2024-06-11T14:01:02.997 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a74140 2024-06-11T14:01:02.997 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a74660 2024-06-11T14:01:02.997 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a74790 2024-06-11T14:01:02.997 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a74270 2024-06-11T14:01:02.997 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a74380 2024-06-11T14:01:02.997 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a74490 2024-06-11T14:01:02.997 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a748e0 2024-06-11T14:01:02.997 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a74ae0 2024-06-11T14:01:02.997 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a750c0 2024-06-11T14:01:02.997 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a74c10 2024-06-11T14:01:02.997 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a74d60 2024-06-11T14:01:02.997 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a74e70 2024-06-11T14:01:02.997 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a751d0 2024-06-11T14:01:02.998 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a756a0 2024-06-11T14:01:02.998 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a757d0 2024-06-11T14:01:02.998 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a75d90 2024-06-11T14:01:02.998 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a75300 2024-06-11T14:01:02.998 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a75430 2024-06-11T14:01:02.998 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a75560 2024-06-11T14:01:02.998 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a75ee0 2024-06-11T14:01:02.998 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a758e0 2024-06-11T14:01:02.998 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a75a10 2024-06-11T14:01:02.998 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a75b40 2024-06-11T14:01:02.998 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a76030 2024-06-11T14:01:02.998 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a76180 2024-06-11T14:01:02.998 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a76780 2024-06-11T14:01:02.999 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a762d0 2024-06-11T14:01:02.999 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a76400 2024-06-11T14:01:02.999 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a75c50 2024-06-11T14:01:02.999 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a76510 2024-06-11T14:01:02.999 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a76640 2024-06-11T14:01:02.999 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a76890 2024-06-11T14:01:02.999 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a76d80 2024-06-11T14:01:02.999 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a76ed0 2024-06-11T14:01:02.999 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a774d0 2024-06-11T14:01:02.999 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a769e0 2024-06-11T14:01:02.999 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a76b30 2024-06-11T14:01:02.999 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a76c60 2024-06-11T14:01:02.999 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a775e0 2024-06-11T14:01:02.999 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a77020 2024-06-11T14:01:03.000 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a77170 2024-06-11T14:01:03.000 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a772c0 2024-06-11T14:01:03.000 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a77730 2024-06-11T14:01:03.000 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a77c00 2024-06-11T14:01:03.000 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a77d30 2024-06-11T14:01:03.000 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a72080 2024-06-11T14:01:03.000 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a72190 2024-06-11T14:01:03.000 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a722e0 2024-06-11T14:01:03.000 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a77e60 2024-06-11T14:01:03.000 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a78310 2024-06-11T14:01:03.000 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a78460 2024-06-11T14:01:03.000 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a77860 2024-06-11T14:01:03.000 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a77990 2024-06-11T14:01:03.000 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a77ae0 2024-06-11T14:01:03.001 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a78590 2024-06-11T14:01:03.001 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a78a60 2024-06-11T14:01:03.001 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a78b90 2024-06-11T14:01:03.001 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a77f70 2024-06-11T14:01:03.001 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a780a0 2024-06-11T14:01:03.001 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a781d0 2024-06-11T14:01:03.001 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a78cc0 2024-06-11T14:01:03.001 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a79190 2024-06-11T14:01:03.001 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a792e0 2024-06-11T14:01:03.001 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a798c0 2024-06-11T14:01:03.001 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a78df0 2024-06-11T14:01:03.001 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a78f20 2024-06-11T14:01:03.001 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a79050 2024-06-11T14:01:03.001 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a799d0 2024-06-11T14:01:03.001 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a79410 2024-06-11T14:01:03.002 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a79540 2024-06-11T14:01:03.002 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a796b0 2024-06-11T14:01:03.002 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a79b20 2024-06-11T14:01:03.002 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7a010 2024-06-11T14:01:03.002 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7a120 2024-06-11T14:01:03.002 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7a700 2024-06-11T14:01:03.002 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a79c70 2024-06-11T14:01:03.002 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a79dc0 2024-06-11T14:01:03.002 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a79ef0 2024-06-11T14:01:03.002 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7a810 2024-06-11T14:01:03.002 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7a250 2024-06-11T14:01:03.002 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7a380 2024-06-11T14:01:03.002 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7a4d0 2024-06-11T14:01:03.002 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7a920 2024-06-11T14:01:03.003 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7adf0 2024-06-11T14:01:03.003 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7af20 2024-06-11T14:01:03.005 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7a5e0 2024-06-11T14:01:03.005 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7b050 2024-06-11T14:01:03.006 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7b1a0 2024-06-11T14:01:03.006 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7aa30 2024-06-11T14:01:03.006 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7ab60 2024-06-11T14:01:03.006 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7acb0 2024-06-11T14:01:03.006 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7b2f0 2024-06-11T14:01:03.006 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7b7c0 2024-06-11T14:01:03.006 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7b8f0 2024-06-11T14:01:03.006 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7bef0 2024-06-11T14:01:03.006 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7b420 2024-06-11T14:01:03.006 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7b530 2024-06-11T14:01:03.006 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7b660 2024-06-11T14:01:03.007 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7c040 2024-06-11T14:01:03.007 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7ba40 2024-06-11T14:01:03.007 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7bb90 2024-06-11T14:01:03.007 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7bcc0 2024-06-11T14:01:03.007 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7c190 2024-06-11T14:01:03.007 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7c660 2024-06-11T14:01:03.007 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7c7b0 2024-06-11T14:01:03.007 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a786c0 2024-06-11T14:01:03.007 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a787d0 2024-06-11T14:01:03.007 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a78900 2024-06-11T14:01:03.007 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7c900 2024-06-11T14:01:03.007 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7cdd0 2024-06-11T14:01:03.007 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7cee0 2024-06-11T14:01:03.008 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7c2c0 2024-06-11T14:01:03.008 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7c3f0 2024-06-11T14:01:03.008 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7c520 2024-06-11T14:01:03.008 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7d010 2024-06-11T14:01:03.008 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7d4e0 2024-06-11T14:01:03.008 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7d610 2024-06-11T14:01:03.008 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7ca30 2024-06-11T14:01:03.008 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7cb60 2024-06-11T14:01:03.008 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7ccb0 2024-06-11T14:01:03.008 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7d760 2024-06-11T14:01:03.008 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7dc30 2024-06-11T14:01:03.008 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7dd80 2024-06-11T14:01:03.009 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7e360 2024-06-11T14:01:03.009 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7d890 2024-06-11T14:01:03.009 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7d9c0 2024-06-11T14:01:03.009 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7daf0 2024-06-11T14:01:03.009 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7e490 2024-06-11T14:01:03.009 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7deb0 2024-06-11T14:01:03.009 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7dfe0 2024-06-11T14:01:03.009 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7e130 2024-06-11T14:01:03.009 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7e5c0 2024-06-11T14:01:03.009 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7e710 2024-06-11T14:01:03.009 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7ed10 2024-06-11T14:01:03.009 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7e860 2024-06-11T14:01:03.009 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7e9b0 2024-06-11T14:01:03.010 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7eac0 2024-06-11T14:01:03.010 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7ebf0 2024-06-11T14:01:03.010 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7f2d0 2024-06-11T14:01:03.010 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7ee20 2024-06-11T14:01:03.010 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7ef90 2024-06-11T14:01:03.010 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7f0a0 2024-06-11T14:01:03.010 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7f420 2024-06-11T14:01:03.010 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7f8f0 2024-06-11T14:01:03.010 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7fa00 2024-06-11T14:01:03.010 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7e240 2024-06-11T14:01:03.010 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7fb50 2024-06-11T14:01:03.010 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7fca0 2024-06-11T14:01:03.010 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7f550 2024-06-11T14:01:03.011 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7f680 2024-06-11T14:01:03.011 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7f7d0 2024-06-11T14:01:03.011 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7fdd0 2024-06-11T14:01:03.011 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a802a0 2024-06-11T14:01:03.011 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a803d0 2024-06-11T14:01:03.011 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7d140 2024-06-11T14:01:03.011 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7d250 2024-06-11T14:01:03.011 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7d3a0 2024-06-11T14:01:03.011 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a80500 2024-06-11T14:01:03.011 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a809d0 2024-06-11T14:01:03.011 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a80b00 2024-06-11T14:01:03.011 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a7fee0 2024-06-11T14:01:03.011 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a80010 2024-06-11T14:01:03.012 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a80140 2024-06-11T14:01:03.012 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a80c50 2024-06-11T14:01:03.012 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a80da0 2024-06-11T14:01:03.012 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a813a0 2024-06-11T14:01:03.012 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a80ef0 2024-06-11T14:01:03.012 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a81040 2024-06-11T14:01:03.012 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a81150 2024-06-11T14:01:03.012 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a81280 2024-06-11T14:01:03.012 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a81960 2024-06-11T14:01:03.012 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a814b0 2024-06-11T14:01:03.012 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a81600 2024-06-11T14:01:03.012 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a81730 2024-06-11T14:01:03.012 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a81a90 2024-06-11T14:01:03.012 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a81f60 2024-06-11T14:01:03.013 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a82070 2024-06-11T14:01:03.013 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a82670 2024-06-11T14:01:03.013 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a81bc0 2024-06-11T14:01:03.013 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a81d10 2024-06-11T14:01:03.013 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a821c0 2024-06-11T14:01:03.013 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a82310 2024-06-11T14:01:03.013 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a82440 2024-06-11T14:01:03.013 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a827c0 2024-06-11T14:01:03.013 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a82c90 2024-06-11T14:01:03.013 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a82dc0 2024-06-11T14:01:03.013 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a81840 2024-06-11T14:01:03.013 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a82ef0 2024-06-11T14:01:03.013 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a83040 2024-06-11T14:01:03.013 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a82550 2024-06-11T14:01:03.014 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a83170 2024-06-11T14:01:03.014 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a832e0 2024-06-11T14:01:03.014 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a80630 2024-06-11T14:01:03.014 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a80760 2024-06-11T14:01:03.014 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a80890 2024-06-11T14:01:03.014 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a83430 2024-06-11T14:01:03.014 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a83920 2024-06-11T14:01:03.014 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a83a50 2024-06-11T14:01:03.014 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a84050 2024-06-11T14:01:03.014 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a83580 2024-06-11T14:01:03.014 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a836b0 2024-06-11T14:01:03.014 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a837e0 2024-06-11T14:01:03.014 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a841a0 2024-06-11T14:01:03.015 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a83ba0 2024-06-11T14:01:03.015 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a83cf0 2024-06-11T14:01:03.015 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a83e20 2024-06-11T14:01:03.015 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a842d0 2024-06-11T14:01:03.015 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a847f0 2024-06-11T14:01:03.015 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a84920 2024-06-11T14:01:03.015 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a828f0 2024-06-11T14:01:03.015 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a82a20 2024-06-11T14:01:03.015 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a82b50 2024-06-11T14:01:03.015 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a84a50 2024-06-11T14:01:03.015 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a84f20 2024-06-11T14:01:03.015 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a85050 2024-06-11T14:01:03.015 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a85650 2024-06-11T14:01:03.015 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a84b80 2024-06-11T14:01:03.016 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a84cd0 2024-06-11T14:01:03.016 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a84e00 2024-06-11T14:01:03.016 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a85760 2024-06-11T14:01:03.016 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a851a0 2024-06-11T14:01:03.016 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a852d0 2024-06-11T14:01:03.016 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a85400 2024-06-11T14:01:03.016 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a85530 2024-06-11T14:01:03.016 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a85d60 2024-06-11T14:01:03.016 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a858b0 2024-06-11T14:01:03.016 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a859e0 2024-06-11T14:01:03.016 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a85b30 2024-06-11T14:01:03.016 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a85e70 2024-06-11T14:01:03.017 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a86320 2024-06-11T14:01:03.017 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a86470 2024-06-11T14:01:03.017 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a84400 2024-06-11T14:01:03.017 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a84530 2024-06-11T14:01:03.017 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a84660 2024-06-11T14:01:03.017 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a865a0 2024-06-11T14:01:03.017 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a866f0 2024-06-11T14:01:03.017 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a86cf0 2024-06-11T14:01:03.017 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a86840 2024-06-11T14:01:03.017 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a86970 2024-06-11T14:01:03.017 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a86aa0 2024-06-11T14:01:03.017 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a86bd0 2024-06-11T14:01:03.017 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a872f0 2024-06-11T14:01:03.018 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a86e40 2024-06-11T14:01:03.018 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a86f70 2024-06-11T14:01:03.018 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a870a0 2024-06-11T14:01:03.018 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a871d0 2024-06-11T14:01:03.018 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a878d0 2024-06-11T14:01:03.018 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a87420 2024-06-11T14:01:03.018 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a87570 2024-06-11T14:01:03.018 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a876c0 2024-06-11T14:01:03.018 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a87a00 2024-06-11T14:01:03.018 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a87eb0 2024-06-11T14:01:03.018 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a87fe0 2024-06-11T14:01:03.018 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a85f80 2024-06-11T14:01:03.018 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a860b0 2024-06-11T14:01:03.019 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a861e0 2024-06-11T14:01:03.019 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a88110 2024-06-11T14:01:03.019 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a885e0 2024-06-11T14:01:03.019 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a88730 2024-06-11T14:01:03.019 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a87b10 2024-06-11T14:01:03.019 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a87c40 2024-06-11T14:01:03.019 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a87d70 2024-06-11T14:01:03.019 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a88860 2024-06-11T14:01:03.019 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a88d30 2024-06-11T14:01:03.019 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a88e80 2024-06-11T14:01:03.019 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a88240 2024-06-11T14:01:03.019 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a88370 2024-06-11T14:01:03.019 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a884a0 2024-06-11T14:01:03.019 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a88fb0 2024-06-11T14:01:03.019 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a89460 2024-06-11T14:01:03.020 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a895b0 2024-06-11T14:01:03.020 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a88990 2024-06-11T14:01:03.020 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a88ac0 2024-06-11T14:01:03.020 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a88bf0 2024-06-11T14:01:03.020 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a89700 2024-06-11T14:01:03.020 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a89c10 2024-06-11T14:01:03.020 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a89d60 2024-06-11T14:01:03.020 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a890c0 2024-06-11T14:01:03.020 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a891f0 2024-06-11T14:01:03.020 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a89320 2024-06-11T14:01:03.020 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a89e90 2024-06-11T14:01:03.020 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8a340 2024-06-11T14:01:03.020 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8a490 2024-06-11T14:01:03.020 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8aa70 2024-06-11T14:01:03.020 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a89fa0 2024-06-11T14:01:03.021 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8a0d0 2024-06-11T14:01:03.021 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8a200 2024-06-11T14:01:03.021 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8ab80 2024-06-11T14:01:03.021 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8a5c0 2024-06-11T14:01:03.021 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8a6f0 2024-06-11T14:01:03.021 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8a840 2024-06-11T14:01:03.021 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8acb0 2024-06-11T14:01:03.021 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8b180 2024-06-11T14:01:03.021 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8b2b0 2024-06-11T14:01:03.021 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a89850 2024-06-11T14:01:03.021 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a89980 2024-06-11T14:01:03.021 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a89ab0 2024-06-11T14:01:03.021 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8b3e0 2024-06-11T14:01:03.021 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8b900 2024-06-11T14:01:03.022 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8baa0 2024-06-11T14:01:03.022 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8ade0 2024-06-11T14:01:03.022 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8af10 2024-06-11T14:01:03.022 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8b040 2024-06-11T14:01:03.022 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8bbd0 2024-06-11T14:01:03.022 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8c0a0 2024-06-11T14:01:03.022 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8c1f0 2024-06-11T14:01:03.022 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8b510 2024-06-11T14:01:03.022 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8b640 2024-06-11T14:01:03.022 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8b770 2024-06-11T14:01:03.022 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8c320 2024-06-11T14:01:03.022 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8c7f0 2024-06-11T14:01:03.022 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8c940 2024-06-11T14:01:03.023 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8bd00 2024-06-11T14:01:03.023 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8be30 2024-06-11T14:01:03.023 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8bf60 2024-06-11T14:01:03.023 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8ca70 2024-06-11T14:01:03.023 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8cf40 2024-06-11T14:01:03.023 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8d090 2024-06-11T14:01:03.023 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8d670 2024-06-11T14:01:03.023 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8cba0 2024-06-11T14:01:03.023 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8ccd0 2024-06-11T14:01:03.023 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8ce00 2024-06-11T14:01:03.023 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8d7a0 2024-06-11T14:01:03.023 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8d1c0 2024-06-11T14:01:03.023 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8d310 2024-06-11T14:01:03.023 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8d440 2024-06-11T14:01:03.023 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8d8d0 2024-06-11T14:01:03.201 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 14:01:02 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T14:01:03.201 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:03 smithi107 ceph-mon[33178]: pgmap v1666: 228 pgs: 228 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T14:01:03.202 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:03 smithi107 ceph-mon[33178]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T14:01:03.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:03 smithi057 ceph-mon[29805]: pgmap v1666: 228 pgs: 228 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T14:01:03.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:03 smithi057 ceph-mon[29805]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T14:01:03.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:03 smithi057 ceph-mon[40588]: pgmap v1666: 228 pgs: 228 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T14:01:03.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:03 smithi057 ceph-mon[40588]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T14:01:04.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 14:01:03 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:14:01:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T14:01:04.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:04 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:01:04.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:04 smithi057 ceph-mon[29805]: osdmap e767: 8 total, 8 up, 8 in 2024-06-11T14:01:04.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:04 smithi057 ceph-mon[29805]: osdmap e768: 8 total, 8 up, 8 in 2024-06-11T14:01:04.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:04 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:01:04.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:04 smithi057 ceph-mon[40588]: osdmap e767: 8 total, 8 up, 8 in 2024-06-11T14:01:04.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:04 smithi057 ceph-mon[40588]: osdmap e768: 8 total, 8 up, 8 in 2024-06-11T14:01:04.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:04 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:01:04.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:04 smithi107 ceph-mon[33178]: osdmap e767: 8 total, 8 up, 8 in 2024-06-11T14:01:04.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:04 smithi107 ceph-mon[33178]: osdmap e768: 8 total, 8 up, 8 in 2024-06-11T14:01:05.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:05 smithi057 ceph-mon[29805]: pgmap v1668: 196 pgs: 196 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:01:05.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:05 smithi057 ceph-mon[40588]: pgmap v1668: 196 pgs: 196 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:01:05.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:05 smithi107 ceph-mon[33178]: pgmap v1668: 196 pgs: 196 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:01:06.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:06 smithi057 ceph-mon[29805]: osdmap e769: 8 total, 8 up, 8 in 2024-06-11T14:01:06.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:06 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2032097369' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi057-77532-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T14:01:06.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:06 smithi057 ceph-mon[29805]: from='client.36437 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi057-77532-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T14:01:06.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:06 smithi057 ceph-mon[29805]: pgmap v1671: 196 pgs: 32 unknown, 164 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail 2024-06-11T14:01:06.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:06 smithi057 ceph-mon[40588]: osdmap e769: 8 total, 8 up, 8 in 2024-06-11T14:01:06.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:06 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2032097369' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi057-77532-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T14:01:06.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:06 smithi057 ceph-mon[40588]: from='client.36437 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi057-77532-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T14:01:06.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:06 smithi057 ceph-mon[40588]: pgmap v1671: 196 pgs: 32 unknown, 164 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail 2024-06-11T14:01:06.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:06 smithi107 ceph-mon[33178]: osdmap e769: 8 total, 8 up, 8 in 2024-06-11T14:01:06.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:06 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2032097369' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi057-77532-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T14:01:06.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:06 smithi107 ceph-mon[33178]: from='client.36437 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi057-77532-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-06-11T14:01:06.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:06 smithi107 ceph-mon[33178]: pgmap v1671: 196 pgs: 32 unknown, 164 active+clean; 583 KiB data, 886 MiB used, 714 GiB / 715 GiB avail 2024-06-11T14:01:07.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:07 smithi057 ceph-mon[40588]: from='client.36437 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi057-77532-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T14:01:07.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:07 smithi057 ceph-mon[40588]: osdmap e770: 8 total, 8 up, 8 in 2024-06-11T14:01:07.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:07 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2032097369' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-06-11T14:01:07.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:07 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2032097369' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi057-77532-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-06-11T14:01:07.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:07 smithi057 ceph-mon[40588]: from='client.36437 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi057-77532-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-06-11T14:01:07.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:07 smithi057 ceph-mon[29805]: from='client.36437 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi057-77532-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T14:01:07.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:07 smithi057 ceph-mon[29805]: osdmap e770: 8 total, 8 up, 8 in 2024-06-11T14:01:07.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:07 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2032097369' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-06-11T14:01:07.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:07 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2032097369' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi057-77532-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-06-11T14:01:07.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:07 smithi057 ceph-mon[29805]: from='client.36437 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi057-77532-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-06-11T14:01:07.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:07 smithi107 ceph-mon[33178]: from='client.36437 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi057-77532-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-06-11T14:01:07.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:07 smithi107 ceph-mon[33178]: osdmap e770: 8 total, 8 up, 8 in 2024-06-11T14:01:07.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:07 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2032097369' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-06-11T14:01:07.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:07 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2032097369' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi057-77532-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-06-11T14:01:07.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:07 smithi107 ceph-mon[33178]: from='client.36437 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi057-77532-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-06-11T14:01:08.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:08 smithi057 ceph-mon[29805]: from='client.36437 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi057-77532-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-06-11T14:01:08.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:08 smithi057 ceph-mon[29805]: osdmap e771: 8 total, 8 up, 8 in 2024-06-11T14:01:08.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:08 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2032097369' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi057-77532-1","app": "app2"}]: dispatch 2024-06-11T14:01:08.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:08 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2032097369' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi057-77532-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-06-11T14:01:08.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:08 smithi057 ceph-mon[29805]: from='client.36437 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi057-77532-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-06-11T14:01:08.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:08 smithi057 ceph-mon[29805]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T14:01:08.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:08 smithi057 ceph-mon[29805]: pgmap v1674: 196 pgs: 32 creating+peering, 164 active+clean; 583 KiB data, 903 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:01:08.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:08 smithi057 ceph-mon[40588]: from='client.36437 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi057-77532-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-06-11T14:01:08.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:08 smithi057 ceph-mon[40588]: osdmap e771: 8 total, 8 up, 8 in 2024-06-11T14:01:08.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:08 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2032097369' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi057-77532-1","app": "app2"}]: dispatch 2024-06-11T14:01:08.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:08 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2032097369' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi057-77532-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-06-11T14:01:08.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:08 smithi057 ceph-mon[40588]: from='client.36437 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi057-77532-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-06-11T14:01:08.538 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:08 smithi057 ceph-mon[40588]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T14:01:08.538 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:08 smithi057 ceph-mon[40588]: pgmap v1674: 196 pgs: 32 creating+peering, 164 active+clean; 583 KiB data, 903 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:01:08.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:08 smithi107 ceph-mon[33178]: from='client.36437 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi057-77532-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-06-11T14:01:08.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:08 smithi107 ceph-mon[33178]: osdmap e771: 8 total, 8 up, 8 in 2024-06-11T14:01:08.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:08 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2032097369' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi057-77532-1","app": "app2"}]: dispatch 2024-06-11T14:01:08.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:08 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2032097369' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi057-77532-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-06-11T14:01:08.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:08 smithi107 ceph-mon[33178]: from='client.36437 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi057-77532-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-06-11T14:01:08.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:08 smithi107 ceph-mon[33178]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T14:01:08.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:08 smithi107 ceph-mon[33178]: pgmap v1674: 196 pgs: 32 creating+peering, 164 active+clean; 583 KiB data, 903 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:01:09.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:09 smithi057 ceph-mon[40588]: from='client.36437 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi057-77532-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-06-11T14:01:09.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:09 smithi057 ceph-mon[40588]: osdmap e772: 8 total, 8 up, 8 in 2024-06-11T14:01:09.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:09 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2032097369' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi057-77532-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-06-11T14:01:09.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:09 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2032097369' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi057-77532-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-06-11T14:01:09.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:09 smithi057 ceph-mon[40588]: from='client.36437 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi057-77532-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-06-11T14:01:09.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:09 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:01:09.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:09 smithi057 ceph-mon[29805]: from='client.36437 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi057-77532-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-06-11T14:01:09.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:09 smithi057 ceph-mon[29805]: osdmap e772: 8 total, 8 up, 8 in 2024-06-11T14:01:09.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:09 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2032097369' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi057-77532-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-06-11T14:01:09.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:09 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2032097369' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi057-77532-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-06-11T14:01:09.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:09 smithi057 ceph-mon[29805]: from='client.36437 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi057-77532-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-06-11T14:01:09.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:09 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:01:09.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:09 smithi107 ceph-mon[33178]: from='client.36437 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi057-77532-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-06-11T14:01:09.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:09 smithi107 ceph-mon[33178]: osdmap e772: 8 total, 8 up, 8 in 2024-06-11T14:01:09.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:09 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2032097369' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi057-77532-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-06-11T14:01:09.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:09 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2032097369' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi057-77532-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-06-11T14:01:09.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:09 smithi107 ceph-mon[33178]: from='client.36437 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi057-77532-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-06-11T14:01:09.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:09 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:01:10.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:10 smithi057 ceph-mon[40588]: from='client.36437 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi057-77532-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-06-11T14:01:10.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:10 smithi057 ceph-mon[40588]: osdmap e773: 8 total, 8 up, 8 in 2024-06-11T14:01:10.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:10 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2032097369' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi057-77532-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-06-11T14:01:10.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:10 smithi057 ceph-mon[40588]: from='client.36437 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi057-77532-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-06-11T14:01:10.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:10 smithi057 ceph-mon[40588]: pgmap v1677: 196 pgs: 32 creating+peering, 164 active+clean; 583 KiB data, 903 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:01:10.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:10 smithi057 ceph-mon[29805]: from='client.36437 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi057-77532-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-06-11T14:01:10.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:10 smithi057 ceph-mon[29805]: osdmap e773: 8 total, 8 up, 8 in 2024-06-11T14:01:10.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:10 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2032097369' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi057-77532-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-06-11T14:01:10.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:10 smithi057 ceph-mon[29805]: from='client.36437 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi057-77532-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-06-11T14:01:10.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:10 smithi057 ceph-mon[29805]: pgmap v1677: 196 pgs: 32 creating+peering, 164 active+clean; 583 KiB data, 903 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:01:10.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:10 smithi107 ceph-mon[33178]: from='client.36437 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi057-77532-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-06-11T14:01:10.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:10 smithi107 ceph-mon[33178]: osdmap e773: 8 total, 8 up, 8 in 2024-06-11T14:01:10.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:10 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2032097369' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi057-77532-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-06-11T14:01:10.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:10 smithi107 ceph-mon[33178]: from='client.36437 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi057-77532-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-06-11T14:01:10.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:10 smithi107 ceph-mon[33178]: pgmap v1677: 196 pgs: 32 creating+peering, 164 active+clean; 583 KiB data, 903 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:01:11.192 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8dda0 2024-06-11T14:01:11.192 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8ded0 2024-06-11T14:01:11.192 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8c450 2024-06-11T14:01:11.192 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8c580 2024-06-11T14:01:11.192 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8c6b0 2024-06-11T14:01:11.193 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8e000 2024-06-11T14:01:11.193 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8e4d0 2024-06-11T14:01:11.193 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8e600 2024-06-11T14:01:11.193 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8e110 2024-06-11T14:01:11.193 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8e240 2024-06-11T14:01:11.193 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8e390 2024-06-11T14:01:11.193 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8e750 2024-06-11T14:01:11.193 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8ec20 2024-06-11T14:01:11.193 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8ed70 2024-06-11T14:01:11.193 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8da00 2024-06-11T14:01:11.193 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8db50 2024-06-11T14:01:11.193 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8dc80 2024-06-11T14:01:11.193 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8ee80 2024-06-11T14:01:11.193 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8f390 2024-06-11T14:01:11.194 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8f4e0 2024-06-11T14:01:11.194 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8e880 2024-06-11T14:01:11.194 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8e9b0 2024-06-11T14:01:11.194 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8eae0 2024-06-11T14:01:11.194 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8f610 2024-06-11T14:01:11.194 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8fae0 2024-06-11T14:01:11.194 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8fc30 2024-06-11T14:01:11.194 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8efd0 2024-06-11T14:01:11.194 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8f100 2024-06-11T14:01:11.194 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8f230 2024-06-11T14:01:11.194 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8fd60 2024-06-11T14:01:11.194 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a90230 2024-06-11T14:01:11.194 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a90380 2024-06-11T14:01:11.195 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8f740 2024-06-11T14:01:11.195 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8f870 2024-06-11T14:01:11.195 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8f9a0 2024-06-11T14:01:11.195 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a904b0 2024-06-11T14:01:11.195 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a90980 2024-06-11T14:01:11.195 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a90ad0 2024-06-11T14:01:11.195 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a910b0 2024-06-11T14:01:11.195 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a905e0 2024-06-11T14:01:11.195 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a90710 2024-06-11T14:01:11.195 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a90840 2024-06-11T14:01:11.195 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a911c0 2024-06-11T14:01:11.195 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a90c00 2024-06-11T14:01:11.195 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a90d50 2024-06-11T14:01:11.195 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a90ea0 2024-06-11T14:01:11.195 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a912f0 2024-06-11T14:01:11.196 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a917c0 2024-06-11T14:01:11.196 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a918d0 2024-06-11T14:01:11.196 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8fe90 2024-06-11T14:01:11.196 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a8ffc0 2024-06-11T14:01:11.196 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a900f0 2024-06-11T14:01:11.196 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a91a20 2024-06-11T14:01:11.196 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a91b50 2024-06-11T14:01:11.196 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a92150 2024-06-11T14:01:11.196 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a91ca0 2024-06-11T14:01:11.196 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a91dd0 2024-06-11T14:01:11.196 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a91f00 2024-06-11T14:01:11.196 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a92030 2024-06-11T14:01:11.196 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a92730 2024-06-11T14:01:11.196 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a92280 2024-06-11T14:01:11.196 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a923b0 2024-06-11T14:01:11.196 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a92500 2024-06-11T14:01:11.197 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a92860 2024-06-11T14:01:11.197 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: waiting 0x564078a92d30 2024-06-11T14:01:11.197 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: done. 2024-06-11T14:01:11.197 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: [ OK ] LibRadosMiscPool.PoolCreationRace (1818324 ms) 2024-06-11T14:01:11.197 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: [----------] 1 test from LibRadosMiscPool (1818324 ms total) 2024-06-11T14:01:11.197 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: 2024-06-11T14:01:11.197 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: [----------] 8 tests from LibRadosMisc 2024-06-11T14:01:11.197 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: [ RUN ] LibRadosMisc.ClusterFSID 2024-06-11T14:01:11.197 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: [ OK ] LibRadosMisc.ClusterFSID (0 ms) 2024-06-11T14:01:11.197 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: [ RUN ] LibRadosMisc.Exec 2024-06-11T14:01:11.197 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: [ OK ] LibRadosMisc.Exec (20 ms) 2024-06-11T14:01:11.197 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: [ RUN ] LibRadosMisc.WriteSame 2024-06-11T14:01:11.197 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: [ OK ] LibRadosMisc.WriteSame (4 ms) 2024-06-11T14:01:11.197 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: [ RUN ] LibRadosMisc.CmpExt 2024-06-11T14:01:11.197 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: [ OK ] LibRadosMisc.CmpExt (3 ms) 2024-06-11T14:01:11.198 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: [ RUN ] LibRadosMisc.Applications 2024-06-11T14:01:11.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:11 smithi057 ceph-mon[40588]: from='client.36437 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi057-77532-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-06-11T14:01:11.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:11 smithi057 ceph-mon[40588]: osdmap e774: 8 total, 8 up, 8 in 2024-06-11T14:01:11.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:11 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2032097369' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi057-77532-1","app":"app1","key":"key1"}]: dispatch 2024-06-11T14:01:11.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:11 smithi057 ceph-mon[40588]: from='client.36437 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi057-77532-1","app":"app1","key":"key1"}]: dispatch 2024-06-11T14:01:11.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:11 smithi057 ceph-mon[29805]: from='client.36437 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi057-77532-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-06-11T14:01:11.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:11 smithi057 ceph-mon[29805]: osdmap e774: 8 total, 8 up, 8 in 2024-06-11T14:01:11.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:11 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2032097369' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi057-77532-1","app":"app1","key":"key1"}]: dispatch 2024-06-11T14:01:11.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:11 smithi057 ceph-mon[29805]: from='client.36437 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi057-77532-1","app":"app1","key":"key1"}]: dispatch 2024-06-11T14:01:11.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:11 smithi107 ceph-mon[33178]: from='client.36437 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi057-77532-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-06-11T14:01:11.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:11 smithi107 ceph-mon[33178]: osdmap e774: 8 total, 8 up, 8 in 2024-06-11T14:01:11.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:11 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2032097369' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi057-77532-1","app":"app1","key":"key1"}]: dispatch 2024-06-11T14:01:11.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:11 smithi107 ceph-mon[33178]: from='client.36437 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi057-77532-1","app":"app1","key":"key1"}]: dispatch 2024-06-11T14:01:12.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:12 smithi057 ceph-mon[40588]: from='client.36437 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi057-77532-1","app":"app1","key":"key1"}]': finished 2024-06-11T14:01:12.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:12 smithi057 ceph-mon[40588]: osdmap e775: 8 total, 8 up, 8 in 2024-06-11T14:01:12.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:12 smithi057 ceph-mon[40588]: pgmap v1680: 196 pgs: 196 active+clean; 583 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-06-11T14:01:12.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:12 smithi057 ceph-mon[29805]: from='client.36437 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi057-77532-1","app":"app1","key":"key1"}]': finished 2024-06-11T14:01:12.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:12 smithi057 ceph-mon[29805]: osdmap e775: 8 total, 8 up, 8 in 2024-06-11T14:01:12.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:12 smithi057 ceph-mon[29805]: pgmap v1680: 196 pgs: 196 active+clean; 583 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-06-11T14:01:12.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:12 smithi107 ceph-mon[33178]: from='client.36437 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi057-77532-1","app":"app1","key":"key1"}]': finished 2024-06-11T14:01:12.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:12 smithi107 ceph-mon[33178]: osdmap e775: 8 total, 8 up, 8 in 2024-06-11T14:01:12.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:12 smithi107 ceph-mon[33178]: pgmap v1680: 196 pgs: 196 active+clean; 583 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-06-11T14:01:13.200 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 14:01:12 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T14:01:13.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:13 smithi057 ceph-mon[40588]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T14:01:13.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:13 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:01:13.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:13 smithi057 ceph-mon[29805]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T14:01:13.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:13 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:01:13.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:13 smithi107 ceph-mon[33178]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T14:01:13.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:13 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:01:14.037 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 14:01:13 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:14:01:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T14:01:14.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:14 smithi057 ceph-mon[29805]: pgmap v1681: 196 pgs: 196 active+clean; 583 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 886 B/s wr, 2 op/s 2024-06-11T14:01:14.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:14 smithi057 ceph-mon[40588]: pgmap v1681: 196 pgs: 196 active+clean; 583 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 886 B/s wr, 2 op/s 2024-06-11T14:01:14.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:14 smithi107 ceph-mon[33178]: pgmap v1681: 196 pgs: 196 active+clean; 583 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 886 B/s wr, 2 op/s 2024-06-11T14:01:17.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:16 smithi057 ceph-mon[40588]: pgmap v1682: 196 pgs: 196 active+clean; 583 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 755 B/s wr, 1 op/s 2024-06-11T14:01:17.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:16 smithi057 ceph-mon[29805]: pgmap v1682: 196 pgs: 196 active+clean; 583 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 755 B/s wr, 1 op/s 2024-06-11T14:01:17.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:16 smithi107 ceph-mon[33178]: pgmap v1682: 196 pgs: 196 active+clean; 583 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 755 B/s wr, 1 op/s 2024-06-11T14:01:19.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:18 smithi057 ceph-mon[29805]: pgmap v1683: 196 pgs: 196 active+clean; 583 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 639 B/s wr, 2 op/s 2024-06-11T14:01:19.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:18 smithi057 ceph-mon[40588]: pgmap v1683: 196 pgs: 196 active+clean; 583 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 639 B/s wr, 2 op/s 2024-06-11T14:01:19.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:18 smithi107 ceph-mon[33178]: pgmap v1683: 196 pgs: 196 active+clean; 583 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 639 B/s wr, 2 op/s 2024-06-11T14:01:21.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:20 smithi057 ceph-mon[29805]: pgmap v1684: 196 pgs: 196 active+clean; 583 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 525 B/s wr, 1 op/s 2024-06-11T14:01:21.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:20 smithi057 ceph-mon[40588]: pgmap v1684: 196 pgs: 196 active+clean; 583 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 525 B/s wr, 1 op/s 2024-06-11T14:01:21.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:20 smithi107 ceph-mon[33178]: pgmap v1684: 196 pgs: 196 active+clean; 583 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 525 B/s wr, 1 op/s 2024-06-11T14:01:23.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:22 smithi057 ceph-mon[40588]: pgmap v1685: 196 pgs: 196 active+clean; 583 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 954 B/s rd, 0 op/s 2024-06-11T14:01:23.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:22 smithi057 ceph-mon[29805]: pgmap v1685: 196 pgs: 196 active+clean; 583 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 954 B/s rd, 0 op/s 2024-06-11T14:01:23.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:22 smithi107 ceph-mon[33178]: pgmap v1685: 196 pgs: 196 active+clean; 583 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 954 B/s rd, 0 op/s 2024-06-11T14:01:23.300 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 14:01:22 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T14:01:23.986 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 14:01:23 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:14:01:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T14:01:23.987 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:23 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:01:23.987 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:23 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:01:23.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:23 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:01:23.987 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:23 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:01:24.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:23 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:01:24.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:23 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:01:25.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:24 smithi057 ceph-mon[29805]: pgmap v1686: 196 pgs: 196 active+clean; 583 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:01:25.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:24 smithi057 ceph-mon[40588]: pgmap v1686: 196 pgs: 196 active+clean; 583 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:01:25.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:24 smithi107 ceph-mon[33178]: pgmap v1686: 196 pgs: 196 active+clean; 583 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:01:27.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:26 smithi057 ceph-mon[29805]: pgmap v1687: 196 pgs: 196 active+clean; 583 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:01:27.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:26 smithi057 ceph-mon[40588]: pgmap v1687: 196 pgs: 196 active+clean; 583 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:01:27.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:26 smithi107 ceph-mon[33178]: pgmap v1687: 196 pgs: 196 active+clean; 583 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:01:29.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:28 smithi057 ceph-mon[29805]: pgmap v1688: 196 pgs: 196 active+clean; 583 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:01:29.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:28 smithi057 ceph-mon[40588]: pgmap v1688: 196 pgs: 196 active+clean; 583 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:01:29.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:28 smithi107 ceph-mon[33178]: pgmap v1688: 196 pgs: 196 active+clean; 583 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:01:30.006 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: [ OK ] LibRadosMisc.Applications (5072 ms) 2024-06-11T14:01:30.006 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatOSD 2024-06-11T14:01:30.006 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatOSD (0 ms) 2024-06-11T14:01:30.006 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatClient 2024-06-11T14:01:30.007 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatClient (0 ms) 2024-06-11T14:01:30.007 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: [ RUN ] LibRadosMisc.ShutdownRace 2024-06-11T14:01:30.007 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: [ OK ] LibRadosMisc.ShutdownRace (18566 ms) 2024-06-11T14:01:30.007 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: [----------] 8 tests from LibRadosMisc (23665 ms total) 2024-06-11T14:01:30.007 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: 2024-06-11T14:01:30.007 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: [----------] Global test environment tear-down 2024-06-11T14:01:30.007 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: [==========] 12 tests from 4 test suites ran. (1846339 ms total) 2024-06-11T14:01:30.007 INFO:tasks.workunit.client.0.smithi057.stdout: api_misc: [ PASSED ] 12 tests. 2024-06-11T14:01:30.013 INFO:tasks.workunit.client.0.smithi057.stderr:+ for t in "${!pids[@]}" 2024-06-11T14:01:30.013 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77733 2024-06-11T14:01:30.013 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 77733 2024-06-11T14:01:30.013 INFO:tasks.workunit.client.0.smithi057.stderr:+ for t in "${!pids[@]}" 2024-06-11T14:01:30.013 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=78271 2024-06-11T14:01:30.013 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 78271 2024-06-11T14:01:30.014 INFO:tasks.workunit.client.0.smithi057.stderr:+ for t in "${!pids[@]}" 2024-06-11T14:01:30.014 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=78321 2024-06-11T14:01:30.014 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 78321 2024-06-11T14:01:30.014 INFO:tasks.workunit.client.0.smithi057.stderr:+ for t in "${!pids[@]}" 2024-06-11T14:01:30.014 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77420 2024-06-11T14:01:30.014 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 77420 2024-06-11T14:01:30.014 INFO:tasks.workunit.client.0.smithi057.stderr:+ for t in "${!pids[@]}" 2024-06-11T14:01:30.014 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=78106 2024-06-11T14:01:30.014 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 78106 2024-06-11T14:01:30.014 INFO:tasks.workunit.client.0.smithi057.stderr:+ for t in "${!pids[@]}" 2024-06-11T14:01:30.014 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77963 2024-06-11T14:01:30.014 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 77963 2024-06-11T14:01:30.014 INFO:tasks.workunit.client.0.smithi057.stderr:+ for t in "${!pids[@]}" 2024-06-11T14:01:30.015 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77291 2024-06-11T14:01:30.015 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 77291 2024-06-11T14:01:30.015 INFO:tasks.workunit.client.0.smithi057.stderr:+ for t in "${!pids[@]}" 2024-06-11T14:01:30.015 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77352 2024-06-11T14:01:30.015 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 77352 2024-06-11T14:01:30.015 INFO:tasks.workunit.client.0.smithi057.stderr:+ for t in "${!pids[@]}" 2024-06-11T14:01:30.015 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77977 2024-06-11T14:01:30.015 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 77977 2024-06-11T14:01:30.015 INFO:tasks.workunit.client.0.smithi057.stderr:+ for t in "${!pids[@]}" 2024-06-11T14:01:30.015 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77282 2024-06-11T14:01:30.015 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 77282 2024-06-11T14:01:30.015 INFO:tasks.workunit.client.0.smithi057.stderr:+ for t in "${!pids[@]}" 2024-06-11T14:01:30.015 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77302 2024-06-11T14:01:30.016 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 77302 2024-06-11T14:01:30.016 INFO:tasks.workunit.client.0.smithi057.stderr:+ for t in "${!pids[@]}" 2024-06-11T14:01:30.016 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=78498 2024-06-11T14:01:30.016 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 78498 2024-06-11T14:01:30.016 INFO:tasks.workunit.client.0.smithi057.stderr:+ for t in "${!pids[@]}" 2024-06-11T14:01:30.016 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77618 2024-06-11T14:01:30.016 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 77618 2024-06-11T14:01:30.016 INFO:tasks.workunit.client.0.smithi057.stderr:+ for t in "${!pids[@]}" 2024-06-11T14:01:30.016 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=78574 2024-06-11T14:01:30.016 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 78574 2024-06-11T14:01:30.016 INFO:tasks.workunit.client.0.smithi057.stderr:+ for t in "${!pids[@]}" 2024-06-11T14:01:30.016 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77708 2024-06-11T14:01:30.016 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 77708 2024-06-11T14:01:30.016 INFO:tasks.workunit.client.0.smithi057.stderr:+ for t in "${!pids[@]}" 2024-06-11T14:01:30.017 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=78195 2024-06-11T14:01:30.017 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 78195 2024-06-11T14:01:30.017 INFO:tasks.workunit.client.0.smithi057.stderr:+ for t in "${!pids[@]}" 2024-06-11T14:01:30.017 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=78655 2024-06-11T14:01:30.017 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 78655 2024-06-11T14:01:30.017 INFO:tasks.workunit.client.0.smithi057.stderr:+ for t in "${!pids[@]}" 2024-06-11T14:01:30.017 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77644 2024-06-11T14:01:30.017 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 77644 2024-06-11T14:01:30.017 INFO:tasks.workunit.client.0.smithi057.stderr:+ for t in "${!pids[@]}" 2024-06-11T14:01:30.017 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77996 2024-06-11T14:01:30.017 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 77996 2024-06-11T14:01:30.017 INFO:tasks.workunit.client.0.smithi057.stderr:+ for t in "${!pids[@]}" 2024-06-11T14:01:30.017 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=78441 2024-06-11T14:01:30.017 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 78441 2024-06-11T14:01:30.018 INFO:tasks.workunit.client.0.smithi057.stderr:+ for t in "${!pids[@]}" 2024-06-11T14:01:30.018 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=78149 2024-06-11T14:01:30.018 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 78149 2024-06-11T14:01:30.018 INFO:tasks.workunit.client.0.smithi057.stderr:+ for t in "${!pids[@]}" 2024-06-11T14:01:30.018 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=78533 2024-06-11T14:01:30.018 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 78533 2024-06-11T14:01:30.018 INFO:tasks.workunit.client.0.smithi057.stderr:+ for t in "${!pids[@]}" 2024-06-11T14:01:30.018 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77921 2024-06-11T14:01:30.018 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 77921 2024-06-11T14:01:30.018 INFO:tasks.workunit.client.0.smithi057.stderr:+ for t in "${!pids[@]}" 2024-06-11T14:01:30.018 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77543 2024-06-11T14:01:30.018 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 77543 2024-06-11T14:01:30.018 INFO:tasks.workunit.client.0.smithi057.stderr:+ for t in "${!pids[@]}" 2024-06-11T14:01:30.019 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=78598 2024-06-11T14:01:30.019 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 78598 2024-06-11T14:01:30.019 INFO:tasks.workunit.client.0.smithi057.stderr:+ for t in "${!pids[@]}" 2024-06-11T14:01:30.019 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77861 2024-06-11T14:01:30.019 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 77861 2024-06-11T14:01:30.019 INFO:tasks.workunit.client.0.smithi057.stderr:+ for t in "${!pids[@]}" 2024-06-11T14:01:30.019 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77275 2024-06-11T14:01:30.019 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 77275 2024-06-11T14:01:30.019 INFO:tasks.workunit.client.0.smithi057.stderr:+ for t in "${!pids[@]}" 2024-06-11T14:01:30.019 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77332 2024-06-11T14:01:30.019 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 77332 2024-06-11T14:01:30.019 INFO:tasks.workunit.client.0.smithi057.stderr:+ for t in "${!pids[@]}" 2024-06-11T14:01:30.019 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77805 2024-06-11T14:01:30.019 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 77805 2024-06-11T14:01:30.019 INFO:tasks.workunit.client.0.smithi057.stderr:+ for t in "${!pids[@]}" 2024-06-11T14:01:30.020 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77382 2024-06-11T14:01:30.020 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 77382 2024-06-11T14:01:30.020 INFO:tasks.workunit.client.0.smithi057.stderr:+ for t in "${!pids[@]}" 2024-06-11T14:01:30.020 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77877 2024-06-11T14:01:30.020 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 77877 2024-06-11T14:01:30.020 INFO:tasks.workunit.client.0.smithi057.stderr:+ for t in "${!pids[@]}" 2024-06-11T14:01:30.020 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=77908 2024-06-11T14:01:30.020 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 77908 2024-06-11T14:01:30.020 INFO:tasks.workunit.client.0.smithi057.stderr:+ for t in "${!pids[@]}" 2024-06-11T14:01:30.020 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=78361 2024-06-11T14:01:30.020 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 78361 2024-06-11T14:01:30.020 INFO:tasks.workunit.client.0.smithi057.stderr:+ for t in "${!pids[@]}" 2024-06-11T14:01:30.020 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=78675 2024-06-11T14:01:30.020 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 78675 2024-06-11T14:01:30.020 INFO:tasks.workunit.client.0.smithi057.stderr:+ exit 0 2024-06-11T14:01:30.021 INFO:tasks.workunit.client.0.smithi057.stderr:+ cleanup 2024-06-11T14:01:30.021 INFO:tasks.workunit.client.0.smithi057.stderr:+ pkill -P 77269 2024-06-11T14:01:30.024 INFO:tasks.workunit.client.0.smithi057.stderr:+ true 2024-06-11T14:01:30.025 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-11T14:01:30.025 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-11T14:01:30.053 INFO:tasks.workunit:Running workunits matching rados/test_pool_quota.sh on client.0... 2024-06-11T14:01:30.055 INFO:tasks.workunit:Running workunit rados/test_pool_quota.sh... 2024-06-11T14:01:30.055 DEBUG:teuthology.orchestra.run.smithi057:workunit test rados/test_pool_quota.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3f51ea3944d39ca13aa40153c9a4ee298306679d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_pool_quota.sh 2024-06-11T14:01:30.114 INFO:tasks.workunit.client.0.smithi057.stderr:++ uuidgen 2024-06-11T14:01:30.122 INFO:tasks.workunit.client.0.smithi057.stderr:+ p=5bd44950-56a4-47a9-a376-eee5f24dbed6 2024-06-11T14:01:30.122 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph osd pool create 5bd44950-56a4-47a9-a376-eee5f24dbed6 12 2024-06-11T14:01:30.208 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:30.208+0000 7fdf40246640 1 -- 172.21.15.57:0/3677875891 >> v1:172.21.15.57:6789/0 conn(0x7fdf3810cbe0 legacy=0x7fdf3810efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:01:30.208 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:30.208+0000 7fdf40246640 1 -- 172.21.15.57:0/3677875891 shutdown_connections 2024-06-11T14:01:30.208 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:30.208+0000 7fdf40246640 1 -- 172.21.15.57:0/3677875891 >> 172.21.15.57:0/3677875891 conn(0x7fdf380fc230 msgr2=0x7fdf380fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T14:01:30.208 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:30.208+0000 7fdf40246640 1 -- 172.21.15.57:0/3677875891 shutdown_connections 2024-06-11T14:01:30.208 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:30.208+0000 7fdf40246640 1 -- 172.21.15.57:0/3677875891 wait complete. 2024-06-11T14:01:30.210 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:30.209+0000 7fdf40246640 1 Processor -- start 2024-06-11T14:01:30.211 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:30.209+0000 7fdf40246640 1 -- start start 2024-06-11T14:01:30.211 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:30.209+0000 7fdf40246640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fdf381ad130 con 0x7fdf3810cbe0 2024-06-11T14:01:30.211 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:30.209+0000 7fdf40246640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fdf381ae310 con 0x7fdf38100600 2024-06-11T14:01:30.211 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:30.209+0000 7fdf40246640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fdf381af4f0 con 0x7fdf381106f0 2024-06-11T14:01:30.211 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:30.209+0000 7fdf3d7e1640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fdf3810cbe0 0x7fdf38109b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:58242/0 (socket says 172.21.15.57:58242) 2024-06-11T14:01:30.211 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:30.209+0000 7fdf3d7e1640 1 -- 172.21.15.57:0/3414453095 learned_addr learned my addr 172.21.15.57:0/3414453095 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T14:01:30.211 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:30.209+0000 7fdf2effd640 1 -- 172.21.15.57:0/3414453095 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4113721261 0 0) 0x7fdf381af4f0 con 0x7fdf381106f0 2024-06-11T14:01:30.211 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:30.209+0000 7fdf2effd640 1 -- 172.21.15.57:0/3414453095 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf24003520 con 0x7fdf381106f0 2024-06-11T14:01:30.211 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:30.209+0000 7fdf2effd640 1 -- 172.21.15.57:0/3414453095 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 5647474 0 0) 0x7fdf381ad130 con 0x7fdf3810cbe0 2024-06-11T14:01:30.211 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:30.209+0000 7fdf2effd640 1 -- 172.21.15.57:0/3414453095 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf381af4f0 con 0x7fdf3810cbe0 2024-06-11T14:01:30.211 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:30.210+0000 7fdf2effd640 1 -- 172.21.15.57:0/3414453095 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1294007946 0 0) 0x7fdf381ae310 con 0x7fdf38100600 2024-06-11T14:01:30.211 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:30.210+0000 7fdf2effd640 1 -- 172.21.15.57:0/3414453095 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf381ad130 con 0x7fdf38100600 2024-06-11T14:01:30.211 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:30.210+0000 7fdf2effd640 1 -- 172.21.15.57:0/3414453095 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2849000698 0 0) 0x7fdf24003520 con 0x7fdf381106f0 2024-06-11T14:01:30.212 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:30.210+0000 7fdf2effd640 1 -- 172.21.15.57:0/3414453095 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fdf381ae310 con 0x7fdf381106f0 2024-06-11T14:01:30.212 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:30.210+0000 7fdf2effd640 1 -- 172.21.15.57:0/3414453095 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1856838565 0 0) 0x7fdf381af4f0 con 0x7fdf3810cbe0 2024-06-11T14:01:30.212 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:30.210+0000 7fdf2effd640 1 -- 172.21.15.57:0/3414453095 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fdf24003520 con 0x7fdf3810cbe0 2024-06-11T14:01:30.212 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:30.210+0000 7fdf2effd640 1 -- 172.21.15.57:0/3414453095 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fdf30001db0 con 0x7fdf381106f0 2024-06-11T14:01:30.212 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:30.210+0000 7fdf2effd640 1 -- 172.21.15.57:0/3414453095 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1525854483 0 0) 0x7fdf381ad130 con 0x7fdf38100600 2024-06-11T14:01:30.212 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:30.210+0000 7fdf2effd640 1 -- 172.21.15.57:0/3414453095 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fdf381af4f0 con 0x7fdf38100600 2024-06-11T14:01:30.212 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:30.210+0000 7fdf2effd640 1 -- 172.21.15.57:0/3414453095 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fdf280043a0 con 0x7fdf3810cbe0 2024-06-11T14:01:30.212 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:30.210+0000 7fdf2effd640 1 -- 172.21.15.57:0/3414453095 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fdf34001830 con 0x7fdf38100600 2024-06-11T14:01:30.212 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:30.210+0000 7fdf2effd640 1 -- 172.21.15.57:0/3414453095 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3424259438 0 0) 0x7fdf24003520 con 0x7fdf3810cbe0 2024-06-11T14:01:30.212 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:30.210+0000 7fdf2effd640 1 -- 172.21.15.57:0/3414453095 >> v1:172.21.15.57:6790/0 conn(0x7fdf381106f0 legacy=0x7fdf3810a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:01:30.212 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:30.210+0000 7fdf2effd640 1 -- 172.21.15.57:0/3414453095 >> v1:172.21.15.107:6789/0 conn(0x7fdf38100600 legacy=0x7fdf3810c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:01:30.212 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:30.210+0000 7fdf2effd640 1 -- 172.21.15.57:0/3414453095 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf381b06d0 con 0x7fdf3810cbe0 2024-06-11T14:01:30.212 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:30.210+0000 7fdf40246640 1 -- 172.21.15.57:0/3414453095 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fdf381ad360 con 0x7fdf3810cbe0 2024-06-11T14:01:30.213 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:30.210+0000 7fdf40246640 1 -- 172.21.15.57:0/3414453095 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fdf381ad830 con 0x7fdf3810cbe0 2024-06-11T14:01:30.213 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:30.210+0000 7fdf2effd640 1 -- 172.21.15.57:0/3414453095 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7fdf280011a0 con 0x7fdf3810cbe0 2024-06-11T14:01:30.213 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:30.210+0000 7fdf2effd640 1 -- 172.21.15.57:0/3414453095 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fdf28004b70 con 0x7fdf3810cbe0 2024-06-11T14:01:30.213 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:30.211+0000 7fdf40246640 1 -- 172.21.15.57:0/3414453095 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf1c005080 con 0x7fdf3810cbe0 2024-06-11T14:01:30.213 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:30.211+0000 7fdf2effd640 1 -- 172.21.15.57:0/3414453095 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 22) ==== 92138+0+0 (unknown 311317106 0 0) 0x7fdf28001ab0 con 0x7fdf3810cbe0 2024-06-11T14:01:30.213 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:30.212+0000 7fdf2effd640 1 -- 172.21.15.57:0/3414453095 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(776..776 src has 265..776) ==== 7443+0+0 (unknown 2826644399 0 0) 0x7fdf280897e0 con 0x7fdf3810cbe0 2024-06-11T14:01:30.216 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:30.217+0000 7fdf2effd640 1 -- 172.21.15.57:0/3414453095 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7fdf28056a40 con 0x7fdf3810cbe0 2024-06-11T14:01:30.405 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:30.406+0000 7fdf40246640 1 -- 172.21.15.57:0/3414453095 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "osd pool create", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "pg_num": 12} v 0) -- 0x7fdf1c005370 con 0x7fdf3810cbe0 2024-06-11T14:01:31.001 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.002+0000 7fdf2effd640 1 -- 172.21.15.57:0/3414453095 <== mon.0 v1:172.21.15.57:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "pg_num": 12}]=0 pool '5bd44950-56a4-47a9-a376-eee5f24dbed6' created v777) ==== 176+0+0 (unknown 1055779756 0 0) 0x7fdf2805a6f0 con 0x7fdf3810cbe0 2024-06-11T14:01:31.125 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.125+0000 7fdf40246640 1 -- 172.21.15.57:0/3414453095 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "osd pool create", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "pg_num": 12} v 0) -- 0x7fdf1c001170 con 0x7fdf3810cbe0 2024-06-11T14:01:31.125 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.125+0000 7fdf2effd640 1 -- 172.21.15.57:0/3414453095 <== mon.0 v1:172.21.15.57:6789/0 11 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "pg_num": 12}]=0 pool '5bd44950-56a4-47a9-a376-eee5f24dbed6' already exists v777) ==== 183+0+0 (unknown 1486975912 0 0) 0x7fdf1c001170 con 0x7fdf3810cbe0 2024-06-11T14:01:31.125 INFO:tasks.workunit.client.0.smithi057.stderr:pool '5bd44950-56a4-47a9-a376-eee5f24dbed6' already exists 2024-06-11T14:01:31.127 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.127+0000 7fdf40246640 1 -- 172.21.15.57:0/3414453095 >> v1:172.21.15.57:6800/2077416415 conn(0x7fdf2406eee0 legacy=0x7fdf240713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:01:31.127 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.127+0000 7fdf40246640 1 -- 172.21.15.57:0/3414453095 >> v1:172.21.15.57:6789/0 conn(0x7fdf3810cbe0 legacy=0x7fdf38109b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:01:31.127 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.127+0000 7fdf40246640 1 -- 172.21.15.57:0/3414453095 shutdown_connections 2024-06-11T14:01:31.127 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.127+0000 7fdf40246640 1 -- 172.21.15.57:0/3414453095 >> 172.21.15.57:0/3414453095 conn(0x7fdf380fc230 msgr2=0x7fdf380fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T14:01:31.127 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.127+0000 7fdf40246640 1 -- 172.21.15.57:0/3414453095 shutdown_connections 2024-06-11T14:01:31.127 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.127+0000 7fdf40246640 1 -- 172.21.15.57:0/3414453095 wait complete. 2024-06-11T14:01:31.136 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph osd pool set-quota 5bd44950-56a4-47a9-a376-eee5f24dbed6 max_objects 10 2024-06-11T14:01:31.222 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.223+0000 7f5a58fc9640 1 -- 172.21.15.57:0/1344365175 >> v1:172.21.15.57:6789/0 conn(0x7f5a54073d50 legacy=0x7f5a54074130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:01:31.223 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.223+0000 7f5a58fc9640 1 -- 172.21.15.57:0/1344365175 shutdown_connections 2024-06-11T14:01:31.223 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.223+0000 7f5a58fc9640 1 -- 172.21.15.57:0/1344365175 >> 172.21.15.57:0/1344365175 conn(0x7f5a540fd2c0 msgr2=0x7f5a540ff6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T14:01:31.223 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.223+0000 7f5a58fc9640 1 -- 172.21.15.57:0/1344365175 shutdown_connections 2024-06-11T14:01:31.223 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.223+0000 7f5a58fc9640 1 -- 172.21.15.57:0/1344365175 wait complete. 2024-06-11T14:01:31.226 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.224+0000 7f5a58fc9640 1 Processor -- start 2024-06-11T14:01:31.226 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.224+0000 7f5a58fc9640 1 -- start start 2024-06-11T14:01:31.226 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.224+0000 7f5a58fc9640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f5a541a8820 con 0x7f5a541104b0 2024-06-11T14:01:31.226 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.224+0000 7f5a58fc9640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f5a541a9a40 con 0x7f5a54073d50 2024-06-11T14:01:31.226 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.224+0000 7f5a58fc9640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f5a541aac80 con 0x7f5a54074840 2024-06-11T14:01:31.226 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.224+0000 7f5a52d9d640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f5a541104b0 0x7f5a541a7010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:58272/0 (socket says 172.21.15.57:58272) 2024-06-11T14:01:31.226 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.224+0000 7f5a51d9b640 1 --1- >> v1:172.21.15.57:6790/0 conn(0x7f5a54074840 0x7f5a541a38e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6790/0 says I am v1:172.21.15.57:52004/0 (socket says 172.21.15.57:52004) 2024-06-11T14:01:31.226 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.224+0000 7f5a52d9d640 1 -- 172.21.15.57:0/817736095 learned_addr learned my addr 172.21.15.57:0/817736095 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T14:01:31.226 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.224+0000 7f5a437fe640 1 -- 172.21.15.57:0/817736095 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2642105734 0 0) 0x7f5a541a8820 con 0x7f5a541104b0 2024-06-11T14:01:31.226 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.224+0000 7f5a437fe640 1 -- 172.21.15.57:0/817736095 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a3c003240 con 0x7f5a541104b0 2024-06-11T14:01:31.226 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.224+0000 7f5a437fe640 1 -- 172.21.15.57:0/817736095 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1508706519 0 0) 0x7f5a541aac80 con 0x7f5a54074840 2024-06-11T14:01:31.226 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.224+0000 7f5a437fe640 1 -- 172.21.15.57:0/817736095 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a541a8820 con 0x7f5a54074840 2024-06-11T14:01:31.226 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.224+0000 7f5a437fe640 1 -- 172.21.15.57:0/817736095 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 150359823 0 0) 0x7f5a541a9a40 con 0x7f5a54073d50 2024-06-11T14:01:31.227 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.224+0000 7f5a437fe640 1 -- 172.21.15.57:0/817736095 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a541aac80 con 0x7f5a54073d50 2024-06-11T14:01:31.227 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.225+0000 7f5a437fe640 1 -- 172.21.15.57:0/817736095 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2889257732 0 0) 0x7f5a3c003240 con 0x7f5a541104b0 2024-06-11T14:01:31.227 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.225+0000 7f5a437fe640 1 -- 172.21.15.57:0/817736095 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5a541a9a40 con 0x7f5a541104b0 2024-06-11T14:01:31.227 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.225+0000 7f5a437fe640 1 -- 172.21.15.57:0/817736095 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f5a4c001750 con 0x7f5a541104b0 2024-06-11T14:01:31.227 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.225+0000 7f5a437fe640 1 -- 172.21.15.57:0/817736095 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 954300282 0 0) 0x7f5a541a8820 con 0x7f5a54074840 2024-06-11T14:01:31.227 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.225+0000 7f5a437fe640 1 -- 172.21.15.57:0/817736095 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5a3c003240 con 0x7f5a54074840 2024-06-11T14:01:31.227 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.225+0000 7f5a437fe640 1 -- 172.21.15.57:0/817736095 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f5a480017b0 con 0x7f5a54074840 2024-06-11T14:01:31.227 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.225+0000 7f5a437fe640 1 -- 172.21.15.57:0/817736095 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1502121772 0 0) 0x7f5a541aac80 con 0x7f5a54073d50 2024-06-11T14:01:31.227 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.225+0000 7f5a437fe640 1 -- 172.21.15.57:0/817736095 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5a541a8820 con 0x7f5a54073d50 2024-06-11T14:01:31.227 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.225+0000 7f5a437fe640 1 -- 172.21.15.57:0/817736095 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f5a440044c0 con 0x7f5a54073d50 2024-06-11T14:01:31.227 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.225+0000 7f5a437fe640 1 -- 172.21.15.57:0/817736095 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3344475961 0 0) 0x7f5a541a9a40 con 0x7f5a541104b0 2024-06-11T14:01:31.227 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.225+0000 7f5a437fe640 1 -- 172.21.15.57:0/817736095 >> v1:172.21.15.57:6790/0 conn(0x7f5a54074840 legacy=0x7f5a541a38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:01:31.227 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.225+0000 7f5a437fe640 1 -- 172.21.15.57:0/817736095 >> v1:172.21.15.107:6789/0 conn(0x7f5a54073d50 legacy=0x7f5a5410e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:01:31.227 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.225+0000 7f5a437fe640 1 -- 172.21.15.57:0/817736095 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a541abec0 con 0x7f5a541104b0 2024-06-11T14:01:31.227 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.225+0000 7f5a58fc9640 1 -- 172.21.15.57:0/817736095 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f5a541aaeb0 con 0x7f5a541104b0 2024-06-11T14:01:31.227 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.225+0000 7f5a58fc9640 1 -- 172.21.15.57:0/817736095 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f5a541ab380 con 0x7f5a541104b0 2024-06-11T14:01:31.228 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.225+0000 7f5a437fe640 1 -- 172.21.15.57:0/817736095 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f5a4c0020b0 con 0x7f5a541104b0 2024-06-11T14:01:31.228 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.225+0000 7f5a437fe640 1 -- 172.21.15.57:0/817736095 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f5a4c005e90 con 0x7f5a541104b0 2024-06-11T14:01:31.228 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.226+0000 7f5a58fc9640 1 -- 172.21.15.57:0/817736095 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a34005080 con 0x7f5a541104b0 2024-06-11T14:01:31.228 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.227+0000 7f5a437fe640 1 -- 172.21.15.57:0/817736095 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 22) ==== 92138+0+0 (unknown 311317106 0 0) 0x7f5a4c003c20 con 0x7f5a541104b0 2024-06-11T14:01:31.228 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.227+0000 7f5a437fe640 1 -- 172.21.15.57:0/817736095 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(777..777 src has 265..777) ==== 7818+0+0 (unknown 874419557 0 0) 0x7f5a4c089d30 con 0x7f5a541104b0 2024-06-11T14:01:31.230 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.231+0000 7f5a437fe640 1 -- 172.21.15.57:0/817736095 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f5a4c056e80 con 0x7f5a541104b0 2024-06-11T14:01:31.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:30 smithi057 ceph-mon[29805]: pgmap v1689: 196 pgs: 196 active+clean; 583 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:01:31.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:30 smithi057 ceph-mon[29805]: osdmap e776: 8 total, 8 up, 8 in 2024-06-11T14:01:31.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:30 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3414453095' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "pg_num": 12}]: dispatch 2024-06-11T14:01:31.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:30 smithi057 ceph-mon[40588]: pgmap v1689: 196 pgs: 196 active+clean; 583 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:01:31.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:30 smithi057 ceph-mon[40588]: osdmap e776: 8 total, 8 up, 8 in 2024-06-11T14:01:31.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:30 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3414453095' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "pg_num": 12}]: dispatch 2024-06-11T14:01:31.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:30 smithi107 ceph-mon[33178]: pgmap v1689: 196 pgs: 196 active+clean; 583 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:01:31.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:30 smithi107 ceph-mon[33178]: osdmap e776: 8 total, 8 up, 8 in 2024-06-11T14:01:31.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:30 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3414453095' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "pg_num": 12}]: dispatch 2024-06-11T14:01:31.415 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:31.416+0000 7f5a58fc9640 1 -- 172.21.15.57:0/817736095 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "10"} v 0) -- 0x7f5a34005370 con 0x7f5a541104b0 2024-06-11T14:01:32.006 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:32.007+0000 7f5a437fe640 1 -- 172.21.15.57:0/817736095 <== mon.0 v1:172.21.15.57:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 5bd44950-56a4-47a9-a376-eee5f24dbed6 v778) ==== 223+0+0 (unknown 303467086 0 0) 0x7f5a4c05ab30 con 0x7f5a541104b0 2024-06-11T14:01:32.124 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:32.125+0000 7f5a58fc9640 1 -- 172.21.15.57:0/817736095 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "10"} v 0) -- 0x7f5a34001090 con 0x7f5a541104b0 2024-06-11T14:01:32.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:32 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3414453095' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "pg_num": 12}]': finished 2024-06-11T14:01:32.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:32 smithi057 ceph-mon[29805]: osdmap e777: 8 total, 8 up, 8 in 2024-06-11T14:01:32.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:32 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3414453095' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "pg_num": 12}]: dispatch 2024-06-11T14:01:32.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:32 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/817736095' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "10"}]: dispatch 2024-06-11T14:01:32.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:32 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3414453095' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "pg_num": 12}]': finished 2024-06-11T14:01:32.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:32 smithi057 ceph-mon[40588]: osdmap e777: 8 total, 8 up, 8 in 2024-06-11T14:01:32.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:32 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3414453095' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "pg_num": 12}]: dispatch 2024-06-11T14:01:32.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:32 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/817736095' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "10"}]: dispatch 2024-06-11T14:01:32.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:32 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3414453095' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "pg_num": 12}]': finished 2024-06-11T14:01:32.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:32 smithi107 ceph-mon[33178]: osdmap e777: 8 total, 8 up, 8 in 2024-06-11T14:01:32.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:32 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3414453095' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "pg_num": 12}]: dispatch 2024-06-11T14:01:32.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:32 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/817736095' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "10"}]: dispatch 2024-06-11T14:01:33.016 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:33.017+0000 7f5a437fe640 1 -- 172.21.15.57:0/817736095 <== mon.0 v1:172.21.15.57:6789/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 5bd44950-56a4-47a9-a376-eee5f24dbed6 v779) ==== 223+0+0 (unknown 1532178238 0 0) 0x7f5a4c05f3f0 con 0x7f5a541104b0 2024-06-11T14:01:33.017 INFO:tasks.workunit.client.0.smithi057.stderr:set-quota max_objects = 10 for pool 5bd44950-56a4-47a9-a376-eee5f24dbed6 2024-06-11T14:01:33.020 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:33.020+0000 7f5a58fc9640 1 -- 172.21.15.57:0/817736095 >> v1:172.21.15.57:6800/2077416415 conn(0x7f5a3c06f000 legacy=0x7f5a3c0714c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:01:33.020 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:33.020+0000 7f5a58fc9640 1 -- 172.21.15.57:0/817736095 >> v1:172.21.15.57:6789/0 conn(0x7f5a541104b0 legacy=0x7f5a541a7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:01:33.020 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:33.020+0000 7f5a58fc9640 1 -- 172.21.15.57:0/817736095 shutdown_connections 2024-06-11T14:01:33.020 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:33.020+0000 7f5a58fc9640 1 -- 172.21.15.57:0/817736095 >> 172.21.15.57:0/817736095 conn(0x7f5a540fd2c0 msgr2=0x7f5a540ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T14:01:33.020 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:33.020+0000 7f5a58fc9640 1 -- 172.21.15.57:0/817736095 shutdown_connections 2024-06-11T14:01:33.020 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:33.020+0000 7f5a58fc9640 1 -- 172.21.15.57:0/817736095 wait complete. 2024-06-11T14:01:33.035 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph osd pool application enable 5bd44950-56a4-47a9-a376-eee5f24dbed6 rados 2024-06-11T14:01:33.126 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:33.127+0000 7ff03d9c9640 1 -- 172.21.15.57:0/312394639 >> v1:172.21.15.57:6790/0 conn(0x7ff03810e300 legacy=0x7ff038110790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:01:33.127 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:33.127+0000 7ff03d9c9640 1 -- 172.21.15.57:0/312394639 shutdown_connections 2024-06-11T14:01:33.127 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:33.127+0000 7ff03d9c9640 1 -- 172.21.15.57:0/312394639 >> 172.21.15.57:0/312394639 conn(0x7ff0380fc230 msgr2=0x7ff0380fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T14:01:33.127 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:33.127+0000 7ff03d9c9640 1 -- 172.21.15.57:0/312394639 shutdown_connections 2024-06-11T14:01:33.127 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:33.127+0000 7ff03d9c9640 1 -- 172.21.15.57:0/312394639 wait complete. 2024-06-11T14:01:33.127 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:33.127+0000 7ff03d9c9640 1 Processor -- start 2024-06-11T14:01:33.129 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:33.128+0000 7ff03d9c9640 1 -- start start 2024-06-11T14:01:33.129 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:33.128+0000 7ff03d9c9640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ff0381a6960 con 0x7ff0381029f0 2024-06-11T14:01:33.129 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:33.128+0000 7ff03d9c9640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ff0381a7b60 con 0x7ff03810e300 2024-06-11T14:01:33.129 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:33.128+0000 7ff03d9c9640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ff0381a8da0 con 0x7ff03810a7f0 2024-06-11T14:01:33.129 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:33.128+0000 7ff036ffd640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7ff0381029f0 0x7ff03810a0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:58294/0 (socket says 172.21.15.57:58294) 2024-06-11T14:01:33.129 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:33.128+0000 7ff036ffd640 1 -- 172.21.15.57:0/210433587 learned_addr learned my addr 172.21.15.57:0/210433587 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T14:01:33.129 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:33.128+0000 7ff0367fc640 1 --1- 172.21.15.57:0/210433587 >> v1:172.21.15.57:6790/0 conn(0x7ff03810a7f0 0x7ff0381a1700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6790/0 says I am v1:172.21.15.57:52018/0 (socket says 172.21.15.57:52018) 2024-06-11T14:01:33.130 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:33.128+0000 7ff03c9c7640 1 -- 172.21.15.57:0/210433587 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2926297404 0 0) 0x7ff0381a6960 con 0x7ff0381029f0 2024-06-11T14:01:33.130 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:33.128+0000 7ff03c9c7640 1 -- 172.21.15.57:0/210433587 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff020003520 con 0x7ff0381029f0 2024-06-11T14:01:33.130 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:33.128+0000 7ff03c9c7640 1 -- 172.21.15.57:0/210433587 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1675231550 0 0) 0x7ff0381a8da0 con 0x7ff03810a7f0 2024-06-11T14:01:33.130 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:33.128+0000 7ff03c9c7640 1 -- 172.21.15.57:0/210433587 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0381a6960 con 0x7ff03810a7f0 2024-06-11T14:01:33.130 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:33.128+0000 7ff03c9c7640 1 -- 172.21.15.57:0/210433587 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 292241448 0 0) 0x7ff020003520 con 0x7ff0381029f0 2024-06-11T14:01:33.130 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:33.128+0000 7ff03c9c7640 1 -- 172.21.15.57:0/210433587 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff0381a8da0 con 0x7ff0381029f0 2024-06-11T14:01:33.130 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:33.128+0000 7ff03c9c7640 1 -- 172.21.15.57:0/210433587 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7ff028001930 con 0x7ff0381029f0 2024-06-11T14:01:33.130 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:33.128+0000 7ff03c9c7640 1 -- 172.21.15.57:0/210433587 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3239025804 0 0) 0x7ff0381a6960 con 0x7ff03810a7f0 2024-06-11T14:01:33.130 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:33.128+0000 7ff03c9c7640 1 -- 172.21.15.57:0/210433587 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff020003520 con 0x7ff03810a7f0 2024-06-11T14:01:33.130 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:33.128+0000 7ff03c9c7640 1 -- 172.21.15.57:0/210433587 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1449032824 0 0) 0x7ff0381a7b60 con 0x7ff03810e300 2024-06-11T14:01:33.130 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:33.128+0000 7ff03c9c7640 1 -- 172.21.15.57:0/210433587 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0381a6960 con 0x7ff03810e300 2024-06-11T14:01:33.130 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:33.128+0000 7ff03c9c7640 1 -- 172.21.15.57:0/210433587 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7ff02c001890 con 0x7ff03810a7f0 2024-06-11T14:01:33.131 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:33.129+0000 7ff03c9c7640 1 -- 172.21.15.57:0/210433587 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 953068438 0 0) 0x7ff0381a8da0 con 0x7ff0381029f0 2024-06-11T14:01:33.131 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:33.129+0000 7ff03c9c7640 1 -- 172.21.15.57:0/210433587 >> v1:172.21.15.57:6790/0 conn(0x7ff03810a7f0 legacy=0x7ff0381a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:01:33.131 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:33.129+0000 7ff03c9c7640 1 -- 172.21.15.57:0/210433587 >> v1:172.21.15.107:6789/0 conn(0x7ff03810e300 legacy=0x7ff0381a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:01:33.131 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:33.129+0000 7ff03c9c7640 1 -- 172.21.15.57:0/210433587 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0381a9fe0 con 0x7ff0381029f0 2024-06-11T14:01:33.131 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:33.129+0000 7ff03d9c9640 1 -- 172.21.15.57:0/210433587 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7ff0381a8fd0 con 0x7ff0381029f0 2024-06-11T14:01:33.131 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:33.129+0000 7ff03d9c9640 1 -- 172.21.15.57:0/210433587 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7ff0381a94a0 con 0x7ff0381029f0 2024-06-11T14:01:33.131 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:33.129+0000 7ff03c9c7640 1 -- 172.21.15.57:0/210433587 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7ff028003aa0 con 0x7ff0381029f0 2024-06-11T14:01:33.131 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:33.129+0000 7ff03c9c7640 1 -- 172.21.15.57:0/210433587 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7ff028005ca0 con 0x7ff0381029f0 2024-06-11T14:01:33.131 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:33.130+0000 7ff03d9c9640 1 -- 172.21.15.57:0/210433587 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff024005080 con 0x7ff0381029f0 2024-06-11T14:01:33.131 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:33.130+0000 7ff03c9c7640 1 -- 172.21.15.57:0/210433587 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 22) ==== 92138+0+0 (unknown 311317106 0 0) 0x7ff0280035f0 con 0x7ff0381029f0 2024-06-11T14:01:33.132 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:33.131+0000 7ff03c9c7640 1 -- 172.21.15.57:0/210433587 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(779..779 src has 265..779) ==== 7818+0+0 (unknown 3745780821 0 0) 0x7ff028089d60 con 0x7ff0381029f0 2024-06-11T14:01:33.134 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:33.135+0000 7ff03c9c7640 1 -- 172.21.15.57:0/210433587 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7ff028056e50 con 0x7ff0381029f0 2024-06-11T14:01:33.201 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:33 smithi107 ceph-mon[33178]: pgmap v1692: 176 pgs: 12 unknown, 164 active+clean; 583 KiB data, 910 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T14:01:33.201 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:33 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/817736095' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "10"}]': finished 2024-06-11T14:01:33.201 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:33 smithi107 ceph-mon[33178]: osdmap e778: 8 total, 8 up, 8 in 2024-06-11T14:01:33.201 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:33 smithi107 ceph-mon[33178]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T14:01:33.201 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:33 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/817736095' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "10"}]: dispatch 2024-06-11T14:01:33.202 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 14:01:32 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T14:01:33.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:33 smithi057 ceph-mon[29805]: pgmap v1692: 176 pgs: 12 unknown, 164 active+clean; 583 KiB data, 910 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T14:01:33.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:33 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/817736095' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "10"}]': finished 2024-06-11T14:01:33.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:33 smithi057 ceph-mon[29805]: osdmap e778: 8 total, 8 up, 8 in 2024-06-11T14:01:33.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:33 smithi057 ceph-mon[29805]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T14:01:33.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:33 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/817736095' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "10"}]: dispatch 2024-06-11T14:01:33.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:33 smithi057 ceph-mon[40588]: pgmap v1692: 176 pgs: 12 unknown, 164 active+clean; 583 KiB data, 910 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T14:01:33.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:33 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/817736095' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "10"}]': finished 2024-06-11T14:01:33.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:33 smithi057 ceph-mon[40588]: osdmap e778: 8 total, 8 up, 8 in 2024-06-11T14:01:33.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:33 smithi057 ceph-mon[40588]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T14:01:33.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:33 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/817736095' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "10"}]: dispatch 2024-06-11T14:01:33.324 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:33.325+0000 7ff03d9c9640 1 -- 172.21.15.57:0/210433587 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "osd pool application enable", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "app": "rados"} v 0) -- 0x7ff024005370 con 0x7ff0381029f0 2024-06-11T14:01:34.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 14:01:33 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:14:01:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T14:01:34.039 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:34.039+0000 7ff03c9c7640 1 -- 172.21.15.57:0/210433587 <== mon.0 v1:172.21.15.57:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "app": "rados"}]=0 enabled application 'rados' on pool '5bd44950-56a4-47a9-a376-eee5f24dbed6' v780) ==== 213+0+0 (unknown 246865507 0 0) 0x7ff02805ab00 con 0x7ff0381029f0 2024-06-11T14:01:34.156 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:34.156+0000 7ff03d9c9640 1 -- 172.21.15.57:0/210433587 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "osd pool application enable", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "app": "rados"} v 0) -- 0x7ff024001090 con 0x7ff0381029f0 2024-06-11T14:01:34.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:34 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:01:34.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:34 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/817736095' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "10"}]': finished 2024-06-11T14:01:34.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:34 smithi107 ceph-mon[33178]: osdmap e779: 8 total, 8 up, 8 in 2024-06-11T14:01:34.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:34 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/210433587' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "app": "rados"}]: dispatch 2024-06-11T14:01:34.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:34 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:01:34.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:34 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/817736095' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "10"}]': finished 2024-06-11T14:01:34.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:34 smithi057 ceph-mon[29805]: osdmap e779: 8 total, 8 up, 8 in 2024-06-11T14:01:34.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:34 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/210433587' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "app": "rados"}]: dispatch 2024-06-11T14:01:34.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:34 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:01:34.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:34 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/817736095' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "10"}]': finished 2024-06-11T14:01:34.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:34 smithi057 ceph-mon[40588]: osdmap e779: 8 total, 8 up, 8 in 2024-06-11T14:01:34.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:34 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/210433587' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "app": "rados"}]: dispatch 2024-06-11T14:01:35.041 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:35.041+0000 7ff03c9c7640 1 -- 172.21.15.57:0/210433587 <== mon.0 v1:172.21.15.57:6789/0 11 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "app": "rados"}]=0 enabled application 'rados' on pool '5bd44950-56a4-47a9-a376-eee5f24dbed6' v781) ==== 213+0+0 (unknown 882321541 0 0) 0x7ff02805f3c0 con 0x7ff0381029f0 2024-06-11T14:01:35.041 INFO:tasks.workunit.client.0.smithi057.stderr:enabled application 'rados' on pool '5bd44950-56a4-47a9-a376-eee5f24dbed6' 2024-06-11T14:01:35.044 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:35.044+0000 7ff03d9c9640 1 -- 172.21.15.57:0/210433587 >> v1:172.21.15.57:6800/2077416415 conn(0x7ff02006eec0 legacy=0x7ff020071380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:01:35.044 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:35.044+0000 7ff03d9c9640 1 -- 172.21.15.57:0/210433587 >> v1:172.21.15.57:6789/0 conn(0x7ff0381029f0 legacy=0x7ff03810a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:01:35.044 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:35.044+0000 7ff03d9c9640 1 -- 172.21.15.57:0/210433587 shutdown_connections 2024-06-11T14:01:35.044 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:35.044+0000 7ff03d9c9640 1 -- 172.21.15.57:0/210433587 >> 172.21.15.57:0/210433587 conn(0x7ff0380fc230 msgr2=0x7ff0380fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T14:01:35.044 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:35.044+0000 7ff03d9c9640 1 -- 172.21.15.57:0/210433587 shutdown_connections 2024-06-11T14:01:35.044 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:01:35.044+0000 7ff03d9c9640 1 -- 172.21.15.57:0/210433587 wait complete. 2024-06-11T14:01:35.058 INFO:tasks.workunit.client.0.smithi057.stderr:++ seq 1 10 2024-06-11T14:01:35.073 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in `seq 1 10` 2024-06-11T14:01:35.073 INFO:tasks.workunit.client.0.smithi057.stderr:+ rados -p 5bd44950-56a4-47a9-a376-eee5f24dbed6 put obj1 /etc/passwd 2024-06-11T14:01:35.110 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in `seq 1 10` 2024-06-11T14:01:35.110 INFO:tasks.workunit.client.0.smithi057.stderr:+ rados -p 5bd44950-56a4-47a9-a376-eee5f24dbed6 put obj2 /etc/passwd 2024-06-11T14:01:35.147 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in `seq 1 10` 2024-06-11T14:01:35.147 INFO:tasks.workunit.client.0.smithi057.stderr:+ rados -p 5bd44950-56a4-47a9-a376-eee5f24dbed6 put obj3 /etc/passwd 2024-06-11T14:01:35.182 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in `seq 1 10` 2024-06-11T14:01:35.182 INFO:tasks.workunit.client.0.smithi057.stderr:+ rados -p 5bd44950-56a4-47a9-a376-eee5f24dbed6 put obj4 /etc/passwd 2024-06-11T14:01:35.216 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in `seq 1 10` 2024-06-11T14:01:35.216 INFO:tasks.workunit.client.0.smithi057.stderr:+ rados -p 5bd44950-56a4-47a9-a376-eee5f24dbed6 put obj5 /etc/passwd 2024-06-11T14:01:35.251 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in `seq 1 10` 2024-06-11T14:01:35.251 INFO:tasks.workunit.client.0.smithi057.stderr:+ rados -p 5bd44950-56a4-47a9-a376-eee5f24dbed6 put obj6 /etc/passwd 2024-06-11T14:01:35.286 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in `seq 1 10` 2024-06-11T14:01:35.286 INFO:tasks.workunit.client.0.smithi057.stderr:+ rados -p 5bd44950-56a4-47a9-a376-eee5f24dbed6 put obj7 /etc/passwd 2024-06-11T14:01:35.320 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in `seq 1 10` 2024-06-11T14:01:35.320 INFO:tasks.workunit.client.0.smithi057.stderr:+ rados -p 5bd44950-56a4-47a9-a376-eee5f24dbed6 put obj8 /etc/passwd 2024-06-11T14:01:35.355 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in `seq 1 10` 2024-06-11T14:01:35.355 INFO:tasks.workunit.client.0.smithi057.stderr:+ rados -p 5bd44950-56a4-47a9-a376-eee5f24dbed6 put obj9 /etc/passwd 2024-06-11T14:01:35.389 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in `seq 1 10` 2024-06-11T14:01:35.389 INFO:tasks.workunit.client.0.smithi057.stderr:+ rados -p 5bd44950-56a4-47a9-a376-eee5f24dbed6 put obj10 /etc/passwd 2024-06-11T14:01:35.423 INFO:tasks.workunit.client.0.smithi057.stderr:+ sleep 30 2024-06-11T14:01:35.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:35 smithi057 ceph-mon[29805]: pgmap v1695: 176 pgs: 12 unknown, 164 active+clean; 583 KiB data, 910 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-11T14:01:35.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:35 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/210433587' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "app": "rados"}]': finished 2024-06-11T14:01:35.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:35 smithi057 ceph-mon[29805]: osdmap e780: 8 total, 8 up, 8 in 2024-06-11T14:01:35.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:35 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/210433587' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "app": "rados"}]: dispatch 2024-06-11T14:01:35.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:35 smithi057 ceph-mon[40588]: pgmap v1695: 176 pgs: 12 unknown, 164 active+clean; 583 KiB data, 910 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-11T14:01:35.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:35 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/210433587' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "app": "rados"}]': finished 2024-06-11T14:01:35.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:35 smithi057 ceph-mon[40588]: osdmap e780: 8 total, 8 up, 8 in 2024-06-11T14:01:35.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:35 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/210433587' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "app": "rados"}]: dispatch 2024-06-11T14:01:35.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:35 smithi107 ceph-mon[33178]: pgmap v1695: 176 pgs: 12 unknown, 164 active+clean; 583 KiB data, 910 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-11T14:01:35.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:35 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/210433587' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "app": "rados"}]': finished 2024-06-11T14:01:35.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:35 smithi107 ceph-mon[33178]: osdmap e780: 8 total, 8 up, 8 in 2024-06-11T14:01:35.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:35 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/210433587' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "app": "rados"}]: dispatch 2024-06-11T14:01:36.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:36 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/210433587' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "app": "rados"}]': finished 2024-06-11T14:01:36.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:36 smithi057 ceph-mon[29805]: osdmap e781: 8 total, 8 up, 8 in 2024-06-11T14:01:36.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:36 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/210433587' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "app": "rados"}]': finished 2024-06-11T14:01:36.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:36 smithi057 ceph-mon[40588]: osdmap e781: 8 total, 8 up, 8 in 2024-06-11T14:01:36.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:36 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/210433587' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "app": "rados"}]': finished 2024-06-11T14:01:36.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:36 smithi107 ceph-mon[33178]: osdmap e781: 8 total, 8 up, 8 in 2024-06-11T14:01:37.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:37 smithi057 ceph-mon[29805]: pgmap v1698: 176 pgs: 12 unknown, 164 active+clean; 583 KiB data, 910 MiB used, 714 GiB / 715 GiB avail 2024-06-11T14:01:37.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:37 smithi057 ceph-mon[40588]: pgmap v1698: 176 pgs: 12 unknown, 164 active+clean; 583 KiB data, 910 MiB used, 714 GiB / 715 GiB avail 2024-06-11T14:01:37.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:37 smithi107 ceph-mon[33178]: pgmap v1698: 176 pgs: 12 unknown, 164 active+clean; 583 KiB data, 910 MiB used, 714 GiB / 715 GiB avail 2024-06-11T14:01:38.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:38 smithi057 ceph-mon[40588]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T14:01:38.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:38 smithi057 ceph-mon[29805]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T14:01:38.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:38 smithi107 ceph-mon[33178]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T14:01:39.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:39 smithi057 ceph-mon[29805]: pgmap v1699: 176 pgs: 176 active+clean; 701 KiB data, 904 MiB used, 714 GiB / 715 GiB avail; 868 B/s rd, 20 KiB/s wr, 2 op/s 2024-06-11T14:01:39.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:39 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:01:39.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:39 smithi057 ceph-mon[40588]: pgmap v1699: 176 pgs: 176 active+clean; 701 KiB data, 904 MiB used, 714 GiB / 715 GiB avail; 868 B/s rd, 20 KiB/s wr, 2 op/s 2024-06-11T14:01:39.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:39 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:01:39.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:39 smithi107 ceph-mon[33178]: pgmap v1699: 176 pgs: 176 active+clean; 701 KiB data, 904 MiB used, 714 GiB / 715 GiB avail; 868 B/s rd, 20 KiB/s wr, 2 op/s 2024-06-11T14:01:39.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:39 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:01:40.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:40 smithi057 ceph-mon[29805]: pgmap v1700: 176 pgs: 176 active+clean; 701 KiB data, 904 MiB used, 714 GiB / 715 GiB avail; 743 B/s rd, 17 KiB/s wr, 2 op/s 2024-06-11T14:01:40.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:40 smithi057 ceph-mon[40588]: pgmap v1700: 176 pgs: 176 active+clean; 701 KiB data, 904 MiB used, 714 GiB / 715 GiB avail; 743 B/s rd, 17 KiB/s wr, 2 op/s 2024-06-11T14:01:40.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:40 smithi107 ceph-mon[33178]: pgmap v1700: 176 pgs: 176 active+clean; 701 KiB data, 904 MiB used, 714 GiB / 715 GiB avail; 743 B/s rd, 17 KiB/s wr, 2 op/s 2024-06-11T14:01:43.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:42 smithi057 ceph-mon[29805]: pgmap v1701: 176 pgs: 176 active+clean; 701 KiB data, 904 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 15 KiB/s wr, 2 op/s 2024-06-11T14:01:43.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:42 smithi057 ceph-mon[29805]: pool '5bd44950-56a4-47a9-a376-eee5f24dbed6' is full (reached quota's max_objects: 10) 2024-06-11T14:01:43.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:42 smithi057 ceph-mon[29805]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-06-11T14:01:43.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:42 smithi057 ceph-mon[29805]: osdmap e782: 8 total, 8 up, 8 in 2024-06-11T14:01:43.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:42 smithi057 ceph-mon[40588]: pgmap v1701: 176 pgs: 176 active+clean; 701 KiB data, 904 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 15 KiB/s wr, 2 op/s 2024-06-11T14:01:43.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:42 smithi057 ceph-mon[40588]: pool '5bd44950-56a4-47a9-a376-eee5f24dbed6' is full (reached quota's max_objects: 10) 2024-06-11T14:01:43.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:42 smithi057 ceph-mon[40588]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-06-11T14:01:43.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:42 smithi057 ceph-mon[40588]: osdmap e782: 8 total, 8 up, 8 in 2024-06-11T14:01:43.299 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 14:01:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T14:01:43.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:42 smithi107 ceph-mon[33178]: pgmap v1701: 176 pgs: 176 active+clean; 701 KiB data, 904 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 15 KiB/s wr, 2 op/s 2024-06-11T14:01:43.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:42 smithi107 ceph-mon[33178]: pool '5bd44950-56a4-47a9-a376-eee5f24dbed6' is full (reached quota's max_objects: 10) 2024-06-11T14:01:43.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:42 smithi107 ceph-mon[33178]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-06-11T14:01:43.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:42 smithi107 ceph-mon[33178]: osdmap e782: 8 total, 8 up, 8 in 2024-06-11T14:01:44.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:43 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:01:44.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:43 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:01:44.037 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 14:01:43 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:14:01:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T14:01:44.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:43 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:01:45.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:44 smithi057 ceph-mon[29805]: pgmap v1703: 176 pgs: 176 active+clean; 701 KiB data, 904 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 14 KiB/s wr, 2 op/s 2024-06-11T14:01:45.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:44 smithi057 ceph-mon[40588]: pgmap v1703: 176 pgs: 176 active+clean; 701 KiB data, 904 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 14 KiB/s wr, 2 op/s 2024-06-11T14:01:45.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:44 smithi107 ceph-mon[33178]: pgmap v1703: 176 pgs: 176 active+clean; 701 KiB data, 904 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 14 KiB/s wr, 2 op/s 2024-06-11T14:01:47.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:46 smithi057 ceph-mon[40588]: pgmap v1704: 176 pgs: 176 active+clean; 701 KiB data, 904 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-06-11T14:01:47.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:46 smithi057 ceph-mon[29805]: pgmap v1704: 176 pgs: 176 active+clean; 701 KiB data, 904 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-06-11T14:01:47.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:46 smithi107 ceph-mon[33178]: pgmap v1704: 176 pgs: 176 active+clean; 701 KiB data, 904 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-06-11T14:01:49.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:48 smithi057 ceph-mon[40588]: pgmap v1705: 176 pgs: 176 active+clean; 701 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T14:01:49.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:48 smithi057 ceph-mon[29805]: pgmap v1705: 176 pgs: 176 active+clean; 701 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T14:01:49.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:48 smithi107 ceph-mon[33178]: pgmap v1705: 176 pgs: 176 active+clean; 701 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T14:01:51.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:51 smithi057 ceph-mon[29805]: pgmap v1706: 176 pgs: 176 active+clean; 701 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T14:01:51.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:51 smithi057 ceph-mon[40588]: pgmap v1706: 176 pgs: 176 active+clean; 701 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T14:01:51.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:51 smithi107 ceph-mon[33178]: pgmap v1706: 176 pgs: 176 active+clean; 701 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T14:01:53.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:53 smithi057 ceph-mon[29805]: pgmap v1707: 176 pgs: 176 active+clean; 701 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T14:01:53.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:53 smithi057 ceph-mon[40588]: pgmap v1707: 176 pgs: 176 active+clean; 701 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T14:01:53.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:53 smithi107 ceph-mon[33178]: pgmap v1707: 176 pgs: 176 active+clean; 701 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T14:01:53.300 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 14:01:52 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T14:01:54.014 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 14:01:53 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:14:01:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T14:01:54.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:54 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:01:54.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:54 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T14:01:54.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:54 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:01:54.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:54 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:01:54.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:54 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T14:01:54.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:54 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:01:54.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:54 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:01:54.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:54 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T14:01:54.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:54 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:01:55.066 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:55 smithi057 ceph-mon[40588]: pgmap v1708: 176 pgs: 176 active+clean; 701 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 905 B/s rd, 0 op/s 2024-06-11T14:01:55.066 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:55 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T14:01:55.066 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:55 smithi057 ceph-mon[29805]: pgmap v1708: 176 pgs: 176 active+clean; 701 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 905 B/s rd, 0 op/s 2024-06-11T14:01:55.066 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:55 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T14:01:55.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:55 smithi107 ceph-mon[33178]: pgmap v1708: 176 pgs: 176 active+clean; 701 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 905 B/s rd, 0 op/s 2024-06-11T14:01:55.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:55 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T14:01:57.137 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:57 smithi057 ceph-mon[40588]: pgmap v1709: 176 pgs: 176 active+clean; 701 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:01:57.137 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:57 smithi057 ceph-mon[29805]: pgmap v1709: 176 pgs: 176 active+clean; 701 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:01:57.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:57 smithi107 ceph-mon[33178]: pgmap v1709: 176 pgs: 176 active+clean; 701 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:01:58.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:58 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T14:01:58.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:58 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T14:01:58.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:58 smithi057 ceph-mon[29805]: pgmap v1710: 176 pgs: 176 active+clean; 701 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:01:58.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:58 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T14:01:58.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:58 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T14:01:58.787 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:58 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T14:01:58.787 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:58 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T14:01:58.787 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:58 smithi057 ceph-mon[40588]: pgmap v1710: 176 pgs: 176 active+clean; 701 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:01:58.787 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:58 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T14:01:58.787 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:58 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T14:01:58.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:58 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T14:01:58.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:58 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T14:01:58.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:58 smithi107 ceph-mon[33178]: pgmap v1710: 176 pgs: 176 active+clean; 701 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:01:58.800 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:58 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T14:01:58.800 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:58 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T14:01:59.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:59 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T14:01:59.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:59 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T14:01:59.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:01:59 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T14:01:59.787 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:59 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T14:01:59.787 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:59 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T14:01:59.787 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:01:59 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T14:01:59.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:59 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T14:01:59.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:59 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T14:01:59.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:01:59 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T14:02:00.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:00 smithi057 ceph-mon[29805]: pgmap v1711: 176 pgs: 176 active+clean; 701 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:02:00.787 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:00 smithi057 ceph-mon[40588]: pgmap v1711: 176 pgs: 176 active+clean; 701 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:02:00.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:00 smithi107 ceph-mon[33178]: pgmap v1711: 176 pgs: 176 active+clean; 701 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:02:03.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:02 smithi057 ceph-mon[29805]: pgmap v1712: 176 pgs: 176 active+clean; 701 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:02:03.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:02 smithi057 ceph-mon[40588]: pgmap v1712: 176 pgs: 176 active+clean; 701 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:02:03.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:02 smithi107 ceph-mon[33178]: pgmap v1712: 176 pgs: 176 active+clean; 701 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:02:03.300 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 14:02:02 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T14:02:03.999 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 14:02:03 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:14:02:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T14:02:04.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:04 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:02:04.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:04 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:02:04.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:04 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:02:05.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:05 smithi057 ceph-mon[29805]: pgmap v1713: 176 pgs: 176 active+clean; 701 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:02:05.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:05 smithi057 ceph-mon[40588]: pgmap v1713: 176 pgs: 176 active+clean; 701 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:02:05.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:05 smithi107 ceph-mon[33178]: pgmap v1713: 176 pgs: 176 active+clean; 701 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:02:05.425 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=154462 2024-06-11T14:02:05.425 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph osd pool set-quota 5bd44950-56a4-47a9-a376-eee5f24dbed6 max_objects 100 2024-06-11T14:02:05.425 INFO:tasks.workunit.client.0.smithi057.stderr:+ rados -p 5bd44950-56a4-47a9-a376-eee5f24dbed6 put onemore /etc/passwd 2024-06-11T14:02:05.524 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:05.523+0000 7f4071713640 1 -- 172.21.15.57:0/948259520 >> v1:172.21.15.57:6789/0 conn(0x7f406c10a6c0 legacy=0x7f406c10cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:02:05.524 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:05.523+0000 7f4071713640 1 -- 172.21.15.57:0/948259520 shutdown_connections 2024-06-11T14:02:05.524 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:05.523+0000 7f4071713640 1 -- 172.21.15.57:0/948259520 >> 172.21.15.57:0/948259520 conn(0x7f406c0fd2c0 msgr2=0x7f406c0ff6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T14:02:05.524 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:05.524+0000 7f4071713640 1 -- 172.21.15.57:0/948259520 shutdown_connections 2024-06-11T14:02:05.524 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:05.524+0000 7f4071713640 1 -- 172.21.15.57:0/948259520 wait complete. 2024-06-11T14:02:05.524 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:05.524+0000 7f4071713640 1 Processor -- start 2024-06-11T14:02:05.524 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:05.524+0000 7f4071713640 1 -- start start 2024-06-11T14:02:05.527 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:05.525+0000 7f4071713640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f406c1a8c00 con 0x7f406c10a6c0 2024-06-11T14:02:05.527 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:05.525+0000 7f4071713640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f406c1a9e40 con 0x7f406c107820 2024-06-11T14:02:05.527 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:05.525+0000 7f4071713640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f406c1ab080 con 0x7f406c10e290 2024-06-11T14:02:05.527 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:05.525+0000 7f406b7fe640 1 --1- >> v1:172.21.15.57:6790/0 conn(0x7f406c10e290 0x7f406c1a7300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6790/0 says I am v1:172.21.15.57:48728/0 (socket says 172.21.15.57:48728) 2024-06-11T14:02:05.527 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:05.525+0000 7f406a7fc640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f406c10a6c0 0x7f406c1a3ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:57142/0 (socket says 172.21.15.57:57142) 2024-06-11T14:02:05.528 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:05.525+0000 7f406b7fe640 1 -- 172.21.15.57:0/2923687971 learned_addr learned my addr 172.21.15.57:0/2923687971 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T14:02:05.528 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:05.525+0000 7f4053fff640 1 -- 172.21.15.57:0/2923687971 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2551446777 0 0) 0x7f406c1ab080 con 0x7f406c10e290 2024-06-11T14:02:05.528 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:05.525+0000 7f4053fff640 1 -- 172.21.15.57:0/2923687971 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4054003520 con 0x7f406c10e290 2024-06-11T14:02:05.528 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:05.525+0000 7f4053fff640 1 -- 172.21.15.57:0/2923687971 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1706660185 0 0) 0x7f406c1a8c00 con 0x7f406c10a6c0 2024-06-11T14:02:05.528 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:05.525+0000 7f4053fff640 1 -- 172.21.15.57:0/2923687971 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f406c1ab080 con 0x7f406c10a6c0 2024-06-11T14:02:05.528 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:05.525+0000 7f4053fff640 1 -- 172.21.15.57:0/2923687971 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4256420927 0 0) 0x7f406c1a9e40 con 0x7f406c107820 2024-06-11T14:02:05.528 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:05.525+0000 7f4053fff640 1 -- 172.21.15.57:0/2923687971 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f406c1a8c00 con 0x7f406c107820 2024-06-11T14:02:05.528 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:05.525+0000 7f4053fff640 1 -- 172.21.15.57:0/2923687971 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3242959167 0 0) 0x7f406c1ab080 con 0x7f406c10a6c0 2024-06-11T14:02:05.528 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:05.525+0000 7f4053fff640 1 -- 172.21.15.57:0/2923687971 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f406c1a9e40 con 0x7f406c10a6c0 2024-06-11T14:02:05.528 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:05.525+0000 7f4053fff640 1 -- 172.21.15.57:0/2923687971 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 703511319 0 0) 0x7f4054003520 con 0x7f406c10e290 2024-06-11T14:02:05.528 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:05.525+0000 7f4053fff640 1 -- 172.21.15.57:0/2923687971 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f406c1ab080 con 0x7f406c10e290 2024-06-11T14:02:05.528 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:05.525+0000 7f4053fff640 1 -- 172.21.15.57:0/2923687971 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f4060001a20 con 0x7f406c10a6c0 2024-06-11T14:02:05.528 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:05.525+0000 7f4053fff640 1 -- 172.21.15.57:0/2923687971 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f4064001bf0 con 0x7f406c10e290 2024-06-11T14:02:05.529 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:05.526+0000 7f4053fff640 1 -- 172.21.15.57:0/2923687971 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1749266496 0 0) 0x7f406c1a8c00 con 0x7f406c107820 2024-06-11T14:02:05.529 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:05.526+0000 7f4053fff640 1 -- 172.21.15.57:0/2923687971 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4054003520 con 0x7f406c107820 2024-06-11T14:02:05.529 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:05.526+0000 7f4053fff640 1 -- 172.21.15.57:0/2923687971 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f405c001800 con 0x7f406c107820 2024-06-11T14:02:05.529 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:05.526+0000 7f4053fff640 1 -- 172.21.15.57:0/2923687971 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2820591403 0 0) 0x7f406c1a9e40 con 0x7f406c10a6c0 2024-06-11T14:02:05.529 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:05.526+0000 7f4053fff640 1 -- 172.21.15.57:0/2923687971 >> v1:172.21.15.57:6790/0 conn(0x7f406c10e290 legacy=0x7f406c1a7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:02:05.529 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:05.526+0000 7f4053fff640 1 -- 172.21.15.57:0/2923687971 >> v1:172.21.15.107:6789/0 conn(0x7f406c107820 legacy=0x7f406c075620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:02:05.529 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:05.526+0000 7f4053fff640 1 -- 172.21.15.57:0/2923687971 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f406c1ac2c0 con 0x7f406c10a6c0 2024-06-11T14:02:05.529 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:05.526+0000 7f4071713640 1 -- 172.21.15.57:0/2923687971 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f406c1ab2b0 con 0x7f406c10a6c0 2024-06-11T14:02:05.529 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:05.526+0000 7f4071713640 1 -- 172.21.15.57:0/2923687971 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f406c1ab780 con 0x7f406c10a6c0 2024-06-11T14:02:05.529 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:05.526+0000 7f4053fff640 1 -- 172.21.15.57:0/2923687971 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f4060001d10 con 0x7f406c10a6c0 2024-06-11T14:02:05.529 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:05.526+0000 7f4053fff640 1 -- 172.21.15.57:0/2923687971 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f4060004b70 con 0x7f406c10a6c0 2024-06-11T14:02:05.529 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:05.527+0000 7f4071713640 1 -- 172.21.15.57:0/2923687971 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4058005080 con 0x7f406c10a6c0 2024-06-11T14:02:05.529 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:05.527+0000 7f4053fff640 1 -- 172.21.15.57:0/2923687971 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 22) ==== 92138+0+0 (unknown 311317106 0 0) 0x7f4060003730 con 0x7f406c10a6c0 2024-06-11T14:02:05.529 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:05.528+0000 7f4053fff640 1 -- 172.21.15.57:0/2923687971 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(782..782 src has 265..782) ==== 7831+0+0 (unknown 82432636 0 0) 0x7f4060089af0 con 0x7f406c10a6c0 2024-06-11T14:02:05.530 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:05.528+0000 7f4053fff640 1 -- 172.21.15.57:0/2923687971 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=783}) -- 0x7f406c1a9e40 con 0x7f406c10a6c0 2024-06-11T14:02:05.531 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:05.532+0000 7f4053fff640 1 -- 172.21.15.57:0/2923687971 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f4060056bd0 con 0x7f406c10a6c0 2024-06-11T14:02:05.720 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:05.721+0000 7f4071713640 1 -- 172.21.15.57:0/2923687971 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "100"} v 0) -- 0x7f4058005370 con 0x7f406c10a6c0 2024-06-11T14:02:06.023 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:06.023+0000 7f4053fff640 1 -- 172.21.15.57:0/2923687971 <== mon.0 v1:172.21.15.57:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool 5bd44950-56a4-47a9-a376-eee5f24dbed6 v783) ==== 225+0+0 (unknown 564780287 0 0) 0x7f406005a880 con 0x7f406c10a6c0 2024-06-11T14:02:06.024 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:06.024+0000 7f4053fff640 1 -- 172.21.15.57:0/2923687971 <== mon.0 v1:172.21.15.57:6789/0 11 ==== osd_map(783..783 src has 265..783) ==== 628+0+0 (unknown 2540305028 0 0) 0x7f4060004b70 con 0x7f406c10a6c0 2024-06-11T14:02:06.024 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:06.025+0000 7f4053fff640 1 -- 172.21.15.57:0/2923687971 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=784}) -- 0x7f406c1ab080 con 0x7f406c10a6c0 2024-06-11T14:02:06.138 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:06.138+0000 7f4071713640 1 -- 172.21.15.57:0/2923687971 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "100"} v 0) -- 0x7f4058001090 con 0x7f406c10a6c0 2024-06-11T14:02:06.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:06 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2923687971' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "100"}]: dispatch 2024-06-11T14:02:06.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:06 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2923687971' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "100"}]: dispatch 2024-06-11T14:02:06.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:06 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2923687971' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "100"}]: dispatch 2024-06-11T14:02:07.034 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.034+0000 7f4053fff640 1 -- 172.21.15.57:0/2923687971 <== mon.0 v1:172.21.15.57:6789/0 12 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool 5bd44950-56a4-47a9-a376-eee5f24dbed6 v784) ==== 225+0+0 (unknown 1008654020 0 0) 0x7f406005f140 con 0x7f406c10a6c0 2024-06-11T14:02:07.034 INFO:tasks.workunit.client.0.smithi057.stderr:set-quota max_objects = 100 for pool 5bd44950-56a4-47a9-a376-eee5f24dbed6 2024-06-11T14:02:07.037 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.038+0000 7f4071713640 1 -- 172.21.15.57:0/2923687971 >> v1:172.21.15.57:6800/2077416415 conn(0x7f405406ef70 legacy=0x7f4054071430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:02:07.037 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.038+0000 7f4071713640 1 -- 172.21.15.57:0/2923687971 >> v1:172.21.15.57:6789/0 conn(0x7f406c10a6c0 legacy=0x7f406c1a3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:02:07.037 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.038+0000 7f4071713640 1 -- 172.21.15.57:0/2923687971 shutdown_connections 2024-06-11T14:02:07.037 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.038+0000 7f4071713640 1 -- 172.21.15.57:0/2923687971 >> 172.21.15.57:0/2923687971 conn(0x7f406c0fd2c0 msgr2=0x7f406c101810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T14:02:07.038 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.038+0000 7f4071713640 1 -- 172.21.15.57:0/2923687971 shutdown_connections 2024-06-11T14:02:07.038 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.038+0000 7f4071713640 1 -- 172.21.15.57:0/2923687971 wait complete. 2024-06-11T14:02:07.053 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 154462 2024-06-11T14:02:07.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:07 smithi057 ceph-mon[40588]: pgmap v1714: 176 pgs: 176 active+clean; 701 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:02:07.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:07 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2923687971' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "100"}]': finished 2024-06-11T14:02:07.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:07 smithi057 ceph-mon[40588]: osdmap e783: 8 total, 8 up, 8 in 2024-06-11T14:02:07.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:07 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2923687971' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "100"}]: dispatch 2024-06-11T14:02:07.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:07 smithi057 ceph-mon[29805]: pgmap v1714: 176 pgs: 176 active+clean; 701 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:02:07.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:07 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2923687971' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "100"}]': finished 2024-06-11T14:02:07.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:07 smithi057 ceph-mon[29805]: osdmap e783: 8 total, 8 up, 8 in 2024-06-11T14:02:07.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:07 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2923687971' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "100"}]: dispatch 2024-06-11T14:02:07.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:07 smithi107 ceph-mon[33178]: pgmap v1714: 176 pgs: 176 active+clean; 701 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:02:07.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:07 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2923687971' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "100"}]': finished 2024-06-11T14:02:07.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:07 smithi107 ceph-mon[33178]: osdmap e783: 8 total, 8 up, 8 in 2024-06-11T14:02:07.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:07 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2923687971' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "100"}]: dispatch 2024-06-11T14:02:07.611 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' 0 -ne 0 ']' 2024-06-11T14:02:07.611 INFO:tasks.workunit.client.0.smithi057.stderr:+ true 2024-06-11T14:02:07.612 INFO:tasks.workunit.client.0.smithi057.stderr:+ rados -p 5bd44950-56a4-47a9-a376-eee5f24dbed6 put twomore /etc/passwd 2024-06-11T14:02:07.650 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph osd pool set-quota 5bd44950-56a4-47a9-a376-eee5f24dbed6 max_bytes 100 2024-06-11T14:02:07.744 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.742+0000 7f0965d1e640 1 -- 172.21.15.57:0/1191378854 >> v1:172.21.15.57:6789/0 conn(0x7f09601095d0 legacy=0x7f096010b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:02:07.744 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.742+0000 7f0965d1e640 1 -- 172.21.15.57:0/1191378854 shutdown_connections 2024-06-11T14:02:07.744 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.742+0000 7f0965d1e640 1 -- 172.21.15.57:0/1191378854 >> 172.21.15.57:0/1191378854 conn(0x7f09600fc230 msgr2=0x7f09600fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T14:02:07.745 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.742+0000 7f0965d1e640 1 -- 172.21.15.57:0/1191378854 shutdown_connections 2024-06-11T14:02:07.745 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.742+0000 7f0965d1e640 1 -- 172.21.15.57:0/1191378854 wait complete. 2024-06-11T14:02:07.745 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.743+0000 7f0965d1e640 1 Processor -- start 2024-06-11T14:02:07.745 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.743+0000 7f0965d1e640 1 -- start start 2024-06-11T14:02:07.745 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.743+0000 7f0965d1e640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f096010f230 con 0x7f096010d1a0 2024-06-11T14:02:07.745 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.743+0000 7f0965d1e640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f096010f460 con 0x7f0960106730 2024-06-11T14:02:07.745 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.743+0000 7f0965d1e640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f09601ab160 con 0x7f09601095d0 2024-06-11T14:02:07.745 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.743+0000 7f0957fff640 1 --1- >> v1:172.21.15.57:6790/0 conn(0x7f09601095d0 0x7f096010de10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6790/0 says I am v1:172.21.15.57:48770/0 (socket says 172.21.15.57:48770) 2024-06-11T14:02:07.745 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.743+0000 7f0957fff640 1 -- 172.21.15.57:0/1119614035 learned_addr learned my addr 172.21.15.57:0/1119614035 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T14:02:07.745 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.743+0000 7f095ffff640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f096010d1a0 0x7f09601a7a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:57200/0 (socket says 172.21.15.57:57200) 2024-06-11T14:02:07.745 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.744+0000 7f095d7fa640 1 -- 172.21.15.57:0/1119614035 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 286325295 0 0) 0x7f096010f230 con 0x7f096010d1a0 2024-06-11T14:02:07.745 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.744+0000 7f095d7fa640 1 -- 172.21.15.57:0/1119614035 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f094c003520 con 0x7f096010d1a0 2024-06-11T14:02:07.745 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.744+0000 7f095d7fa640 1 -- 172.21.15.57:0/1119614035 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3383841271 0 0) 0x7f09601ab160 con 0x7f09601095d0 2024-06-11T14:02:07.745 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.744+0000 7f095d7fa640 1 -- 172.21.15.57:0/1119614035 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f096010f230 con 0x7f09601095d0 2024-06-11T14:02:07.746 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.744+0000 7f095d7fa640 1 -- 172.21.15.57:0/1119614035 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 737297951 0 0) 0x7f096010f460 con 0x7f0960106730 2024-06-11T14:02:07.746 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.744+0000 7f095d7fa640 1 -- 172.21.15.57:0/1119614035 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f09601ab160 con 0x7f0960106730 2024-06-11T14:02:07.746 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.744+0000 7f095d7fa640 1 -- 172.21.15.57:0/1119614035 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1140526422 0 0) 0x7f094c003520 con 0x7f096010d1a0 2024-06-11T14:02:07.746 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.744+0000 7f095d7fa640 1 -- 172.21.15.57:0/1119614035 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f096010f460 con 0x7f096010d1a0 2024-06-11T14:02:07.746 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.744+0000 7f095d7fa640 1 -- 172.21.15.57:0/1119614035 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2799284175 0 0) 0x7f096010f230 con 0x7f09601095d0 2024-06-11T14:02:07.746 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.744+0000 7f095d7fa640 1 -- 172.21.15.57:0/1119614035 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f094c003520 con 0x7f09601095d0 2024-06-11T14:02:07.746 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.744+0000 7f095d7fa640 1 -- 172.21.15.57:0/1119614035 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f0958001bb0 con 0x7f096010d1a0 2024-06-11T14:02:07.746 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.744+0000 7f095d7fa640 1 -- 172.21.15.57:0/1119614035 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1227565756 0 0) 0x7f09601ab160 con 0x7f0960106730 2024-06-11T14:02:07.746 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.744+0000 7f095d7fa640 1 -- 172.21.15.57:0/1119614035 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f096010f230 con 0x7f0960106730 2024-06-11T14:02:07.746 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.744+0000 7f095d7fa640 1 -- 172.21.15.57:0/1119614035 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f09480043a0 con 0x7f09601095d0 2024-06-11T14:02:07.746 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.744+0000 7f095d7fa640 1 -- 172.21.15.57:0/1119614035 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f0950001050 con 0x7f0960106730 2024-06-11T14:02:07.746 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.744+0000 7f095d7fa640 1 -- 172.21.15.57:0/1119614035 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2362786885 0 0) 0x7f096010f460 con 0x7f096010d1a0 2024-06-11T14:02:07.746 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.744+0000 7f095d7fa640 1 -- 172.21.15.57:0/1119614035 >> v1:172.21.15.57:6790/0 conn(0x7f09601095d0 legacy=0x7f096010de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:02:07.747 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.744+0000 7f095d7fa640 1 -- 172.21.15.57:0/1119614035 >> v1:172.21.15.107:6789/0 conn(0x7f0960106730 legacy=0x7f096010d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:02:07.747 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.744+0000 7f095d7fa640 1 -- 172.21.15.57:0/1119614035 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09601ac340 con 0x7f096010d1a0 2024-06-11T14:02:07.747 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.744+0000 7f0965d1e640 1 -- 172.21.15.57:0/1119614035 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f09601ab330 con 0x7f096010d1a0 2024-06-11T14:02:07.747 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.744+0000 7f0965d1e640 1 -- 172.21.15.57:0/1119614035 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f09601ab800 con 0x7f096010d1a0 2024-06-11T14:02:07.747 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.744+0000 7f095d7fa640 1 -- 172.21.15.57:0/1119614035 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f0958003e20 con 0x7f096010d1a0 2024-06-11T14:02:07.747 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.744+0000 7f095d7fa640 1 -- 172.21.15.57:0/1119614035 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f0958005f20 con 0x7f096010d1a0 2024-06-11T14:02:07.747 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.745+0000 7f0965d1e640 1 -- 172.21.15.57:0/1119614035 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0940005080 con 0x7f096010d1a0 2024-06-11T14:02:07.747 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.746+0000 7f095d7fa640 1 -- 172.21.15.57:0/1119614035 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 22) ==== 92138+0+0 (unknown 311317106 0 0) 0x7f0958003990 con 0x7f096010d1a0 2024-06-11T14:02:07.748 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.746+0000 7f095d7fa640 1 -- 172.21.15.57:0/1119614035 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(785..785 src has 265..785) ==== 7831+0+0 (unknown 3393865882 0 0) 0x7f0958089d10 con 0x7f096010d1a0 2024-06-11T14:02:07.750 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.751+0000 7f095d7fa640 1 -- 172.21.15.57:0/1119614035 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f0958056df0 con 0x7f096010d1a0 2024-06-11T14:02:07.935 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:07.936+0000 7f0965d1e640 1 -- 172.21.15.57:0/1119614035 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "100"} v 0) -- 0x7f0940005370 con 0x7f096010d1a0 2024-06-11T14:02:08.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:08 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2923687971' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "100"}]': finished 2024-06-11T14:02:08.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:08 smithi107 ceph-mon[33178]: osdmap e784: 8 total, 8 up, 8 in 2024-06-11T14:02:08.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:08 smithi107 ceph-mon[33178]: pool '5bd44950-56a4-47a9-a376-eee5f24dbed6' no longer out of quota; removing NO_QUOTA flag 2024-06-11T14:02:08.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:08 smithi107 ceph-mon[33178]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-06-11T14:02:08.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:08 smithi107 ceph-mon[33178]: osdmap e785: 8 total, 8 up, 8 in 2024-06-11T14:02:08.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:08 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1119614035' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "100"}]: dispatch 2024-06-11T14:02:08.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:08 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2923687971' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "100"}]': finished 2024-06-11T14:02:08.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:08 smithi057 ceph-mon[40588]: osdmap e784: 8 total, 8 up, 8 in 2024-06-11T14:02:08.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:08 smithi057 ceph-mon[40588]: pool '5bd44950-56a4-47a9-a376-eee5f24dbed6' no longer out of quota; removing NO_QUOTA flag 2024-06-11T14:02:08.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:08 smithi057 ceph-mon[40588]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-06-11T14:02:08.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:08 smithi057 ceph-mon[40588]: osdmap e785: 8 total, 8 up, 8 in 2024-06-11T14:02:08.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:08 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1119614035' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "100"}]: dispatch 2024-06-11T14:02:08.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:08 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2923687971' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "100"}]': finished 2024-06-11T14:02:08.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:08 smithi057 ceph-mon[29805]: osdmap e784: 8 total, 8 up, 8 in 2024-06-11T14:02:08.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:08 smithi057 ceph-mon[29805]: pool '5bd44950-56a4-47a9-a376-eee5f24dbed6' no longer out of quota; removing NO_QUOTA flag 2024-06-11T14:02:08.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:08 smithi057 ceph-mon[29805]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-06-11T14:02:08.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:08 smithi057 ceph-mon[29805]: osdmap e785: 8 total, 8 up, 8 in 2024-06-11T14:02:08.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:08 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1119614035' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "100"}]: dispatch 2024-06-11T14:02:08.610 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:08.611+0000 7f095d7fa640 1 -- 172.21.15.57:0/1119614035 <== mon.0 v1:172.21.15.57:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool 5bd44950-56a4-47a9-a376-eee5f24dbed6 v786) ==== 221+0+0 (unknown 2081996800 0 0) 0x7f095805aaa0 con 0x7f096010d1a0 2024-06-11T14:02:08.727 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:08.728+0000 7f0965d1e640 1 -- 172.21.15.57:0/1119614035 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "100"} v 0) -- 0x7f0940000c40 con 0x7f096010d1a0 2024-06-11T14:02:09.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:09 smithi107 ceph-mon[33178]: pgmap v1718: 176 pgs: 176 active+clean; 701 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:02:09.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:09 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T14:02:09.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:09 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:02:09.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:09 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1119614035' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "100"}]': finished 2024-06-11T14:02:09.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:09 smithi107 ceph-mon[33178]: osdmap e786: 8 total, 8 up, 8 in 2024-06-11T14:02:09.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:09 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1119614035' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "100"}]: dispatch 2024-06-11T14:02:09.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:09 smithi057 ceph-mon[29805]: pgmap v1718: 176 pgs: 176 active+clean; 701 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:02:09.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:09 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T14:02:09.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:09 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:02:09.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:09 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1119614035' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "100"}]': finished 2024-06-11T14:02:09.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:09 smithi057 ceph-mon[29805]: osdmap e786: 8 total, 8 up, 8 in 2024-06-11T14:02:09.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:09 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1119614035' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "100"}]: dispatch 2024-06-11T14:02:09.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:09 smithi057 ceph-mon[40588]: pgmap v1718: 176 pgs: 176 active+clean; 701 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:02:09.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:09 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T14:02:09.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:09 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:02:09.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:09 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1119614035' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "100"}]': finished 2024-06-11T14:02:09.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:09 smithi057 ceph-mon[40588]: osdmap e786: 8 total, 8 up, 8 in 2024-06-11T14:02:09.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:09 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1119614035' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "100"}]: dispatch 2024-06-11T14:02:09.616 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:09.617+0000 7f095d7fa640 1 -- 172.21.15.57:0/1119614035 <== mon.0 v1:172.21.15.57:6789/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool 5bd44950-56a4-47a9-a376-eee5f24dbed6 v787) ==== 221+0+0 (unknown 4232581601 0 0) 0x7f095805f360 con 0x7f096010d1a0 2024-06-11T14:02:09.617 INFO:tasks.workunit.client.0.smithi057.stderr:set-quota max_bytes = 100 for pool 5bd44950-56a4-47a9-a376-eee5f24dbed6 2024-06-11T14:02:09.619 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:09.620+0000 7f0965d1e640 1 -- 172.21.15.57:0/1119614035 >> v1:172.21.15.57:6800/2077416415 conn(0x7f094c06eeb0 legacy=0x7f094c071370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:02:09.619 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:09.620+0000 7f0965d1e640 1 -- 172.21.15.57:0/1119614035 >> v1:172.21.15.57:6789/0 conn(0x7f096010d1a0 legacy=0x7f09601a7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:02:09.619 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:09.620+0000 7f0965d1e640 1 -- 172.21.15.57:0/1119614035 shutdown_connections 2024-06-11T14:02:09.619 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:09.620+0000 7f0965d1e640 1 -- 172.21.15.57:0/1119614035 >> 172.21.15.57:0/1119614035 conn(0x7f09600fc230 msgr2=0x7f0960100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T14:02:09.620 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:09.620+0000 7f0965d1e640 1 -- 172.21.15.57:0/1119614035 shutdown_connections 2024-06-11T14:02:09.620 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:09.620+0000 7f0965d1e640 1 -- 172.21.15.57:0/1119614035 wait complete. 2024-06-11T14:02:09.634 INFO:tasks.workunit.client.0.smithi057.stderr:+ sleep 30 2024-06-11T14:02:11.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:10 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1119614035' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "100"}]': finished 2024-06-11T14:02:11.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:10 smithi057 ceph-mon[40588]: osdmap e787: 8 total, 8 up, 8 in 2024-06-11T14:02:11.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:10 smithi057 ceph-mon[40588]: pgmap v1721: 176 pgs: 176 active+clean; 701 KiB data, 900 MiB used, 714 GiB / 715 GiB avail 2024-06-11T14:02:11.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:10 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1119614035' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "100"}]': finished 2024-06-11T14:02:11.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:10 smithi057 ceph-mon[29805]: osdmap e787: 8 total, 8 up, 8 in 2024-06-11T14:02:11.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:10 smithi057 ceph-mon[29805]: pgmap v1721: 176 pgs: 176 active+clean; 701 KiB data, 900 MiB used, 714 GiB / 715 GiB avail 2024-06-11T14:02:11.050 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:10 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1119614035' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "100"}]': finished 2024-06-11T14:02:11.050 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:10 smithi107 ceph-mon[33178]: osdmap e787: 8 total, 8 up, 8 in 2024-06-11T14:02:11.050 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:10 smithi107 ceph-mon[33178]: pgmap v1721: 176 pgs: 176 active+clean; 701 KiB data, 900 MiB used, 714 GiB / 715 GiB avail 2024-06-11T14:02:13.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:12 smithi057 ceph-mon[40588]: pgmap v1722: 176 pgs: 176 active+clean; 724 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 4.9 KiB/s wr, 1 op/s 2024-06-11T14:02:13.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:12 smithi057 ceph-mon[40588]: pool '5bd44950-56a4-47a9-a376-eee5f24dbed6' is full (reached quota's max_bytes: 100 B) 2024-06-11T14:02:13.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:12 smithi057 ceph-mon[40588]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-06-11T14:02:13.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:12 smithi057 ceph-mon[40588]: osdmap e788: 8 total, 8 up, 8 in 2024-06-11T14:02:13.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:12 smithi057 ceph-mon[29805]: pgmap v1722: 176 pgs: 176 active+clean; 724 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 4.9 KiB/s wr, 1 op/s 2024-06-11T14:02:13.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:12 smithi057 ceph-mon[29805]: pool '5bd44950-56a4-47a9-a376-eee5f24dbed6' is full (reached quota's max_bytes: 100 B) 2024-06-11T14:02:13.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:12 smithi057 ceph-mon[29805]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-06-11T14:02:13.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:12 smithi057 ceph-mon[29805]: osdmap e788: 8 total, 8 up, 8 in 2024-06-11T14:02:13.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:12 smithi107 ceph-mon[33178]: pgmap v1722: 176 pgs: 176 active+clean; 724 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 4.9 KiB/s wr, 1 op/s 2024-06-11T14:02:13.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:12 smithi107 ceph-mon[33178]: pool '5bd44950-56a4-47a9-a376-eee5f24dbed6' is full (reached quota's max_bytes: 100 B) 2024-06-11T14:02:13.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:12 smithi107 ceph-mon[33178]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-06-11T14:02:13.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:12 smithi107 ceph-mon[33178]: osdmap e788: 8 total, 8 up, 8 in 2024-06-11T14:02:13.300 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 14:02:12 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T14:02:13.985 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 14:02:13 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:14:02:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T14:02:14.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:13 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:02:14.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:13 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:02:14.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:13 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:02:15.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:14 smithi057 ceph-mon[29805]: pgmap v1724: 176 pgs: 176 active+clean; 724 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-06-11T14:02:15.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:14 smithi057 ceph-mon[40588]: pgmap v1724: 176 pgs: 176 active+clean; 724 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-06-11T14:02:15.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:14 smithi107 ceph-mon[33178]: pgmap v1724: 176 pgs: 176 active+clean; 724 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-06-11T14:02:17.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:16 smithi057 ceph-mon[29805]: pgmap v1725: 176 pgs: 176 active+clean; 724 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 701 B/s rd, 3.3 KiB/s wr, 0 op/s 2024-06-11T14:02:17.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:17 smithi057 ceph-mon[40588]: pgmap v1725: 176 pgs: 176 active+clean; 724 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 701 B/s rd, 3.3 KiB/s wr, 0 op/s 2024-06-11T14:02:17.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:16 smithi107 ceph-mon[33178]: pgmap v1725: 176 pgs: 176 active+clean; 724 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 701 B/s rd, 3.3 KiB/s wr, 0 op/s 2024-06-11T14:02:19.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:19 smithi057 ceph-mon[29805]: pgmap v1726: 176 pgs: 176 active+clean; 724 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.9 KiB/s wr, 1 op/s 2024-06-11T14:02:19.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:19 smithi057 ceph-mon[40588]: pgmap v1726: 176 pgs: 176 active+clean; 724 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.9 KiB/s wr, 1 op/s 2024-06-11T14:02:19.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:19 smithi107 ceph-mon[33178]: pgmap v1726: 176 pgs: 176 active+clean; 724 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.9 KiB/s wr, 1 op/s 2024-06-11T14:02:21.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:21 smithi057 ceph-mon[29805]: pgmap v1727: 176 pgs: 176 active+clean; 724 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 2.4 KiB/s wr, 1 op/s 2024-06-11T14:02:21.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:21 smithi057 ceph-mon[40588]: pgmap v1727: 176 pgs: 176 active+clean; 724 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 2.4 KiB/s wr, 1 op/s 2024-06-11T14:02:21.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:21 smithi107 ceph-mon[33178]: pgmap v1727: 176 pgs: 176 active+clean; 724 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 2.4 KiB/s wr, 1 op/s 2024-06-11T14:02:23.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:23 smithi107 ceph-mon[33178]: pgmap v1728: 176 pgs: 176 active+clean; 724 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T14:02:23.300 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 14:02:22 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T14:02:23.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:23 smithi057 ceph-mon[29805]: pgmap v1728: 176 pgs: 176 active+clean; 724 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T14:02:23.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:23 smithi057 ceph-mon[40588]: pgmap v1728: 176 pgs: 176 active+clean; 724 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T14:02:24.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 14:02:23 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:14:02:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T14:02:24.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:24 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:02:24.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:24 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T14:02:24.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:24 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:02:24.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:24 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:02:24.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:24 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T14:02:24.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:24 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:02:24.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:24 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:02:24.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:24 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T14:02:24.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:24 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:02:25.438 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:25 smithi057 ceph-mon[29805]: pgmap v1729: 176 pgs: 176 active+clean; 724 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 904 B/s rd, 0 op/s 2024-06-11T14:02:25.438 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:25 smithi057 ceph-mon[40588]: pgmap v1729: 176 pgs: 176 active+clean; 724 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 904 B/s rd, 0 op/s 2024-06-11T14:02:25.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:25 smithi107 ceph-mon[33178]: pgmap v1729: 176 pgs: 176 active+clean; 724 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 904 B/s rd, 0 op/s 2024-06-11T14:02:27.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:27 smithi057 ceph-mon[29805]: pgmap v1730: 176 pgs: 176 active+clean; 724 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:02:27.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:27 smithi057 ceph-mon[40588]: pgmap v1730: 176 pgs: 176 active+clean; 724 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:02:27.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:27 smithi107 ceph-mon[33178]: pgmap v1730: 176 pgs: 176 active+clean; 724 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:02:29.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:29 smithi057 ceph-mon[29805]: pgmap v1731: 176 pgs: 176 active+clean; 724 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:02:29.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:29 smithi057 ceph-mon[40588]: pgmap v1731: 176 pgs: 176 active+clean; 724 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:02:29.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:29 smithi107 ceph-mon[33178]: pgmap v1731: 176 pgs: 176 active+clean; 724 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:02:31.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:31 smithi057 ceph-mon[29805]: pgmap v1732: 176 pgs: 176 active+clean; 724 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:02:31.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:31 smithi057 ceph-mon[40588]: pgmap v1732: 176 pgs: 176 active+clean; 724 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:02:31.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:31 smithi107 ceph-mon[33178]: pgmap v1732: 176 pgs: 176 active+clean; 724 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:02:32.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:32 smithi057 ceph-mon[29805]: pgmap v1733: 176 pgs: 176 active+clean; 724 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:02:32.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:32 smithi057 ceph-mon[40588]: pgmap v1733: 176 pgs: 176 active+clean; 724 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:02:32.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:32 smithi107 ceph-mon[33178]: pgmap v1733: 176 pgs: 176 active+clean; 724 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:02:33.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:33 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:02:33.300 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 14:02:32 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T14:02:33.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:33 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:02:33.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:33 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:02:34.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 14:02:33 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:14:02:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T14:02:34.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:34 smithi057 ceph-mon[29805]: pgmap v1734: 176 pgs: 176 active+clean; 724 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:02:34.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:34 smithi057 ceph-mon[40588]: pgmap v1734: 176 pgs: 176 active+clean; 724 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:02:34.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:34 smithi107 ceph-mon[33178]: pgmap v1734: 176 pgs: 176 active+clean; 724 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:02:37.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:36 smithi057 ceph-mon[29805]: pgmap v1735: 176 pgs: 176 active+clean; 724 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:02:37.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:36 smithi057 ceph-mon[40588]: pgmap v1735: 176 pgs: 176 active+clean; 724 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:02:37.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:36 smithi107 ceph-mon[33178]: pgmap v1735: 176 pgs: 176 active+clean; 724 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:02:39.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:38 smithi057 ceph-mon[29805]: pgmap v1736: 176 pgs: 176 active+clean; 724 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:02:39.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:38 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:02:39.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:38 smithi057 ceph-mon[40588]: pgmap v1736: 176 pgs: 176 active+clean; 724 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:02:39.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:38 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:02:39.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:38 smithi107 ceph-mon[33178]: pgmap v1736: 176 pgs: 176 active+clean; 724 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:02:39.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:38 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:02:39.636 INFO:tasks.workunit.client.0.smithi057.stderr:+ pid=154565 2024-06-11T14:02:39.636 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph osd pool set-quota 5bd44950-56a4-47a9-a376-eee5f24dbed6 max_bytes 0 2024-06-11T14:02:39.636 INFO:tasks.workunit.client.0.smithi057.stderr:+ rados -p 5bd44950-56a4-47a9-a376-eee5f24dbed6 put two /etc/passwd 2024-06-11T14:02:39.736 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:39.736+0000 7f6a5120e640 1 -- 172.21.15.57:0/3082672114 >> v1:172.21.15.57:6790/0 conn(0x7f6a4c106720 legacy=0x7f6a4c106b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:02:39.736 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:39.736+0000 7f6a5120e640 1 -- 172.21.15.57:0/3082672114 shutdown_connections 2024-06-11T14:02:39.736 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:39.736+0000 7f6a5120e640 1 -- 172.21.15.57:0/3082672114 >> 172.21.15.57:0/3082672114 conn(0x7f6a4c077400 msgr2=0x7f6a4c0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T14:02:39.736 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:39.736+0000 7f6a5120e640 1 -- 172.21.15.57:0/3082672114 shutdown_connections 2024-06-11T14:02:39.736 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:39.736+0000 7f6a5120e640 1 -- 172.21.15.57:0/3082672114 wait complete. 2024-06-11T14:02:39.739 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:39.737+0000 7f6a5120e640 1 Processor -- start 2024-06-11T14:02:39.739 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:39.737+0000 7f6a5120e640 1 -- start start 2024-06-11T14:02:39.739 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:39.738+0000 7f6a5120e640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f6a4c1a68a0 con 0x7f6a4c106720 2024-06-11T14:02:39.739 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:39.738+0000 7f6a5120e640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f6a4c1a7ae0 con 0x7f6a4c1095c0 2024-06-11T14:02:39.739 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:39.738+0000 7f6a5120e640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f6a4c1a8d20 con 0x7f6a4c10d190 2024-06-11T14:02:39.739 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:39.738+0000 7f6a4ad9d640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f6a4c106720 0x7f6a4c10ca30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:52076/0 (socket says 172.21.15.57:52076) 2024-06-11T14:02:39.740 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:39.738+0000 7f6a4ad9d640 1 -- 172.21.15.57:0/1212405401 learned_addr learned my addr 172.21.15.57:0/1212405401 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T14:02:39.740 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:39.738+0000 7f6a43fff640 1 -- 172.21.15.57:0/1212405401 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2293200257 0 0) 0x7f6a4c1a8d20 con 0x7f6a4c10d190 2024-06-11T14:02:39.740 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:39.738+0000 7f6a43fff640 1 -- 172.21.15.57:0/1212405401 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a38003160 con 0x7f6a4c10d190 2024-06-11T14:02:39.740 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:39.738+0000 7f6a43fff640 1 -- 172.21.15.57:0/1212405401 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 44448792 0 0) 0x7f6a4c1a68a0 con 0x7f6a4c106720 2024-06-11T14:02:39.740 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:39.738+0000 7f6a43fff640 1 -- 172.21.15.57:0/1212405401 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a4c1a8d20 con 0x7f6a4c106720 2024-06-11T14:02:39.740 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:39.738+0000 7f6a43fff640 1 -- 172.21.15.57:0/1212405401 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3012567001 0 0) 0x7f6a4c1a7ae0 con 0x7f6a4c1095c0 2024-06-11T14:02:39.740 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:39.738+0000 7f6a43fff640 1 -- 172.21.15.57:0/1212405401 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a4c1a68a0 con 0x7f6a4c1095c0 2024-06-11T14:02:39.740 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:39.738+0000 7f6a43fff640 1 -- 172.21.15.57:0/1212405401 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1502941691 0 0) 0x7f6a4c1a8d20 con 0x7f6a4c106720 2024-06-11T14:02:39.740 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:39.738+0000 7f6a43fff640 1 -- 172.21.15.57:0/1212405401 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6a4c1a7ae0 con 0x7f6a4c106720 2024-06-11T14:02:39.740 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:39.738+0000 7f6a43fff640 1 -- 172.21.15.57:0/1212405401 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f6a3c0046a0 con 0x7f6a4c106720 2024-06-11T14:02:39.740 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:39.738+0000 7f6a43fff640 1 -- 172.21.15.57:0/1212405401 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2210244558 0 0) 0x7f6a38003160 con 0x7f6a4c10d190 2024-06-11T14:02:39.740 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:39.738+0000 7f6a43fff640 1 -- 172.21.15.57:0/1212405401 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6a4c1a8d20 con 0x7f6a4c10d190 2024-06-11T14:02:39.740 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:39.738+0000 7f6a43fff640 1 -- 172.21.15.57:0/1212405401 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f6a44001830 con 0x7f6a4c10d190 2024-06-11T14:02:39.741 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:39.738+0000 7f6a43fff640 1 -- 172.21.15.57:0/1212405401 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2524614146 0 0) 0x7f6a4c1a68a0 con 0x7f6a4c1095c0 2024-06-11T14:02:39.741 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:39.739+0000 7f6a43fff640 1 -- 172.21.15.57:0/1212405401 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6a38003160 con 0x7f6a4c1095c0 2024-06-11T14:02:39.741 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:39.739+0000 7f6a43fff640 1 -- 172.21.15.57:0/1212405401 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f6a34001960 con 0x7f6a4c1095c0 2024-06-11T14:02:39.741 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:39.739+0000 7f6a43fff640 1 -- 172.21.15.57:0/1212405401 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 168131000 0 0) 0x7f6a4c1a7ae0 con 0x7f6a4c106720 2024-06-11T14:02:39.741 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:39.739+0000 7f6a43fff640 1 -- 172.21.15.57:0/1212405401 >> v1:172.21.15.57:6790/0 conn(0x7f6a4c10d190 legacy=0x7f6a4c1a4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:02:39.741 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:39.739+0000 7f6a43fff640 1 -- 172.21.15.57:0/1212405401 >> v1:172.21.15.107:6789/0 conn(0x7f6a4c1095c0 legacy=0x7f6a4c1a15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:02:39.741 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:39.739+0000 7f6a43fff640 1 -- 172.21.15.57:0/1212405401 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a4c1a9f60 con 0x7f6a4c106720 2024-06-11T14:02:39.741 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:39.739+0000 7f6a5120e640 1 -- 172.21.15.57:0/1212405401 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f6a4c1a6ad0 con 0x7f6a4c106720 2024-06-11T14:02:39.741 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:39.739+0000 7f6a5120e640 1 -- 172.21.15.57:0/1212405401 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f6a4c1a6fa0 con 0x7f6a4c106720 2024-06-11T14:02:39.741 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:39.739+0000 7f6a43fff640 1 -- 172.21.15.57:0/1212405401 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f6a3c002070 con 0x7f6a4c106720 2024-06-11T14:02:39.741 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:39.739+0000 7f6a43fff640 1 -- 172.21.15.57:0/1212405401 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f6a3c004fd0 con 0x7f6a4c106720 2024-06-11T14:02:39.741 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:39.740+0000 7f6a5120e640 1 -- 172.21.15.57:0/1212405401 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a2c005080 con 0x7f6a4c106720 2024-06-11T14:02:39.742 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:39.740+0000 7f6a43fff640 1 -- 172.21.15.57:0/1212405401 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 22) ==== 92138+0+0 (unknown 311317106 0 0) 0x7f6a3c003a70 con 0x7f6a4c106720 2024-06-11T14:02:39.742 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:39.741+0000 7f6a43fff640 1 -- 172.21.15.57:0/1212405401 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(788..788 src has 265..788) ==== 7831+0+0 (unknown 917683635 0 0) 0x7f6a3c089e20 con 0x7f6a4c106720 2024-06-11T14:02:39.742 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:39.741+0000 7f6a43fff640 1 -- 172.21.15.57:0/1212405401 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=789}) -- 0x7f6a4c1a7ae0 con 0x7f6a4c106720 2024-06-11T14:02:39.744 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:39.745+0000 7f6a43fff640 1 -- 172.21.15.57:0/1212405401 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f6a3c056f00 con 0x7f6a4c106720 2024-06-11T14:02:39.930 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:39.931+0000 7f6a5120e640 1 -- 172.21.15.57:0/1212405401 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "0"} v 0) -- 0x7f6a2c005370 con 0x7f6a4c106720 2024-06-11T14:02:39.998 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:39.999+0000 7f6a43fff640 1 -- 172.21.15.57:0/1212405401 <== mon.0 v1:172.21.15.57:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 5bd44950-56a4-47a9-a376-eee5f24dbed6 v789) ==== 217+0+0 (unknown 4031305428 0 0) 0x7f6a3c05abb0 con 0x7f6a4c106720 2024-06-11T14:02:40.001 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:40.001+0000 7f6a43fff640 1 -- 172.21.15.57:0/1212405401 <== mon.0 v1:172.21.15.57:6789/0 11 ==== osd_map(789..789 src has 265..789) ==== 628+0+0 (unknown 61837526 0 0) 0x7f6a3c001980 con 0x7f6a4c106720 2024-06-11T14:02:40.001 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:40.001+0000 7f6a43fff640 1 -- 172.21.15.57:0/1212405401 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=790}) -- 0x7f6a4c1a68a0 con 0x7f6a4c106720 2024-06-11T14:02:40.111 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:40.112+0000 7f6a5120e640 1 -- 172.21.15.57:0/1212405401 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "0"} v 0) -- 0x7f6a2c000c40 con 0x7f6a4c106720 2024-06-11T14:02:40.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:39 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1212405401' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "0"}]: dispatch 2024-06-11T14:02:40.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:39 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1212405401' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "0"}]: dispatch 2024-06-11T14:02:40.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:39 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1212405401' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "0"}]: dispatch 2024-06-11T14:02:41.012 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.012+0000 7f6a43fff640 1 -- 172.21.15.57:0/1212405401 <== mon.0 v1:172.21.15.57:6789/0 12 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 5bd44950-56a4-47a9-a376-eee5f24dbed6 v790) ==== 217+0+0 (unknown 2252977783 0 0) 0x7f6a3c05f470 con 0x7f6a4c106720 2024-06-11T14:02:41.012 INFO:tasks.workunit.client.0.smithi057.stderr:set-quota max_bytes = 0 for pool 5bd44950-56a4-47a9-a376-eee5f24dbed6 2024-06-11T14:02:41.012 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.013+0000 7f6a43fff640 1 -- 172.21.15.57:0/1212405401 <== mon.0 v1:172.21.15.57:6789/0 13 ==== osd_map(790..790 src has 265..790) ==== 628+0+0 (unknown 2471615619 0 0) 0x7f6a3c087bb0 con 0x7f6a4c106720 2024-06-11T14:02:41.012 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.013+0000 7f6a43fff640 1 -- 172.21.15.57:0/1212405401 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=791}) -- 0x7f6a3807c1a0 con 0x7f6a4c106720 2024-06-11T14:02:41.016 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.017+0000 7f6a5120e640 1 -- 172.21.15.57:0/1212405401 >> v1:172.21.15.57:6800/2077416415 conn(0x7f6a3806ee90 legacy=0x7f6a38071350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:02:41.016 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.017+0000 7f6a5120e640 1 -- 172.21.15.57:0/1212405401 >> v1:172.21.15.57:6789/0 conn(0x7f6a4c106720 legacy=0x7f6a4c10ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:02:41.016 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.017+0000 7f6a5120e640 1 -- 172.21.15.57:0/1212405401 shutdown_connections 2024-06-11T14:02:41.017 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.017+0000 7f6a5120e640 1 -- 172.21.15.57:0/1212405401 >> 172.21.15.57:0/1212405401 conn(0x7f6a4c077400 msgr2=0x7f6a4c110670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T14:02:41.017 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.017+0000 7f6a5120e640 1 -- 172.21.15.57:0/1212405401 shutdown_connections 2024-06-11T14:02:41.017 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.017+0000 7f6a5120e640 1 -- 172.21.15.57:0/1212405401 wait complete. 2024-06-11T14:02:41.030 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph osd pool set-quota 5bd44950-56a4-47a9-a376-eee5f24dbed6 max_objects 0 2024-06-11T14:02:41.121 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.122+0000 7fb108e4e640 1 -- 172.21.15.57:0/4002418516 >> v1:172.21.15.57:6789/0 conn(0x7fb1041095d0 legacy=0x7fb10410b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:02:41.122 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.122+0000 7fb108e4e640 1 -- 172.21.15.57:0/4002418516 shutdown_connections 2024-06-11T14:02:41.122 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.122+0000 7fb108e4e640 1 -- 172.21.15.57:0/4002418516 >> 172.21.15.57:0/4002418516 conn(0x7fb1040fc230 msgr2=0x7fb1040fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T14:02:41.122 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.122+0000 7fb108e4e640 1 -- 172.21.15.57:0/4002418516 shutdown_connections 2024-06-11T14:02:41.122 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.122+0000 7fb108e4e640 1 -- 172.21.15.57:0/4002418516 wait complete. 2024-06-11T14:02:41.125 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.122+0000 7fb108e4e640 1 Processor -- start 2024-06-11T14:02:41.125 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.123+0000 7fb108e4e640 1 -- start start 2024-06-11T14:02:41.125 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.123+0000 7fb108e4e640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fb1041a6890 con 0x7fb104106730 2024-06-11T14:02:41.125 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.123+0000 7fb108e4e640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fb1041a7ad0 con 0x7fb10410d1a0 2024-06-11T14:02:41.125 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.123+0000 7fb10259c640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fb104106730 0x7fb10410ca10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:52082/0 (socket says 172.21.15.57:52082) 2024-06-11T14:02:41.125 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.123+0000 7fb10259c640 1 -- 172.21.15.57:0/2211668086 learned_addr learned my addr 172.21.15.57:0/2211668086 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T14:02:41.125 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.123+0000 7fb101d9b640 1 --1- 172.21.15.57:0/2211668086 >> v1:172.21.15.57:6790/0 conn(0x7fb1041095d0 0x7fb1041a1600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6790/0 says I am v1:172.21.15.57:42490/0 (socket says 172.21.15.57:42490) 2024-06-11T14:02:41.125 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.123+0000 7fb108e4e640 1 -- 172.21.15.57:0/2211668086 --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fb1041a8d10 con 0x7fb1041095d0 2024-06-11T14:02:41.125 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.123+0000 7fb0f37fe640 1 -- 172.21.15.57:0/2211668086 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 902718890 0 0) 0x7fb1041a6890 con 0x7fb104106730 2024-06-11T14:02:41.125 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.123+0000 7fb0f37fe640 1 -- 172.21.15.57:0/2211668086 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb0ec001fe0 con 0x7fb104106730 2024-06-11T14:02:41.125 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.123+0000 7fb0f37fe640 1 -- 172.21.15.57:0/2211668086 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1409493556 0 0) 0x7fb1041a8d10 con 0x7fb1041095d0 2024-06-11T14:02:41.126 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.123+0000 7fb0f37fe640 1 -- 172.21.15.57:0/2211668086 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb1041a6890 con 0x7fb1041095d0 2024-06-11T14:02:41.126 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.123+0000 7fb0f37fe640 1 -- 172.21.15.57:0/2211668086 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2844580575 0 0) 0x7fb1041a7ad0 con 0x7fb10410d1a0 2024-06-11T14:02:41.126 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.123+0000 7fb0f37fe640 1 -- 172.21.15.57:0/2211668086 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb1041a8d10 con 0x7fb10410d1a0 2024-06-11T14:02:41.126 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.123+0000 7fb0f37fe640 1 -- 172.21.15.57:0/2211668086 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1849916666 0 0) 0x7fb0ec001fe0 con 0x7fb104106730 2024-06-11T14:02:41.126 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.123+0000 7fb0f37fe640 1 -- 172.21.15.57:0/2211668086 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb1041a7ad0 con 0x7fb104106730 2024-06-11T14:02:41.126 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.123+0000 7fb0f37fe640 1 -- 172.21.15.57:0/2211668086 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fb0f4001a30 con 0x7fb104106730 2024-06-11T14:02:41.126 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.123+0000 7fb0f37fe640 1 -- 172.21.15.57:0/2211668086 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1955402723 0 0) 0x7fb1041a6890 con 0x7fb1041095d0 2024-06-11T14:02:41.126 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.124+0000 7fb0f37fe640 1 -- 172.21.15.57:0/2211668086 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb0ec001fe0 con 0x7fb1041095d0 2024-06-11T14:02:41.126 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.124+0000 7fb0f37fe640 1 -- 172.21.15.57:0/2211668086 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fb0f80044f0 con 0x7fb1041095d0 2024-06-11T14:02:41.126 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.124+0000 7fb0f37fe640 1 -- 172.21.15.57:0/2211668086 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 272671274 0 0) 0x7fb1041a8d10 con 0x7fb10410d1a0 2024-06-11T14:02:41.126 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.124+0000 7fb0f37fe640 1 -- 172.21.15.57:0/2211668086 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb1041a6890 con 0x7fb10410d1a0 2024-06-11T14:02:41.126 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.124+0000 7fb0f37fe640 1 -- 172.21.15.57:0/2211668086 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fb0fc001af0 con 0x7fb10410d1a0 2024-06-11T14:02:41.127 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.124+0000 7fb0f37fe640 1 -- 172.21.15.57:0/2211668086 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 565909714 0 0) 0x7fb1041a7ad0 con 0x7fb104106730 2024-06-11T14:02:41.127 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.124+0000 7fb0f37fe640 1 -- 172.21.15.57:0/2211668086 >> v1:172.21.15.57:6790/0 conn(0x7fb1041095d0 legacy=0x7fb1041a1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:02:41.127 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.124+0000 7fb0f37fe640 1 -- 172.21.15.57:0/2211668086 >> v1:172.21.15.107:6789/0 conn(0x7fb10410d1a0 legacy=0x7fb1041a4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:02:41.127 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.124+0000 7fb0f37fe640 1 -- 172.21.15.57:0/2211668086 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1041a9f50 con 0x7fb104106730 2024-06-11T14:02:41.127 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.124+0000 7fb108e4e640 1 -- 172.21.15.57:0/2211668086 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fb1041a8f40 con 0x7fb104106730 2024-06-11T14:02:41.127 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.124+0000 7fb108e4e640 1 -- 172.21.15.57:0/2211668086 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fb1041a9410 con 0x7fb104106730 2024-06-11T14:02:41.127 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.124+0000 7fb0f37fe640 1 -- 172.21.15.57:0/2211668086 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7fb0f4001c90 con 0x7fb104106730 2024-06-11T14:02:41.127 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.124+0000 7fb0f37fe640 1 -- 172.21.15.57:0/2211668086 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fb0f4005ba0 con 0x7fb104106730 2024-06-11T14:02:41.127 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.125+0000 7fb108e4e640 1 -- 172.21.15.57:0/2211668086 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0e40050b0 con 0x7fb104106730 2024-06-11T14:02:41.127 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.125+0000 7fb0f37fe640 1 -- 172.21.15.57:0/2211668086 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 22) ==== 92138+0+0 (unknown 311317106 0 0) 0x7fb0f401c3a0 con 0x7fb104106730 2024-06-11T14:02:41.128 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.126+0000 7fb0f37fe640 1 -- 172.21.15.57:0/2211668086 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(790..790 src has 265..790) ==== 7831+0+0 (unknown 2911656102 0 0) 0x7fb0f4089ad0 con 0x7fb104106730 2024-06-11T14:02:41.128 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.126+0000 7fb0f37fe640 1 -- 172.21.15.57:0/2211668086 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=791}) -- 0x7fb1041a7ad0 con 0x7fb104106730 2024-06-11T14:02:41.129 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.130+0000 7fb0f37fe640 1 -- 172.21.15.57:0/2211668086 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7fb0f4056bb0 con 0x7fb104106730 2024-06-11T14:02:41.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:41 smithi057 ceph-mon[29805]: pgmap v1737: 176 pgs: 176 active+clean; 724 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:02:41.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:41 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1212405401' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "0"}]': finished 2024-06-11T14:02:41.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:41 smithi057 ceph-mon[29805]: osdmap e789: 8 total, 8 up, 8 in 2024-06-11T14:02:41.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:41 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1212405401' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "0"}]: dispatch 2024-06-11T14:02:41.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:41 smithi057 ceph-mon[40588]: pgmap v1737: 176 pgs: 176 active+clean; 724 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:02:41.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:41 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1212405401' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "0"}]': finished 2024-06-11T14:02:41.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:41 smithi057 ceph-mon[40588]: osdmap e789: 8 total, 8 up, 8 in 2024-06-11T14:02:41.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:41 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1212405401' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "0"}]: dispatch 2024-06-11T14:02:41.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:41 smithi107 ceph-mon[33178]: pgmap v1737: 176 pgs: 176 active+clean; 724 KiB data, 896 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:02:41.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:41 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1212405401' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "0"}]': finished 2024-06-11T14:02:41.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:41 smithi107 ceph-mon[33178]: osdmap e789: 8 total, 8 up, 8 in 2024-06-11T14:02:41.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:41 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1212405401' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "0"}]: dispatch 2024-06-11T14:02:41.314 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:41.315+0000 7fb108e4e640 1 -- 172.21.15.57:0/2211668086 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "0"} v 0) -- 0x7fb0e40053a0 con 0x7fb104106730 2024-06-11T14:02:42.033 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.033+0000 7fb0f37fe640 1 -- 172.21.15.57:0/2211668086 <== mon.0 v1:172.21.15.57:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 5bd44950-56a4-47a9-a376-eee5f24dbed6 v791) ==== 221+0+0 (unknown 2447981417 0 0) 0x7fb0f405a860 con 0x7fb104106730 2024-06-11T14:02:42.033 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.034+0000 7fb0f37fe640 1 -- 172.21.15.57:0/2211668086 <== mon.0 v1:172.21.15.57:6789/0 11 ==== osd_map(791..791 src has 265..791) ==== 628+0+0 (unknown 2025525059 0 0) 0x7fb0f4087860 con 0x7fb104106730 2024-06-11T14:02:42.033 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.034+0000 7fb0f37fe640 1 -- 172.21.15.57:0/2211668086 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=792}) -- 0x7fb1041a8d10 con 0x7fb104106730 2024-06-11T14:02:42.150 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.151+0000 7fb108e4e640 1 -- 172.21.15.57:0/2211668086 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "0"} v 0) -- 0x7fb0e4005ae0 con 0x7fb104106730 2024-06-11T14:02:42.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:42 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1212405401' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "0"}]': finished 2024-06-11T14:02:42.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:42 smithi057 ceph-mon[29805]: osdmap e790: 8 total, 8 up, 8 in 2024-06-11T14:02:42.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:42 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2211668086' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "0"}]: dispatch 2024-06-11T14:02:42.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:42 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1212405401' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "0"}]': finished 2024-06-11T14:02:42.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:42 smithi057 ceph-mon[40588]: osdmap e790: 8 total, 8 up, 8 in 2024-06-11T14:02:42.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:42 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2211668086' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "0"}]: dispatch 2024-06-11T14:02:42.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:42 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1212405401' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "0"}]': finished 2024-06-11T14:02:42.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:42 smithi107 ceph-mon[33178]: osdmap e790: 8 total, 8 up, 8 in 2024-06-11T14:02:42.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:42 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2211668086' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "0"}]: dispatch 2024-06-11T14:02:42.602 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.603+0000 7fb0f37fe640 1 -- 172.21.15.57:0/2211668086 <== mon.0 v1:172.21.15.57:6789/0 12 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 5bd44950-56a4-47a9-a376-eee5f24dbed6 v792) ==== 221+0+0 (unknown 190365969 0 0) 0x7fb0f405f120 con 0x7fb104106730 2024-06-11T14:02:42.603 INFO:tasks.workunit.client.0.smithi057.stderr:set-quota max_objects = 0 for pool 5bd44950-56a4-47a9-a376-eee5f24dbed6 2024-06-11T14:02:42.606 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.606+0000 7fb108e4e640 1 -- 172.21.15.57:0/2211668086 >> v1:172.21.15.57:6800/2077416415 conn(0x7fb0ec06f000 legacy=0x7fb0ec0714c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:02:42.606 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.606+0000 7fb108e4e640 1 -- 172.21.15.57:0/2211668086 >> v1:172.21.15.57:6789/0 conn(0x7fb104106730 legacy=0x7fb10410ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:02:42.606 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.606+0000 7fb108e4e640 1 -- 172.21.15.57:0/2211668086 shutdown_connections 2024-06-11T14:02:42.606 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.606+0000 7fb108e4e640 1 -- 172.21.15.57:0/2211668086 >> 172.21.15.57:0/2211668086 conn(0x7fb1040fc230 msgr2=0x7fb1040fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T14:02:42.606 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.606+0000 7fb108e4e640 1 -- 172.21.15.57:0/2211668086 shutdown_connections 2024-06-11T14:02:42.606 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.606+0000 7fb108e4e640 1 -- 172.21.15.57:0/2211668086 wait complete. 2024-06-11T14:02:42.622 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 154565 2024-06-11T14:02:42.622 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' 0 -ne 0 ']' 2024-06-11T14:02:42.622 INFO:tasks.workunit.client.0.smithi057.stderr:+ true 2024-06-11T14:02:42.623 INFO:tasks.workunit.client.0.smithi057.stderr:+ rados -p 5bd44950-56a4-47a9-a376-eee5f24dbed6 put three /etc/passwd 2024-06-11T14:02:42.660 INFO:tasks.workunit.client.0.smithi057.stderr:++ uuidgen 2024-06-11T14:02:42.661 INFO:tasks.workunit.client.0.smithi057.stderr:+ pp=3308aae0-a66b-4c2f-a482-8b20f197ecc4 2024-06-11T14:02:42.661 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph osd pool create 3308aae0-a66b-4c2f-a482-8b20f197ecc4 12 2024-06-11T14:02:42.751 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.749+0000 7ffb0f15c640 1 -- 172.21.15.57:0/3110502919 >> v1:172.21.15.57:6790/0 conn(0x7ffb0810e350 legacy=0x7ffb08110780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:02:42.751 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.749+0000 7ffb0f15c640 1 -- 172.21.15.57:0/3110502919 shutdown_connections 2024-06-11T14:02:42.751 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.749+0000 7ffb0f15c640 1 -- 172.21.15.57:0/3110502919 >> 172.21.15.57:0/3110502919 conn(0x7ffb08077400 msgr2=0x7ffb080fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T14:02:42.751 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.749+0000 7ffb0f15c640 1 -- 172.21.15.57:0/3110502919 shutdown_connections 2024-06-11T14:02:42.752 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.749+0000 7ffb0f15c640 1 -- 172.21.15.57:0/3110502919 wait complete. 2024-06-11T14:02:42.752 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.750+0000 7ffb0f15c640 1 Processor -- start 2024-06-11T14:02:42.752 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.750+0000 7ffb0f15c640 1 -- start start 2024-06-11T14:02:42.752 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.750+0000 7ffb0f15c640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ffb081a6820 con 0x7ffb08102a40 2024-06-11T14:02:42.752 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.750+0000 7ffb0f15c640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ffb081a7a20 con 0x7ffb0810e350 2024-06-11T14:02:42.752 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.750+0000 7ffb0f15c640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ffb081a8c60 con 0x7ffb0810a840 2024-06-11T14:02:42.752 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.750+0000 7ffb0cef8640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7ffb08102a40 0x7ffb08109fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:52116/0 (socket says 172.21.15.57:52116) 2024-06-11T14:02:42.752 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.750+0000 7ffb07fff640 1 --1- >> v1:172.21.15.57:6790/0 conn(0x7ffb0810a840 0x7ffb081a16f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6790/0 says I am v1:172.21.15.57:42546/0 (socket says 172.21.15.57:42546) 2024-06-11T14:02:42.752 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.750+0000 7ffb0cef8640 1 -- 172.21.15.57:0/2073409438 learned_addr learned my addr 172.21.15.57:0/2073409438 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T14:02:42.752 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.750+0000 7ffb05ffb640 1 -- 172.21.15.57:0/2073409438 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2070411130 0 0) 0x7ffb081a6820 con 0x7ffb08102a40 2024-06-11T14:02:42.752 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.750+0000 7ffb05ffb640 1 -- 172.21.15.57:0/2073409438 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffaf0003160 con 0x7ffb08102a40 2024-06-11T14:02:42.752 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.750+0000 7ffb05ffb640 1 -- 172.21.15.57:0/2073409438 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3461509812 0 0) 0x7ffb081a8c60 con 0x7ffb0810a840 2024-06-11T14:02:42.753 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.750+0000 7ffb05ffb640 1 -- 172.21.15.57:0/2073409438 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb081a6820 con 0x7ffb0810a840 2024-06-11T14:02:42.753 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.751+0000 7ffb05ffb640 1 -- 172.21.15.57:0/2073409438 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1496913094 0 0) 0x7ffaf0003160 con 0x7ffb08102a40 2024-06-11T14:02:42.753 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.751+0000 7ffb05ffb640 1 -- 172.21.15.57:0/2073409438 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ffb081a8c60 con 0x7ffb08102a40 2024-06-11T14:02:42.753 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.751+0000 7ffb05ffb640 1 -- 172.21.15.57:0/2073409438 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7ffaf8001840 con 0x7ffb08102a40 2024-06-11T14:02:42.753 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.751+0000 7ffb05ffb640 1 -- 172.21.15.57:0/2073409438 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 782459602 0 0) 0x7ffb081a6820 con 0x7ffb0810a840 2024-06-11T14:02:42.753 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.751+0000 7ffb05ffb640 1 -- 172.21.15.57:0/2073409438 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ffaf0003160 con 0x7ffb0810a840 2024-06-11T14:02:42.753 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.751+0000 7ffb05ffb640 1 -- 172.21.15.57:0/2073409438 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7ffafc0017b0 con 0x7ffb0810a840 2024-06-11T14:02:42.753 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.751+0000 7ffb05ffb640 1 -- 172.21.15.57:0/2073409438 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1785152660 0 0) 0x7ffb081a7a20 con 0x7ffb0810e350 2024-06-11T14:02:42.753 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.751+0000 7ffb05ffb640 1 -- 172.21.15.57:0/2073409438 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb081a6820 con 0x7ffb0810e350 2024-06-11T14:02:42.753 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.751+0000 7ffb05ffb640 1 -- 172.21.15.57:0/2073409438 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 748817342 0 0) 0x7ffb081a8c60 con 0x7ffb08102a40 2024-06-11T14:02:42.753 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.751+0000 7ffb05ffb640 1 -- 172.21.15.57:0/2073409438 >> v1:172.21.15.57:6790/0 conn(0x7ffb0810a840 legacy=0x7ffb081a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:02:42.753 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.751+0000 7ffb05ffb640 1 -- 172.21.15.57:0/2073409438 >> v1:172.21.15.107:6789/0 conn(0x7ffb0810e350 legacy=0x7ffb081a4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:02:42.753 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.751+0000 7ffb05ffb640 1 -- 172.21.15.57:0/2073409438 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb081a9ea0 con 0x7ffb08102a40 2024-06-11T14:02:42.753 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.751+0000 7ffb0f15c640 1 -- 172.21.15.57:0/2073409438 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7ffb081a6a50 con 0x7ffb08102a40 2024-06-11T14:02:42.754 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.751+0000 7ffb05ffb640 1 -- 172.21.15.57:0/2073409438 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7ffaf8003a80 con 0x7ffb08102a40 2024-06-11T14:02:42.754 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.751+0000 7ffb05ffb640 1 -- 172.21.15.57:0/2073409438 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7ffaf8005be0 con 0x7ffb08102a40 2024-06-11T14:02:42.754 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.751+0000 7ffb0f15c640 1 -- 172.21.15.57:0/2073409438 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7ffb081a6f20 con 0x7ffb08102a40 2024-06-11T14:02:42.754 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.752+0000 7ffb0f15c640 1 -- 172.21.15.57:0/2073409438 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb08069320 con 0x7ffb08102a40 2024-06-11T14:02:42.755 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.753+0000 7ffb05ffb640 1 -- 172.21.15.57:0/2073409438 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 22) ==== 92138+0+0 (unknown 311317106 0 0) 0x7ffaf80035f0 con 0x7ffb08102a40 2024-06-11T14:02:42.755 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.753+0000 7ffb05ffb640 1 -- 172.21.15.57:0/2073409438 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(792..792 src has 265..792) ==== 7831+0+0 (unknown 1644198110 0 0) 0x7ffaf8089c30 con 0x7ffb08102a40 2024-06-11T14:02:42.757 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.758+0000 7ffb05ffb640 1 -- 172.21.15.57:0/2073409438 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7ffaf8056d70 con 0x7ffb08102a40 2024-06-11T14:02:42.947 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:42.947+0000 7ffb0f15c640 1 -- 172.21.15.57:0/2073409438 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "osd pool create", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "pg_num": 12} v 0) -- 0x7ffb08068650 con 0x7ffb08102a40 2024-06-11T14:02:43.299 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 14:02:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T14:02:43.608 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.608+0000 7ffb05ffb640 1 -- 172.21.15.57:0/2073409438 <== mon.0 v1:172.21.15.57:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "pg_num": 12}]=0 pool '3308aae0-a66b-4c2f-a482-8b20f197ecc4' created v793) ==== 176+0+0 (unknown 3200967973 0 0) 0x7ffaf805aa20 con 0x7ffb08102a40 2024-06-11T14:02:43.726 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.726+0000 7ffb0f15c640 1 -- 172.21.15.57:0/2073409438 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "osd pool create", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "pg_num": 12} v 0) -- 0x7ffb081085a0 con 0x7ffb08102a40 2024-06-11T14:02:43.726 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.727+0000 7ffb05ffb640 1 -- 172.21.15.57:0/2073409438 <== mon.0 v1:172.21.15.57:6789/0 11 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "pg_num": 12}]=0 pool '3308aae0-a66b-4c2f-a482-8b20f197ecc4' already exists v793) ==== 183+0+0 (unknown 2975574810 0 0) 0x7ffaf805f2e0 con 0x7ffb08102a40 2024-06-11T14:02:43.726 INFO:tasks.workunit.client.0.smithi057.stderr:pool '3308aae0-a66b-4c2f-a482-8b20f197ecc4' already exists 2024-06-11T14:02:43.731 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.729+0000 7ffb0f15c640 1 -- 172.21.15.57:0/2073409438 >> v1:172.21.15.57:6800/2077416415 conn(0x7ffaf006eee0 legacy=0x7ffaf00713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:02:43.731 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.729+0000 7ffb0f15c640 1 -- 172.21.15.57:0/2073409438 >> v1:172.21.15.57:6789/0 conn(0x7ffb08102a40 legacy=0x7ffb08109fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:02:43.731 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.729+0000 7ffb0f15c640 1 -- 172.21.15.57:0/2073409438 shutdown_connections 2024-06-11T14:02:43.731 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.729+0000 7ffb0f15c640 1 -- 172.21.15.57:0/2073409438 >> 172.21.15.57:0/2073409438 conn(0x7ffb08077400 msgr2=0x7ffb0810f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T14:02:43.731 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.729+0000 7ffb0f15c640 1 -- 172.21.15.57:0/2073409438 shutdown_connections 2024-06-11T14:02:43.732 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.729+0000 7ffb0f15c640 1 -- 172.21.15.57:0/2073409438 wait complete. 2024-06-11T14:02:43.734 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:43 smithi057 ceph-mon[29805]: pgmap v1740: 176 pgs: 176 active+clean; 724 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:02:43.734 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:43 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2211668086' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "0"}]': finished 2024-06-11T14:02:43.734 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:43 smithi057 ceph-mon[29805]: osdmap e791: 8 total, 8 up, 8 in 2024-06-11T14:02:43.734 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:43 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2211668086' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "0"}]: dispatch 2024-06-11T14:02:43.734 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:43 smithi057 ceph-mon[29805]: pool '5bd44950-56a4-47a9-a376-eee5f24dbed6' no longer out of quota; removing NO_QUOTA flag 2024-06-11T14:02:43.735 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:43 smithi057 ceph-mon[29805]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-06-11T14:02:43.735 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:43 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2211668086' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "0"}]': finished 2024-06-11T14:02:43.735 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:43 smithi057 ceph-mon[29805]: osdmap e792: 8 total, 8 up, 8 in 2024-06-11T14:02:43.735 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:43 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2073409438' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "pg_num": 12}]: dispatch 2024-06-11T14:02:43.735 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:43 smithi057 ceph-mon[40588]: pgmap v1740: 176 pgs: 176 active+clean; 724 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:02:43.735 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:43 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2211668086' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "0"}]': finished 2024-06-11T14:02:43.735 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:43 smithi057 ceph-mon[40588]: osdmap e791: 8 total, 8 up, 8 in 2024-06-11T14:02:43.735 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:43 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2211668086' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "0"}]: dispatch 2024-06-11T14:02:43.735 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:43 smithi057 ceph-mon[40588]: pool '5bd44950-56a4-47a9-a376-eee5f24dbed6' no longer out of quota; removing NO_QUOTA flag 2024-06-11T14:02:43.736 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:43 smithi057 ceph-mon[40588]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-06-11T14:02:43.736 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:43 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2211668086' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "0"}]': finished 2024-06-11T14:02:43.736 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:43 smithi057 ceph-mon[40588]: osdmap e792: 8 total, 8 up, 8 in 2024-06-11T14:02:43.736 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:43 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2073409438' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "pg_num": 12}]: dispatch 2024-06-11T14:02:43.738 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph osd pool application enable 3308aae0-a66b-4c2f-a482-8b20f197ecc4 rados 2024-06-11T14:02:43.800 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:43 smithi107 ceph-mon[33178]: pgmap v1740: 176 pgs: 176 active+clean; 724 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:02:43.800 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:43 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2211668086' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "0"}]': finished 2024-06-11T14:02:43.800 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:43 smithi107 ceph-mon[33178]: osdmap e791: 8 total, 8 up, 8 in 2024-06-11T14:02:43.800 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:43 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2211668086' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "0"}]: dispatch 2024-06-11T14:02:43.800 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:43 smithi107 ceph-mon[33178]: pool '5bd44950-56a4-47a9-a376-eee5f24dbed6' no longer out of quota; removing NO_QUOTA flag 2024-06-11T14:02:43.800 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:43 smithi107 ceph-mon[33178]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-06-11T14:02:43.800 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:43 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2211668086' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "0"}]': finished 2024-06-11T14:02:43.800 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:43 smithi107 ceph-mon[33178]: osdmap e792: 8 total, 8 up, 8 in 2024-06-11T14:02:43.800 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:43 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2073409438' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "pg_num": 12}]: dispatch 2024-06-11T14:02:43.824 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.824+0000 7f4c2d2ed640 1 -- 172.21.15.57:0/3454996021 >> v1:172.21.15.57:6790/0 conn(0x7f4c28107820 legacy=0x7f4c28107c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:02:43.824 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.824+0000 7f4c2d2ed640 1 -- 172.21.15.57:0/3454996021 shutdown_connections 2024-06-11T14:02:43.824 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.825+0000 7f4c2d2ed640 1 -- 172.21.15.57:0/3454996021 >> 172.21.15.57:0/3454996021 conn(0x7f4c280fd300 msgr2=0x7f4c280ff720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T14:02:43.824 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.825+0000 7f4c2d2ed640 1 -- 172.21.15.57:0/3454996021 shutdown_connections 2024-06-11T14:02:43.825 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.825+0000 7f4c2d2ed640 1 -- 172.21.15.57:0/3454996021 wait complete. 2024-06-11T14:02:43.825 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.825+0000 7f4c2d2ed640 1 Processor -- start 2024-06-11T14:02:43.827 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.825+0000 7f4c2d2ed640 1 -- start start 2024-06-11T14:02:43.828 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.825+0000 7f4c2d2ed640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f4c281a6840 con 0x7f4c2810a6c0 2024-06-11T14:02:43.828 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.826+0000 7f4c2d2ed640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f4c281a7a40 con 0x7f4c28107820 2024-06-11T14:02:43.828 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.826+0000 7f4c2d2ed640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f4c281a8c80 con 0x7f4c2810e290 2024-06-11T14:02:43.828 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.826+0000 7f4c267fc640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f4c2810a6c0 0x7f4c281a16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:52142/0 (socket says 172.21.15.57:52142) 2024-06-11T14:02:43.828 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.826+0000 7f4c267fc640 1 -- 172.21.15.57:0/3259380636 learned_addr learned my addr 172.21.15.57:0/3259380636 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T14:02:43.828 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.826+0000 7f4c0ffff640 1 -- 172.21.15.57:0/3259380636 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3545070743 0 0) 0x7f4c281a8c80 con 0x7f4c2810e290 2024-06-11T14:02:43.828 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.826+0000 7f4c0ffff640 1 -- 172.21.15.57:0/3259380636 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c10003160 con 0x7f4c2810e290 2024-06-11T14:02:43.828 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.826+0000 7f4c0ffff640 1 -- 172.21.15.57:0/3259380636 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1638425455 0 0) 0x7f4c281a6840 con 0x7f4c2810a6c0 2024-06-11T14:02:43.828 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.826+0000 7f4c0ffff640 1 -- 172.21.15.57:0/3259380636 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c281a8c80 con 0x7f4c2810a6c0 2024-06-11T14:02:43.828 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.826+0000 7f4c0ffff640 1 -- 172.21.15.57:0/3259380636 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 433120973 0 0) 0x7f4c281a7a40 con 0x7f4c28107820 2024-06-11T14:02:43.828 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.826+0000 7f4c0ffff640 1 -- 172.21.15.57:0/3259380636 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c281a6840 con 0x7f4c28107820 2024-06-11T14:02:43.829 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.826+0000 7f4c0ffff640 1 -- 172.21.15.57:0/3259380636 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4277017086 0 0) 0x7f4c10003160 con 0x7f4c2810e290 2024-06-11T14:02:43.829 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.826+0000 7f4c0ffff640 1 -- 172.21.15.57:0/3259380636 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4c281a7a40 con 0x7f4c2810e290 2024-06-11T14:02:43.829 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.826+0000 7f4c0ffff640 1 -- 172.21.15.57:0/3259380636 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f4c20001c30 con 0x7f4c2810e290 2024-06-11T14:02:43.829 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.826+0000 7f4c0ffff640 1 -- 172.21.15.57:0/3259380636 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1183221852 0 0) 0x7f4c281a8c80 con 0x7f4c2810a6c0 2024-06-11T14:02:43.829 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.826+0000 7f4c0ffff640 1 -- 172.21.15.57:0/3259380636 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4c10003160 con 0x7f4c2810a6c0 2024-06-11T14:02:43.829 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.826+0000 7f4c0ffff640 1 -- 172.21.15.57:0/3259380636 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f4c1c001230 con 0x7f4c2810a6c0 2024-06-11T14:02:43.829 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.826+0000 7f4c0ffff640 1 -- 172.21.15.57:0/3259380636 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2068281749 0 0) 0x7f4c281a6840 con 0x7f4c28107820 2024-06-11T14:02:43.829 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.826+0000 7f4c0ffff640 1 -- 172.21.15.57:0/3259380636 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4c281a8c80 con 0x7f4c28107820 2024-06-11T14:02:43.829 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.826+0000 7f4c0ffff640 1 -- 172.21.15.57:0/3259380636 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f4c18001980 con 0x7f4c28107820 2024-06-11T14:02:43.829 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.826+0000 7f4c0ffff640 1 -- 172.21.15.57:0/3259380636 <== mon.2 v1:172.21.15.57:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2242479469 0 0) 0x7f4c281a7a40 con 0x7f4c2810e290 2024-06-11T14:02:43.829 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.826+0000 7f4c0ffff640 1 -- 172.21.15.57:0/3259380636 >> v1:172.21.15.107:6789/0 conn(0x7f4c28107820 legacy=0x7f4c2810da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:02:43.829 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.826+0000 7f4c0ffff640 1 -- 172.21.15.57:0/3259380636 >> v1:172.21.15.57:6789/0 conn(0x7f4c2810a6c0 legacy=0x7f4c281a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:02:43.829 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.826+0000 7f4c0ffff640 1 -- 172.21.15.57:0/3259380636 --> v1:172.21.15.57:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c281a9ec0 con 0x7f4c2810e290 2024-06-11T14:02:43.830 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.826+0000 7f4c2d2ed640 1 -- 172.21.15.57:0/3259380636 --> v1:172.21.15.57:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f4c281a6a70 con 0x7f4c2810e290 2024-06-11T14:02:43.830 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.826+0000 7f4c2d2ed640 1 -- 172.21.15.57:0/3259380636 --> v1:172.21.15.57:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f4c281a6f40 con 0x7f4c2810e290 2024-06-11T14:02:43.830 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.826+0000 7f4c0ffff640 1 -- 172.21.15.57:0/3259380636 <== mon.2 v1:172.21.15.57:6790/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f4c20003e20 con 0x7f4c2810e290 2024-06-11T14:02:43.830 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.827+0000 7f4c0ffff640 1 -- 172.21.15.57:0/3259380636 <== mon.2 v1:172.21.15.57:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f4c20004d20 con 0x7f4c2810e290 2024-06-11T14:02:43.830 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.827+0000 7f4c2d2ed640 1 -- 172.21.15.57:0/3259380636 --> v1:172.21.15.57:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c14005080 con 0x7f4c2810e290 2024-06-11T14:02:43.830 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.828+0000 7f4c0ffff640 1 -- 172.21.15.57:0/3259380636 <== mon.2 v1:172.21.15.57:6790/0 7 ==== mgrmap(e 22) ==== 92138+0+0 (unknown 311317106 0 0) 0x7f4c20003990 con 0x7f4c2810e290 2024-06-11T14:02:43.830 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.828+0000 7f4c0ffff640 1 -- 172.21.15.57:0/3259380636 <== mon.2 v1:172.21.15.57:6790/0 8 ==== osd_map(793..793 src has 265..793) ==== 8206+0+0 (unknown 1072792659 0 0) 0x7f4c20088cf0 con 0x7f4c2810e290 2024-06-11T14:02:43.832 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:43.833+0000 7f4c0ffff640 1 -- 172.21.15.57:0/3259380636 <== mon.2 v1:172.21.15.57:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f4c20055c70 con 0x7f4c2810e290 2024-06-11T14:02:44.020 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:44.020+0000 7f4c2d2ed640 1 -- 172.21.15.57:0/3259380636 --> v1:172.21.15.57:6790/0 -- mon_command({"prefix": "osd pool application enable", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "app": "rados"} v 0) -- 0x7f4c14005370 con 0x7f4c2810e290 2024-06-11T14:02:44.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 14:02:43 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:14:02:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T14:02:44.631 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:44.632+0000 7f4c0ffff640 1 -- 172.21.15.57:0/3259380636 <== mon.2 v1:172.21.15.57:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "app": "rados"}]=0 enabled application 'rados' on pool '3308aae0-a66b-4c2f-a482-8b20f197ecc4' v794) ==== 213+0+0 (unknown 650218203 0 0) 0x7f4c20059920 con 0x7f4c2810e290 2024-06-11T14:02:44.752 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:44.752+0000 7f4c2d2ed640 1 -- 172.21.15.57:0/3259380636 --> v1:172.21.15.57:6790/0 -- mon_command({"prefix": "osd pool application enable", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "app": "rados"} v 0) -- 0x7f4c140010d0 con 0x7f4c2810e290 2024-06-11T14:02:44.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:44 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:02:44.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:44 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2073409438' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "pg_num": 12}]': finished 2024-06-11T14:02:44.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:44 smithi057 ceph-mon[29805]: osdmap e793: 8 total, 8 up, 8 in 2024-06-11T14:02:44.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:44 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2073409438' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "pg_num": 12}]: dispatch 2024-06-11T14:02:44.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:44 smithi057 ceph-mon[29805]: pgmap v1744: 188 pgs: 12 unknown, 176 active+clean; 724 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-06-11T14:02:44.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:44 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3259380636' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "app": "rados"}]: dispatch 2024-06-11T14:02:44.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:44 smithi057 ceph-mon[29805]: from='client.58871 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "app": "rados"}]: dispatch 2024-06-11T14:02:44.788 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:44 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:02:44.788 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:44 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2073409438' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "pg_num": 12}]': finished 2024-06-11T14:02:44.788 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:44 smithi057 ceph-mon[40588]: osdmap e793: 8 total, 8 up, 8 in 2024-06-11T14:02:44.788 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:44 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2073409438' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "pg_num": 12}]: dispatch 2024-06-11T14:02:44.788 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:44 smithi057 ceph-mon[40588]: pgmap v1744: 188 pgs: 12 unknown, 176 active+clean; 724 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-06-11T14:02:44.788 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:44 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3259380636' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "app": "rados"}]: dispatch 2024-06-11T14:02:44.788 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:44 smithi057 ceph-mon[40588]: from='client.58871 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "app": "rados"}]: dispatch 2024-06-11T14:02:44.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:44 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:02:44.800 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:44 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2073409438' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "pg_num": 12}]': finished 2024-06-11T14:02:44.800 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:44 smithi107 ceph-mon[33178]: osdmap e793: 8 total, 8 up, 8 in 2024-06-11T14:02:44.800 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:44 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2073409438' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "pg_num": 12}]: dispatch 2024-06-11T14:02:44.800 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:44 smithi107 ceph-mon[33178]: pgmap v1744: 188 pgs: 12 unknown, 176 active+clean; 724 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-06-11T14:02:44.800 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:44 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3259380636' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "app": "rados"}]: dispatch 2024-06-11T14:02:44.800 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:44 smithi107 ceph-mon[33178]: from='client.58871 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "app": "rados"}]: dispatch 2024-06-11T14:02:45.637 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:45.637+0000 7f4c0ffff640 1 -- 172.21.15.57:0/3259380636 <== mon.2 v1:172.21.15.57:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "app": "rados"}]=0 enabled application 'rados' on pool '3308aae0-a66b-4c2f-a482-8b20f197ecc4' v795) ==== 213+0+0 (unknown 484473917 0 0) 0x7f4c2005e1e0 con 0x7f4c2810e290 2024-06-11T14:02:45.637 INFO:tasks.workunit.client.0.smithi057.stderr:enabled application 'rados' on pool '3308aae0-a66b-4c2f-a482-8b20f197ecc4' 2024-06-11T14:02:45.640 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:45.640+0000 7f4c2d2ed640 1 -- 172.21.15.57:0/3259380636 >> v1:172.21.15.57:6800/2077416415 conn(0x7f4c1006eea0 legacy=0x7f4c10071360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:02:45.640 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:45.640+0000 7f4c2d2ed640 1 -- 172.21.15.57:0/3259380636 >> v1:172.21.15.57:6790/0 conn(0x7f4c2810e290 legacy=0x7f4c281a4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:02:45.640 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:45.640+0000 7f4c2d2ed640 1 -- 172.21.15.57:0/3259380636 shutdown_connections 2024-06-11T14:02:45.640 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:45.640+0000 7f4c2d2ed640 1 -- 172.21.15.57:0/3259380636 >> 172.21.15.57:0/3259380636 conn(0x7f4c280fd300 msgr2=0x7f4c28110ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T14:02:45.640 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:45.641+0000 7f4c2d2ed640 1 -- 172.21.15.57:0/3259380636 shutdown_connections 2024-06-11T14:02:45.640 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:45.641+0000 7f4c2d2ed640 1 -- 172.21.15.57:0/3259380636 wait complete. 2024-06-11T14:02:45.651 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph osd pool set-quota 3308aae0-a66b-4c2f-a482-8b20f197ecc4 max_objects 10 2024-06-11T14:02:45.743 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:45.744+0000 7f06afe66640 1 -- 172.21.15.57:0/1166236189 >> v1:172.21.15.57:6789/0 conn(0x7f06a8106720 legacy=0x7f06a8106b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:02:45.743 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:45.744+0000 7f06afe66640 1 -- 172.21.15.57:0/1166236189 shutdown_connections 2024-06-11T14:02:45.743 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:45.744+0000 7f06afe66640 1 -- 172.21.15.57:0/1166236189 >> 172.21.15.57:0/1166236189 conn(0x7f06a8077400 msgr2=0x7f06a80fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T14:02:45.744 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:45.744+0000 7f06afe66640 1 -- 172.21.15.57:0/1166236189 shutdown_connections 2024-06-11T14:02:45.744 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:45.744+0000 7f06afe66640 1 -- 172.21.15.57:0/1166236189 wait complete. 2024-06-11T14:02:45.746 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:45.744+0000 7f06afe66640 1 Processor -- start 2024-06-11T14:02:45.746 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:45.744+0000 7f06afe66640 1 -- start start 2024-06-11T14:02:45.746 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:45.745+0000 7f06afe66640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f06a81a56b0 con 0x7f06a8106720 2024-06-11T14:02:45.746 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:45.745+0000 7f06afe66640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f06a81a68f0 con 0x7f06a810d190 2024-06-11T14:02:45.746 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:45.745+0000 7f06afe66640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f06a81a7b30 con 0x7f06a81095c0 2024-06-11T14:02:45.746 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:45.745+0000 7f06ad401640 1 --1- >> v1:172.21.15.57:6790/0 conn(0x7f06a81095c0 0x7f06a81a05e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6790/0 says I am v1:172.21.15.57:42574/0 (socket says 172.21.15.57:42574) 2024-06-11T14:02:45.746 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:45.745+0000 7f06ad401640 1 -- 172.21.15.57:0/3793268217 learned_addr learned my addr 172.21.15.57:0/3793268217 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T14:02:45.746 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:45.745+0000 7f06adc02640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f06a8106720 0x7f06a810c900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:52164/0 (socket says 172.21.15.57:52164) 2024-06-11T14:02:45.747 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:45.745+0000 7f06a6ffd640 1 -- 172.21.15.57:0/3793268217 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 373807106 0 0) 0x7f06a81a7b30 con 0x7f06a81095c0 2024-06-11T14:02:45.747 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:45.745+0000 7f06a6ffd640 1 -- 172.21.15.57:0/3793268217 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0690004920 con 0x7f06a81095c0 2024-06-11T14:02:45.747 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:45.745+0000 7f06a6ffd640 1 -- 172.21.15.57:0/3793268217 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3288973306 0 0) 0x7f06a81a56b0 con 0x7f06a8106720 2024-06-11T14:02:45.747 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:45.745+0000 7f06a6ffd640 1 -- 172.21.15.57:0/3793268217 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f06a81a7b30 con 0x7f06a8106720 2024-06-11T14:02:45.747 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:45.745+0000 7f06a6ffd640 1 -- 172.21.15.57:0/3793268217 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 932136957 0 0) 0x7f06a81a68f0 con 0x7f06a810d190 2024-06-11T14:02:45.747 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:45.745+0000 7f06a6ffd640 1 -- 172.21.15.57:0/3793268217 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f06a81a56b0 con 0x7f06a810d190 2024-06-11T14:02:45.747 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:45.745+0000 7f06a6ffd640 1 -- 172.21.15.57:0/3793268217 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3819673196 0 0) 0x7f06a81a7b30 con 0x7f06a8106720 2024-06-11T14:02:45.747 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:45.745+0000 7f06a6ffd640 1 -- 172.21.15.57:0/3793268217 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f06a81a68f0 con 0x7f06a8106720 2024-06-11T14:02:45.747 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:45.745+0000 7f06a6ffd640 1 -- 172.21.15.57:0/3793268217 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4202022061 0 0) 0x7f0690004920 con 0x7f06a81095c0 2024-06-11T14:02:45.747 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:45.745+0000 7f06a6ffd640 1 -- 172.21.15.57:0/3793268217 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f06a81a7b30 con 0x7f06a81095c0 2024-06-11T14:02:45.747 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:45.745+0000 7f06a6ffd640 1 -- 172.21.15.57:0/3793268217 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f06a00043a0 con 0x7f06a8106720 2024-06-11T14:02:45.747 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:45.745+0000 7f06a6ffd640 1 -- 172.21.15.57:0/3793268217 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f069c0017f0 con 0x7f06a81095c0 2024-06-11T14:02:45.747 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:45.745+0000 7f06a6ffd640 1 -- 172.21.15.57:0/3793268217 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3363433904 0 0) 0x7f06a81a56b0 con 0x7f06a810d190 2024-06-11T14:02:45.747 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:45.745+0000 7f06a6ffd640 1 -- 172.21.15.57:0/3793268217 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0690004920 con 0x7f06a810d190 2024-06-11T14:02:45.747 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:45.745+0000 7f06a6ffd640 1 -- 172.21.15.57:0/3793268217 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2973058690 0 0) 0x7f06a81a68f0 con 0x7f06a8106720 2024-06-11T14:02:45.748 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:45.746+0000 7f06a6ffd640 1 -- 172.21.15.57:0/3793268217 >> v1:172.21.15.57:6790/0 conn(0x7f06a81095c0 legacy=0x7f06a81a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:02:45.748 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:45.746+0000 7f06a6ffd640 1 -- 172.21.15.57:0/3793268217 >> v1:172.21.15.107:6789/0 conn(0x7f06a810d190 legacy=0x7f06a81a3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:02:45.748 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:45.746+0000 7f06a6ffd640 1 -- 172.21.15.57:0/3793268217 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a81a8d70 con 0x7f06a8106720 2024-06-11T14:02:45.748 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:45.746+0000 7f06afe66640 1 -- 172.21.15.57:0/3793268217 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f06a81a7d60 con 0x7f06a8106720 2024-06-11T14:02:45.748 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:45.746+0000 7f06afe66640 1 -- 172.21.15.57:0/3793268217 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f06a81a8230 con 0x7f06a8106720 2024-06-11T14:02:45.748 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:45.746+0000 7f06a6ffd640 1 -- 172.21.15.57:0/3793268217 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f06a0001ab0 con 0x7f06a8106720 2024-06-11T14:02:45.748 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:45.746+0000 7f06a6ffd640 1 -- 172.21.15.57:0/3793268217 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f06a0004b50 con 0x7f06a8106720 2024-06-11T14:02:45.748 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:45.747+0000 7f06afe66640 1 -- 172.21.15.57:0/3793268217 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0694005080 con 0x7f06a8106720 2024-06-11T14:02:45.748 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:45.747+0000 7f06a6ffd640 1 -- 172.21.15.57:0/3793268217 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 22) ==== 92138+0+0 (unknown 311317106 0 0) 0x7f06a0004d90 con 0x7f06a8106720 2024-06-11T14:02:45.749 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:45.748+0000 7f06a6ffd640 1 -- 172.21.15.57:0/3793268217 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(795..795 src has 265..795) ==== 8219+0+0 (unknown 263837658 0 0) 0x7f06a0089bb0 con 0x7f06a8106720 2024-06-11T14:02:45.753 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:45.754+0000 7f06a6ffd640 1 -- 172.21.15.57:0/3793268217 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f06a0056b20 con 0x7f06a8106720 2024-06-11T14:02:45.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:45 smithi057 ceph-mon[40588]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T14:02:45.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:45 smithi057 ceph-mon[40588]: from='client.58871 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "app": "rados"}]': finished 2024-06-11T14:02:45.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:45 smithi057 ceph-mon[40588]: osdmap e794: 8 total, 8 up, 8 in 2024-06-11T14:02:45.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:45 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3259380636' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "app": "rados"}]: dispatch 2024-06-11T14:02:45.786 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:45 smithi057 ceph-mon[40588]: from='client.58871 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "app": "rados"}]: dispatch 2024-06-11T14:02:45.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:45 smithi057 ceph-mon[29805]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T14:02:45.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:45 smithi057 ceph-mon[29805]: from='client.58871 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "app": "rados"}]': finished 2024-06-11T14:02:45.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:45 smithi057 ceph-mon[29805]: osdmap e794: 8 total, 8 up, 8 in 2024-06-11T14:02:45.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:45 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3259380636' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "app": "rados"}]: dispatch 2024-06-11T14:02:45.787 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:45 smithi057 ceph-mon[29805]: from='client.58871 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "app": "rados"}]: dispatch 2024-06-11T14:02:45.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:45 smithi107 ceph-mon[33178]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T14:02:45.799 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:45 smithi107 ceph-mon[33178]: from='client.58871 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "app": "rados"}]': finished 2024-06-11T14:02:45.800 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:45 smithi107 ceph-mon[33178]: osdmap e794: 8 total, 8 up, 8 in 2024-06-11T14:02:45.800 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:45 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3259380636' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "app": "rados"}]: dispatch 2024-06-11T14:02:45.800 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:45 smithi107 ceph-mon[33178]: from='client.58871 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "app": "rados"}]: dispatch 2024-06-11T14:02:45.938 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:45.938+0000 7f06afe66640 1 -- 172.21.15.57:0/3793268217 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "field": "max_objects", "val": "10"} v 0) -- 0x7f0694005370 con 0x7f06a8106720 2024-06-11T14:02:46.629 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:46.630+0000 7f06a6ffd640 1 -- 172.21.15.57:0/3793268217 <== mon.0 v1:172.21.15.57:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 3308aae0-a66b-4c2f-a482-8b20f197ecc4 v796) ==== 223+0+0 (unknown 637596756 0 0) 0x7f06a005a7d0 con 0x7f06a8106720 2024-06-11T14:02:46.746 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:46.747+0000 7f06afe66640 1 -- 172.21.15.57:0/3793268217 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "field": "max_objects", "val": "10"} v 0) -- 0x7f06940010d0 con 0x7f06a8106720 2024-06-11T14:02:47.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:46 smithi057 ceph-mon[29805]: from='client.58871 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "app": "rados"}]': finished 2024-06-11T14:02:47.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:46 smithi057 ceph-mon[29805]: osdmap e795: 8 total, 8 up, 8 in 2024-06-11T14:02:47.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:46 smithi057 ceph-mon[29805]: pgmap v1747: 188 pgs: 12 unknown, 176 active+clean; 724 KiB data, 897 MiB used, 714 GiB / 715 GiB avail 2024-06-11T14:02:47.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:46 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3793268217' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "field": "max_objects", "val": "10"}]: dispatch 2024-06-11T14:02:47.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:46 smithi057 ceph-mon[40588]: from='client.58871 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "app": "rados"}]': finished 2024-06-11T14:02:47.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:46 smithi057 ceph-mon[40588]: osdmap e795: 8 total, 8 up, 8 in 2024-06-11T14:02:47.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:46 smithi057 ceph-mon[40588]: pgmap v1747: 188 pgs: 12 unknown, 176 active+clean; 724 KiB data, 897 MiB used, 714 GiB / 715 GiB avail 2024-06-11T14:02:47.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:46 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3793268217' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "field": "max_objects", "val": "10"}]: dispatch 2024-06-11T14:02:47.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:46 smithi107 ceph-mon[33178]: from='client.58871 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "app": "rados"}]': finished 2024-06-11T14:02:47.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:46 smithi107 ceph-mon[33178]: osdmap e795: 8 total, 8 up, 8 in 2024-06-11T14:02:47.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:46 smithi107 ceph-mon[33178]: pgmap v1747: 188 pgs: 12 unknown, 176 active+clean; 724 KiB data, 897 MiB used, 714 GiB / 715 GiB avail 2024-06-11T14:02:47.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:46 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3793268217' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "field": "max_objects", "val": "10"}]: dispatch 2024-06-11T14:02:47.644 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:47.644+0000 7f06a6ffd640 1 -- 172.21.15.57:0/3793268217 <== mon.0 v1:172.21.15.57:6789/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 3308aae0-a66b-4c2f-a482-8b20f197ecc4 v797) ==== 223+0+0 (unknown 1866813732 0 0) 0x7f06a005f090 con 0x7f06a8106720 2024-06-11T14:02:47.644 INFO:tasks.workunit.client.0.smithi057.stderr:set-quota max_objects = 10 for pool 3308aae0-a66b-4c2f-a482-8b20f197ecc4 2024-06-11T14:02:47.648 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:47.649+0000 7f06afe66640 1 -- 172.21.15.57:0/3793268217 >> v1:172.21.15.57:6800/2077416415 conn(0x7f06900711f0 legacy=0x7f06900736b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:02:47.648 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:47.649+0000 7f06afe66640 1 -- 172.21.15.57:0/3793268217 >> v1:172.21.15.57:6789/0 conn(0x7f06a8106720 legacy=0x7f06a810c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:02:47.648 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:47.649+0000 7f06afe66640 1 -- 172.21.15.57:0/3793268217 shutdown_connections 2024-06-11T14:02:47.648 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:47.649+0000 7f06afe66640 1 -- 172.21.15.57:0/3793268217 >> 172.21.15.57:0/3793268217 conn(0x7f06a8077400 msgr2=0x7f06a810d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T14:02:47.648 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:47.649+0000 7f06afe66640 1 -- 172.21.15.57:0/3793268217 shutdown_connections 2024-06-11T14:02:47.649 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:02:47.649+0000 7f06afe66640 1 -- 172.21.15.57:0/3793268217 wait complete. 2024-06-11T14:02:47.660 INFO:tasks.workunit.client.0.smithi057.stderr:+ sleep 30 2024-06-11T14:02:48.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:47 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3793268217' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "field": "max_objects", "val": "10"}]': finished 2024-06-11T14:02:48.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:47 smithi057 ceph-mon[29805]: osdmap e796: 8 total, 8 up, 8 in 2024-06-11T14:02:48.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:47 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3793268217' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "field": "max_objects", "val": "10"}]: dispatch 2024-06-11T14:02:48.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:47 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3793268217' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "field": "max_objects", "val": "10"}]': finished 2024-06-11T14:02:48.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:47 smithi057 ceph-mon[40588]: osdmap e796: 8 total, 8 up, 8 in 2024-06-11T14:02:48.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:47 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3793268217' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "field": "max_objects", "val": "10"}]: dispatch 2024-06-11T14:02:48.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:47 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3793268217' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "field": "max_objects", "val": "10"}]': finished 2024-06-11T14:02:48.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:47 smithi107 ceph-mon[33178]: osdmap e796: 8 total, 8 up, 8 in 2024-06-11T14:02:48.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:47 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3793268217' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "field": "max_objects", "val": "10"}]: dispatch 2024-06-11T14:02:49.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:48 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/3793268217' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "field": "max_objects", "val": "10"}]': finished 2024-06-11T14:02:49.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:48 smithi057 ceph-mon[29805]: osdmap e797: 8 total, 8 up, 8 in 2024-06-11T14:02:49.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:48 smithi057 ceph-mon[29805]: pgmap v1750: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-06-11T14:02:49.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:48 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/3793268217' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "field": "max_objects", "val": "10"}]': finished 2024-06-11T14:02:49.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:48 smithi057 ceph-mon[40588]: osdmap e797: 8 total, 8 up, 8 in 2024-06-11T14:02:49.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:48 smithi057 ceph-mon[40588]: pgmap v1750: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-06-11T14:02:49.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:48 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/3793268217' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "field": "max_objects", "val": "10"}]': finished 2024-06-11T14:02:49.050 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:48 smithi107 ceph-mon[33178]: osdmap e797: 8 total, 8 up, 8 in 2024-06-11T14:02:49.050 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:48 smithi107 ceph-mon[33178]: pgmap v1750: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-06-11T14:02:51.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:50 smithi057 ceph-mon[29805]: pgmap v1751: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 966 B/s rd, 4.5 KiB/s wr, 1 op/s 2024-06-11T14:02:51.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:50 smithi057 ceph-mon[40588]: pgmap v1751: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 966 B/s rd, 4.5 KiB/s wr, 1 op/s 2024-06-11T14:02:51.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:50 smithi107 ceph-mon[33178]: pgmap v1751: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 966 B/s rd, 4.5 KiB/s wr, 1 op/s 2024-06-11T14:02:53.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:53 smithi057 ceph-mon[29805]: pgmap v1752: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 3.8 KiB/s wr, 1 op/s 2024-06-11T14:02:53.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:53 smithi057 ceph-mon[29805]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T14:02:53.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:53 smithi057 ceph-mon[40588]: pgmap v1752: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 3.8 KiB/s wr, 1 op/s 2024-06-11T14:02:53.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:53 smithi057 ceph-mon[40588]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T14:02:53.300 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 14:02:52 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T14:02:53.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:53 smithi107 ceph-mon[33178]: pgmap v1752: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 3.8 KiB/s wr, 1 op/s 2024-06-11T14:02:53.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:53 smithi107 ceph-mon[33178]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-11T14:02:54.003 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 14:02:53 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:14:02:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T14:02:54.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:54 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:02:54.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:54 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T14:02:54.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:54 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:02:54.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:54 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:02:54.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:54 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T14:02:54.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:54 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:02:54.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:54 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:02:54.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:54 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T14:02:54.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:54 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:02:55.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:55 smithi057 ceph-mon[29805]: pgmap v1753: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-06-11T14:02:55.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:55 smithi057 ceph-mon[40588]: pgmap v1753: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-06-11T14:02:55.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:55 smithi107 ceph-mon[33178]: pgmap v1753: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-06-11T14:02:57.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:57 smithi057 ceph-mon[29805]: pgmap v1754: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 550 B/s rd, 0 op/s 2024-06-11T14:02:57.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:57 smithi057 ceph-mon[40588]: pgmap v1754: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 550 B/s rd, 0 op/s 2024-06-11T14:02:57.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:57 smithi107 ceph-mon[33178]: pgmap v1754: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 550 B/s rd, 0 op/s 2024-06-11T14:02:59.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:58 smithi057 ceph-mon[29805]: pgmap v1755: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 995 B/s rd, 0 op/s 2024-06-11T14:02:59.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:58 smithi057 ceph-mon[40588]: pgmap v1755: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 995 B/s rd, 0 op/s 2024-06-11T14:02:59.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:58 smithi107 ceph-mon[33178]: pgmap v1755: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 995 B/s rd, 0 op/s 2024-06-11T14:03:00.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:59 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T14:03:00.114 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:59 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T14:03:00.114 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:59 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T14:03:00.114 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:02:59 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T14:03:00.115 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:59 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T14:03:00.115 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:59 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T14:03:00.115 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:59 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T14:03:00.115 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:02:59 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T14:03:00.115 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:59 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T14:03:00.115 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:59 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T14:03:00.115 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:59 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T14:03:00.115 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:02:59 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T14:03:01.036 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:00 smithi057 ceph-mon[29805]: pgmap v1756: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:03:01.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:00 smithi057 ceph-mon[40588]: pgmap v1756: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:03:01.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:00 smithi107 ceph-mon[33178]: pgmap v1756: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:03:03.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:02 smithi057 ceph-mon[40588]: pgmap v1757: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:03:03.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:02 smithi057 ceph-mon[29805]: pgmap v1757: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:03:03.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:02 smithi107 ceph-mon[33178]: pgmap v1757: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:03:03.299 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 14:03:02 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T14:03:04.016 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 14:03:03 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:14:03:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T14:03:04.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:04 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:03:04.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:04 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:03:04.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:04 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:03:05.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:05 smithi057 ceph-mon[29805]: pgmap v1758: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:03:05.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:05 smithi057 ceph-mon[40588]: pgmap v1758: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:03:05.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:05 smithi107 ceph-mon[33178]: pgmap v1758: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:03:07.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:07 smithi107 ceph-mon[33178]: pgmap v1759: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:03:07.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:07 smithi057 ceph-mon[29805]: pgmap v1759: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:03:07.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:07 smithi057 ceph-mon[40588]: pgmap v1759: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:03:09.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:09 smithi057 ceph-mon[29805]: pgmap v1760: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:03:09.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:09 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:03:09.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:09 smithi057 ceph-mon[40588]: pgmap v1760: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:03:09.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:09 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:03:09.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:09 smithi107 ceph-mon[33178]: pgmap v1760: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:03:09.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:09 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:03:11.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:11 smithi057 ceph-mon[29805]: pgmap v1761: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:03:11.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:11 smithi057 ceph-mon[40588]: pgmap v1761: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:03:11.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:11 smithi107 ceph-mon[33178]: pgmap v1761: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:03:13.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:13 smithi107 ceph-mon[33178]: pgmap v1762: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:03:13.300 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 14:03:12 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T14:03:13.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:13 smithi057 ceph-mon[29805]: pgmap v1762: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:03:13.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:13 smithi057 ceph-mon[40588]: pgmap v1762: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:03:14.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 14:03:13 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:14:03:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T14:03:14.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:14 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:03:14.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:14 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:03:14.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:14 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:03:15.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:15 smithi057 ceph-mon[40588]: pgmap v1763: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:03:15.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:15 smithi057 ceph-mon[29805]: pgmap v1763: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:03:15.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:15 smithi107 ceph-mon[33178]: pgmap v1763: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:03:16.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:16 smithi057 ceph-mon[40588]: pgmap v1764: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:03:16.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:16 smithi057 ceph-mon[29805]: pgmap v1764: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:03:16.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:16 smithi107 ceph-mon[33178]: pgmap v1764: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:03:17.662 INFO:tasks.workunit.client.0.smithi057.stderr:++ seq 1 10 2024-06-11T14:03:17.663 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in `seq 1 10` 2024-06-11T14:03:17.663 INFO:tasks.workunit.client.0.smithi057.stderr:+ rados -p 3308aae0-a66b-4c2f-a482-8b20f197ecc4 put obj1 /etc/passwd 2024-06-11T14:03:17.703 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in `seq 1 10` 2024-06-11T14:03:17.704 INFO:tasks.workunit.client.0.smithi057.stderr:+ rados -p 3308aae0-a66b-4c2f-a482-8b20f197ecc4 put obj2 /etc/passwd 2024-06-11T14:03:17.741 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in `seq 1 10` 2024-06-11T14:03:17.741 INFO:tasks.workunit.client.0.smithi057.stderr:+ rados -p 3308aae0-a66b-4c2f-a482-8b20f197ecc4 put obj3 /etc/passwd 2024-06-11T14:03:17.777 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in `seq 1 10` 2024-06-11T14:03:17.777 INFO:tasks.workunit.client.0.smithi057.stderr:+ rados -p 3308aae0-a66b-4c2f-a482-8b20f197ecc4 put obj4 /etc/passwd 2024-06-11T14:03:17.812 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in `seq 1 10` 2024-06-11T14:03:17.812 INFO:tasks.workunit.client.0.smithi057.stderr:+ rados -p 3308aae0-a66b-4c2f-a482-8b20f197ecc4 put obj5 /etc/passwd 2024-06-11T14:03:17.847 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in `seq 1 10` 2024-06-11T14:03:17.847 INFO:tasks.workunit.client.0.smithi057.stderr:+ rados -p 3308aae0-a66b-4c2f-a482-8b20f197ecc4 put obj6 /etc/passwd 2024-06-11T14:03:17.881 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in `seq 1 10` 2024-06-11T14:03:17.881 INFO:tasks.workunit.client.0.smithi057.stderr:+ rados -p 3308aae0-a66b-4c2f-a482-8b20f197ecc4 put obj7 /etc/passwd 2024-06-11T14:03:17.915 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in `seq 1 10` 2024-06-11T14:03:17.915 INFO:tasks.workunit.client.0.smithi057.stderr:+ rados -p 3308aae0-a66b-4c2f-a482-8b20f197ecc4 put obj8 /etc/passwd 2024-06-11T14:03:17.949 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in `seq 1 10` 2024-06-11T14:03:17.949 INFO:tasks.workunit.client.0.smithi057.stderr:+ rados -p 3308aae0-a66b-4c2f-a482-8b20f197ecc4 put obj9 /etc/passwd 2024-06-11T14:03:17.983 INFO:tasks.workunit.client.0.smithi057.stderr:+ for f in `seq 1 10` 2024-06-11T14:03:17.983 INFO:tasks.workunit.client.0.smithi057.stderr:+ rados -p 3308aae0-a66b-4c2f-a482-8b20f197ecc4 put obj10 /etc/passwd 2024-06-11T14:03:18.018 INFO:tasks.workunit.client.0.smithi057.stderr:+ sleep 30 2024-06-11T14:03:19.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:18 smithi057 ceph-mon[29805]: pgmap v1765: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:03:19.286 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:19 smithi057 ceph-mon[40588]: pgmap v1765: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:03:19.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:18 smithi107 ceph-mon[33178]: pgmap v1765: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:03:21.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:21 smithi057 ceph-mon[29805]: pgmap v1766: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:03:21.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:21 smithi057 ceph-mon[40588]: pgmap v1766: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:03:21.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:21 smithi107 ceph-mon[33178]: pgmap v1766: 188 pgs: 188 active+clean; 748 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:03:23.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:23 smithi057 ceph-mon[29805]: pgmap v1767: 188 pgs: 188 active+clean; 866 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-06-11T14:03:23.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:23 smithi057 ceph-mon[29805]: pool '3308aae0-a66b-4c2f-a482-8b20f197ecc4' is full (reached quota's max_objects: 10) 2024-06-11T14:03:23.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:23 smithi057 ceph-mon[29805]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-06-11T14:03:23.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:23 smithi057 ceph-mon[29805]: osdmap e798: 8 total, 8 up, 8 in 2024-06-11T14:03:23.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:23 smithi057 ceph-mon[40588]: pgmap v1767: 188 pgs: 188 active+clean; 866 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-06-11T14:03:23.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:23 smithi057 ceph-mon[40588]: pool '3308aae0-a66b-4c2f-a482-8b20f197ecc4' is full (reached quota's max_objects: 10) 2024-06-11T14:03:23.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:23 smithi057 ceph-mon[40588]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-06-11T14:03:23.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:23 smithi057 ceph-mon[40588]: osdmap e798: 8 total, 8 up, 8 in 2024-06-11T14:03:23.299 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 14:03:22 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T14:03:23.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:23 smithi107 ceph-mon[33178]: pgmap v1767: 188 pgs: 188 active+clean; 866 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-06-11T14:03:23.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:23 smithi107 ceph-mon[33178]: pool '3308aae0-a66b-4c2f-a482-8b20f197ecc4' is full (reached quota's max_objects: 10) 2024-06-11T14:03:23.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:23 smithi107 ceph-mon[33178]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-06-11T14:03:23.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:23 smithi107 ceph-mon[33178]: osdmap e798: 8 total, 8 up, 8 in 2024-06-11T14:03:24.012 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 14:03:23 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:14:03:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T14:03:24.289 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:24 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:03:24.289 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:24 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T14:03:24.289 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:24 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:03:24.289 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:24 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:03:24.289 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:24 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T14:03:24.289 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:24 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:03:24.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:24 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:03:24.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:24 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T14:03:24.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:24 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:03:25.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:25 smithi057 ceph-mon[29805]: pgmap v1769: 188 pgs: 188 active+clean; 866 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-06-11T14:03:25.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:25 smithi057 ceph-mon[40588]: pgmap v1769: 188 pgs: 188 active+clean; 866 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-06-11T14:03:25.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:25 smithi107 ceph-mon[33178]: pgmap v1769: 188 pgs: 188 active+clean; 866 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-06-11T14:03:27.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:27 smithi057 ceph-mon[29805]: pgmap v1770: 188 pgs: 188 active+clean; 866 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-06-11T14:03:27.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:27 smithi057 ceph-mon[40588]: pgmap v1770: 188 pgs: 188 active+clean; 866 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-06-11T14:03:27.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:27 smithi107 ceph-mon[33178]: pgmap v1770: 188 pgs: 188 active+clean; 866 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-06-11T14:03:29.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:29 smithi057 ceph-mon[29805]: pgmap v1771: 188 pgs: 188 active+clean; 866 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-06-11T14:03:29.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:29 smithi057 ceph-mon[40588]: pgmap v1771: 188 pgs: 188 active+clean; 866 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-06-11T14:03:29.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:29 smithi107 ceph-mon[33178]: pgmap v1771: 188 pgs: 188 active+clean; 866 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-06-11T14:03:31.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:31 smithi057 ceph-mon[29805]: pgmap v1772: 188 pgs: 188 active+clean; 866 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-06-11T14:03:31.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:31 smithi057 ceph-mon[40588]: pgmap v1772: 188 pgs: 188 active+clean; 866 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-06-11T14:03:31.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:31 smithi107 ceph-mon[33178]: pgmap v1772: 188 pgs: 188 active+clean; 866 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-06-11T14:03:33.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:33 smithi107 ceph-mon[33178]: pgmap v1773: 188 pgs: 188 active+clean; 866 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T14:03:33.300 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 14:03:32 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T14:03:33.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:33 smithi057 ceph-mon[29805]: pgmap v1773: 188 pgs: 188 active+clean; 866 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T14:03:33.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:33 smithi057 ceph-mon[40588]: pgmap v1773: 188 pgs: 188 active+clean; 866 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T14:03:34.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 14:03:33 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:14:03:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T14:03:34.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:34 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:03:34.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:34 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:03:34.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:34 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:03:35.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:35 smithi057 ceph-mon[29805]: pgmap v1774: 188 pgs: 188 active+clean; 866 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 904 B/s rd, 0 op/s 2024-06-11T14:03:35.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:35 smithi057 ceph-mon[40588]: pgmap v1774: 188 pgs: 188 active+clean; 866 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 904 B/s rd, 0 op/s 2024-06-11T14:03:35.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:35 smithi107 ceph-mon[33178]: pgmap v1774: 188 pgs: 188 active+clean; 866 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 904 B/s rd, 0 op/s 2024-06-11T14:03:37.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:37 smithi057 ceph-mon[40588]: pgmap v1775: 188 pgs: 188 active+clean; 866 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:03:37.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:37 smithi057 ceph-mon[29805]: pgmap v1775: 188 pgs: 188 active+clean; 866 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:03:37.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:37 smithi107 ceph-mon[33178]: pgmap v1775: 188 pgs: 188 active+clean; 866 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:03:39.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:39 smithi057 ceph-mon[29805]: pgmap v1776: 188 pgs: 188 active+clean; 866 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:03:39.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:39 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:03:39.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:39 smithi057 ceph-mon[40588]: pgmap v1776: 188 pgs: 188 active+clean; 866 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:03:39.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:39 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:03:39.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:39 smithi107 ceph-mon[33178]: pgmap v1776: 188 pgs: 188 active+clean; 866 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:03:39.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:39 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:03:41.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:41 smithi057 ceph-mon[29805]: pgmap v1777: 188 pgs: 188 active+clean; 866 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:03:41.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:41 smithi057 ceph-mon[40588]: pgmap v1777: 188 pgs: 188 active+clean; 866 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:03:41.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:41 smithi107 ceph-mon[33178]: pgmap v1777: 188 pgs: 188 active+clean; 866 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:03:43.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:43 smithi107 ceph-mon[33178]: pgmap v1778: 188 pgs: 188 active+clean; 866 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:03:43.300 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 14:03:42 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T14:03:43.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:43 smithi057 ceph-mon[29805]: pgmap v1778: 188 pgs: 188 active+clean; 866 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:03:43.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:43 smithi057 ceph-mon[40588]: pgmap v1778: 188 pgs: 188 active+clean; 866 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:03:44.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 14:03:43 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:14:03:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T14:03:44.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:44 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:03:44.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:44 smithi057 ceph-mon[29805]: pgmap v1779: 188 pgs: 188 active+clean; 866 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:03:44.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:44 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:03:44.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:44 smithi057 ceph-mon[40588]: pgmap v1779: 188 pgs: 188 active+clean; 866 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:03:44.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:44 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:03:44.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:44 smithi107 ceph-mon[33178]: pgmap v1779: 188 pgs: 188 active+clean; 866 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:03:47.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:47 smithi057 ceph-mon[29805]: pgmap v1780: 188 pgs: 188 active+clean; 866 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:03:47.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:47 smithi057 ceph-mon[40588]: pgmap v1780: 188 pgs: 188 active+clean; 866 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:03:47.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:47 smithi107 ceph-mon[33178]: pgmap v1780: 188 pgs: 188 active+clean; 866 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:03:48.021 INFO:tasks.workunit.client.0.smithi057.stderr:+ rados -p 5bd44950-56a4-47a9-a376-eee5f24dbed6 put threemore /etc/passwd 2024-06-11T14:03:48.059 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph osd pool set-quota 5bd44950-56a4-47a9-a376-eee5f24dbed6 max_bytes 0 2024-06-11T14:03:48.151 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:48.151+0000 7f7b4f5f7640 1 -- 172.21.15.57:0/2462558305 >> v1:172.21.15.57:6789/0 conn(0x7f7b48106720 legacy=0x7f7b48106b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:03:48.151 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:48.151+0000 7f7b4f5f7640 1 -- 172.21.15.57:0/2462558305 shutdown_connections 2024-06-11T14:03:48.151 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:48.151+0000 7f7b4f5f7640 1 -- 172.21.15.57:0/2462558305 >> 172.21.15.57:0/2462558305 conn(0x7f7b48077400 msgr2=0x7f7b480fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T14:03:48.151 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:48.151+0000 7f7b4f5f7640 1 -- 172.21.15.57:0/2462558305 shutdown_connections 2024-06-11T14:03:48.151 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:48.151+0000 7f7b4f5f7640 1 -- 172.21.15.57:0/2462558305 wait complete. 2024-06-11T14:03:48.154 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:48.152+0000 7f7b4f5f7640 1 Processor -- start 2024-06-11T14:03:48.154 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:48.152+0000 7f7b4f5f7640 1 -- start start 2024-06-11T14:03:48.154 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:48.152+0000 7f7b4f5f7640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f7b481a5670 con 0x7f7b4810d190 2024-06-11T14:03:48.154 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:48.152+0000 7f7b4f5f7640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f7b481a6870 con 0x7f7b48106720 2024-06-11T14:03:48.154 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:48.152+0000 7f7b4f5f7640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f7b481a7ab0 con 0x7f7b481095c0 2024-06-11T14:03:48.154 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:48.152+0000 7f7b4cb92640 1 --1- >> v1:172.21.15.57:6790/0 conn(0x7f7b481095c0 0x7f7b481a05e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6790/0 says I am v1:172.21.15.57:56284/0 (socket says 172.21.15.57:56284) 2024-06-11T14:03:48.154 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:48.152+0000 7f7b4db94640 1 --1- 172.21.15.57:0/1922224196 >> v1:172.21.15.57:6789/0 conn(0x7f7b4810d190 0x7f7b481a3d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:36358/0 (socket says 172.21.15.57:36358) 2024-06-11T14:03:48.154 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:48.152+0000 7f7b4cb92640 1 -- 172.21.15.57:0/1922224196 learned_addr learned my addr 172.21.15.57:0/1922224196 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T14:03:48.154 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:48.153+0000 7f7b3e7fc640 1 -- 172.21.15.57:0/1922224196 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1701117689 0 0) 0x7f7b481a5670 con 0x7f7b4810d190 2024-06-11T14:03:48.155 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:48.153+0000 7f7b3e7fc640 1 -- 172.21.15.57:0/1922224196 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b34003160 con 0x7f7b4810d190 2024-06-11T14:03:48.155 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:48.153+0000 7f7b3e7fc640 1 -- 172.21.15.57:0/1922224196 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3213741245 0 0) 0x7f7b481a7ab0 con 0x7f7b481095c0 2024-06-11T14:03:48.155 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:48.153+0000 7f7b3e7fc640 1 -- 172.21.15.57:0/1922224196 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b481a5670 con 0x7f7b481095c0 2024-06-11T14:03:48.155 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:48.153+0000 7f7b3e7fc640 1 -- 172.21.15.57:0/1922224196 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1856928496 0 0) 0x7f7b481a6870 con 0x7f7b48106720 2024-06-11T14:03:48.155 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:48.153+0000 7f7b3e7fc640 1 -- 172.21.15.57:0/1922224196 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b481a7ab0 con 0x7f7b48106720 2024-06-11T14:03:48.155 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:48.153+0000 7f7b3e7fc640 1 -- 172.21.15.57:0/1922224196 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2737463063 0 0) 0x7f7b34003160 con 0x7f7b4810d190 2024-06-11T14:03:48.155 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:48.153+0000 7f7b3e7fc640 1 -- 172.21.15.57:0/1922224196 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7b481a6870 con 0x7f7b4810d190 2024-06-11T14:03:48.155 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:48.153+0000 7f7b3e7fc640 1 -- 172.21.15.57:0/1922224196 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4054491789 0 0) 0x7f7b481a5670 con 0x7f7b481095c0 2024-06-11T14:03:48.155 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:48.153+0000 7f7b3e7fc640 1 -- 172.21.15.57:0/1922224196 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7b34003160 con 0x7f7b481095c0 2024-06-11T14:03:48.155 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:48.153+0000 7f7b3e7fc640 1 -- 172.21.15.57:0/1922224196 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f7b40001d50 con 0x7f7b4810d190 2024-06-11T14:03:48.155 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:48.153+0000 7f7b3e7fc640 1 -- 172.21.15.57:0/1922224196 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3065699182 0 0) 0x7f7b481a7ab0 con 0x7f7b48106720 2024-06-11T14:03:48.155 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:48.153+0000 7f7b3e7fc640 1 -- 172.21.15.57:0/1922224196 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7b481a5670 con 0x7f7b48106720 2024-06-11T14:03:48.155 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:48.153+0000 7f7b3e7fc640 1 -- 172.21.15.57:0/1922224196 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f7b38001a30 con 0x7f7b481095c0 2024-06-11T14:03:48.155 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:48.153+0000 7f7b3e7fc640 1 -- 172.21.15.57:0/1922224196 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f7b440012d0 con 0x7f7b48106720 2024-06-11T14:03:48.155 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:48.153+0000 7f7b3e7fc640 1 -- 172.21.15.57:0/1922224196 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2510938888 0 0) 0x7f7b481a6870 con 0x7f7b4810d190 2024-06-11T14:03:48.156 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:48.153+0000 7f7b3e7fc640 1 -- 172.21.15.57:0/1922224196 >> v1:172.21.15.57:6790/0 conn(0x7f7b481095c0 legacy=0x7f7b481a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:03:48.156 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:48.153+0000 7f7b3e7fc640 1 -- 172.21.15.57:0/1922224196 >> v1:172.21.15.107:6789/0 conn(0x7f7b48106720 legacy=0x7f7b4810c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:03:48.156 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:48.153+0000 7f7b3e7fc640 1 -- 172.21.15.57:0/1922224196 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b481a8cf0 con 0x7f7b4810d190 2024-06-11T14:03:48.156 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:48.153+0000 7f7b4f5f7640 1 -- 172.21.15.57:0/1922224196 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f7b481a58a0 con 0x7f7b4810d190 2024-06-11T14:03:48.156 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:48.153+0000 7f7b4f5f7640 1 -- 172.21.15.57:0/1922224196 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f7b481a5d70 con 0x7f7b4810d190 2024-06-11T14:03:48.156 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:48.153+0000 7f7b3e7fc640 1 -- 172.21.15.57:0/1922224196 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f7b40003e00 con 0x7f7b4810d190 2024-06-11T14:03:48.156 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:48.153+0000 7f7b3e7fc640 1 -- 172.21.15.57:0/1922224196 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f7b40005ee0 con 0x7f7b4810d190 2024-06-11T14:03:48.156 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:48.154+0000 7f7b4f5f7640 1 -- 172.21.15.57:0/1922224196 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b2c005080 con 0x7f7b4810d190 2024-06-11T14:03:48.157 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:48.155+0000 7f7b3e7fc640 1 -- 172.21.15.57:0/1922224196 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 22) ==== 92138+0+0 (unknown 311317106 0 0) 0x7f7b40003970 con 0x7f7b4810d190 2024-06-11T14:03:48.157 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:48.155+0000 7f7b3e7fc640 1 -- 172.21.15.57:0/1922224196 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(798..798 src has 265..798) ==== 8219+0+0 (unknown 1145517828 0 0) 0x7f7b40089ee0 con 0x7f7b4810d190 2024-06-11T14:03:48.157 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:48.156+0000 7f7b3e7fc640 1 -- 172.21.15.57:0/1922224196 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=799}) -- 0x7f7b481a6870 con 0x7f7b4810d190 2024-06-11T14:03:48.159 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:48.160+0000 7f7b3e7fc640 1 -- 172.21.15.57:0/1922224196 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f7b40056e30 con 0x7f7b4810d190 2024-06-11T14:03:48.349 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:48.349+0000 7f7b4f5f7640 1 -- 172.21.15.57:0/1922224196 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "0"} v 0) -- 0x7f7b2c005370 con 0x7f7b4810d190 2024-06-11T14:03:49.017 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:49.018+0000 7f7b3e7fc640 1 -- 172.21.15.57:0/1922224196 <== mon.0 v1:172.21.15.57:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 5bd44950-56a4-47a9-a376-eee5f24dbed6 v799) ==== 217+0+0 (unknown 3248743407 0 0) 0x7f7b4005aae0 con 0x7f7b4810d190 2024-06-11T14:03:49.022 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:49.023+0000 7f7b3e7fc640 1 -- 172.21.15.57:0/1922224196 <== mon.0 v1:172.21.15.57:6789/0 11 ==== osd_map(799..799 src has 265..799) ==== 628+0+0 (unknown 3672656658 0 0) 0x7f7b40052d10 con 0x7f7b4810d190 2024-06-11T14:03:49.022 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:49.023+0000 7f7b3e7fc640 1 -- 172.21.15.57:0/1922224196 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=800}) -- 0x7f7b481a5670 con 0x7f7b4810d190 2024-06-11T14:03:49.141 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:49.141+0000 7f7b4f5f7640 1 -- 172.21.15.57:0/1922224196 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "0"} v 0) -- 0x7f7b2c000c40 con 0x7f7b4810d190 2024-06-11T14:03:49.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:49 smithi057 ceph-mon[29805]: pgmap v1781: 188 pgs: 188 active+clean; 866 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:03:49.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:49 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1922224196' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "0"}]: dispatch 2024-06-11T14:03:49.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:49 smithi057 ceph-mon[40588]: pgmap v1781: 188 pgs: 188 active+clean; 866 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:03:49.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:49 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1922224196' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "0"}]: dispatch 2024-06-11T14:03:49.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:49 smithi107 ceph-mon[33178]: pgmap v1781: 188 pgs: 188 active+clean; 866 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:03:49.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:49 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1922224196' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "0"}]: dispatch 2024-06-11T14:03:50.054 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.054+0000 7f7b3e7fc640 1 -- 172.21.15.57:0/1922224196 <== mon.0 v1:172.21.15.57:6789/0 12 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 5bd44950-56a4-47a9-a376-eee5f24dbed6 v800) ==== 217+0+0 (unknown 332938710 0 0) 0x7f7b4005f3a0 con 0x7f7b4810d190 2024-06-11T14:03:50.054 INFO:tasks.workunit.client.0.smithi057.stderr:set-quota max_bytes = 0 for pool 5bd44950-56a4-47a9-a376-eee5f24dbed6 2024-06-11T14:03:50.054 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.054+0000 7f7b3e7fc640 1 -- 172.21.15.57:0/1922224196 <== mon.0 v1:172.21.15.57:6789/0 13 ==== osd_map(800..800 src has 265..800) ==== 628+0+0 (unknown 923694595 0 0) 0x7f7b40052770 con 0x7f7b4810d190 2024-06-11T14:03:50.054 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.054+0000 7f7b3e7fc640 1 -- 172.21.15.57:0/1922224196 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=801}) -- 0x7f7b34003160 con 0x7f7b4810d190 2024-06-11T14:03:50.057 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.057+0000 7f7b4f5f7640 1 -- 172.21.15.57:0/1922224196 >> v1:172.21.15.57:6800/2077416415 conn(0x7f7b34071000 legacy=0x7f7b340734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:03:50.057 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.057+0000 7f7b4f5f7640 1 -- 172.21.15.57:0/1922224196 >> v1:172.21.15.57:6789/0 conn(0x7f7b4810d190 legacy=0x7f7b481a3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:03:50.057 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.057+0000 7f7b4f5f7640 1 -- 172.21.15.57:0/1922224196 shutdown_connections 2024-06-11T14:03:50.057 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.057+0000 7f7b4f5f7640 1 -- 172.21.15.57:0/1922224196 >> 172.21.15.57:0/1922224196 conn(0x7f7b48077400 msgr2=0x7f7b4810d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T14:03:50.057 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.057+0000 7f7b4f5f7640 1 -- 172.21.15.57:0/1922224196 shutdown_connections 2024-06-11T14:03:50.057 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.057+0000 7f7b4f5f7640 1 -- 172.21.15.57:0/1922224196 wait complete. 2024-06-11T14:03:50.072 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph osd pool set-quota 5bd44950-56a4-47a9-a376-eee5f24dbed6 max_objects 0 2024-06-11T14:03:50.163 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.163+0000 7fd001a72640 1 -- 172.21.15.57:0/2987241861 >> v1:172.21.15.57:6790/0 conn(0x7fcffc10cbd0 legacy=0x7fcffc10efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:03:50.163 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.163+0000 7fd001a72640 1 -- 172.21.15.57:0/2987241861 shutdown_connections 2024-06-11T14:03:50.163 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.163+0000 7fd001a72640 1 -- 172.21.15.57:0/2987241861 >> 172.21.15.57:0/2987241861 conn(0x7fcffc077400 msgr2=0x7fcffc0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T14:03:50.164 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.163+0000 7fd001a72640 1 -- 172.21.15.57:0/2987241861 shutdown_connections 2024-06-11T14:03:50.164 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.163+0000 7fd001a72640 1 -- 172.21.15.57:0/2987241861 wait complete. 2024-06-11T14:03:50.164 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.164+0000 7fd001a72640 1 Processor -- start 2024-06-11T14:03:50.166 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.164+0000 7fd001a72640 1 -- start start 2024-06-11T14:03:50.166 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.164+0000 7fd001a72640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fcffc1a89c0 con 0x7fcffc1106e0 2024-06-11T14:03:50.166 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.164+0000 7fd001a72640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fcffc1a9c00 con 0x7fcffc10cbd0 2024-06-11T14:03:50.166 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.164+0000 7fd001a72640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fcffc1aae40 con 0x7fcffc1005f0 2024-06-11T14:03:50.166 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.164+0000 7fcffb7fe640 1 --1- >> v1:172.21.15.57:6790/0 conn(0x7fcffc1005f0 0x7fcffc10c320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6790/0 says I am v1:172.21.15.57:48710/0 (socket says 172.21.15.57:48710) 2024-06-11T14:03:50.166 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.164+0000 7fcffb7fe640 1 -- 172.21.15.57:0/2790316657 learned_addr learned my addr 172.21.15.57:0/2790316657 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T14:03:50.166 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.165+0000 7fcff8ff9640 1 -- 172.21.15.57:0/2790316657 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1436849448 0 0) 0x7fcffc1aae40 con 0x7fcffc1005f0 2024-06-11T14:03:50.167 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.165+0000 7fcff8ff9640 1 -- 172.21.15.57:0/2790316657 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcfe4003160 con 0x7fcffc1005f0 2024-06-11T14:03:50.167 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.165+0000 7fcff8ff9640 1 -- 172.21.15.57:0/2790316657 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2604120248 0 0) 0x7fcffc1a89c0 con 0x7fcffc1106e0 2024-06-11T14:03:50.167 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.165+0000 7fcff8ff9640 1 -- 172.21.15.57:0/2790316657 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcffc1aae40 con 0x7fcffc1106e0 2024-06-11T14:03:50.167 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.165+0000 7fcff8ff9640 1 -- 172.21.15.57:0/2790316657 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3862219652 0 0) 0x7fcffc1a9c00 con 0x7fcffc10cbd0 2024-06-11T14:03:50.167 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.165+0000 7fcff8ff9640 1 -- 172.21.15.57:0/2790316657 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcffc1a89c0 con 0x7fcffc10cbd0 2024-06-11T14:03:50.167 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.165+0000 7fcff8ff9640 1 -- 172.21.15.57:0/2790316657 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 746490419 0 0) 0x7fcfe4003160 con 0x7fcffc1005f0 2024-06-11T14:03:50.167 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.165+0000 7fcff8ff9640 1 -- 172.21.15.57:0/2790316657 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fcffc1a9c00 con 0x7fcffc1005f0 2024-06-11T14:03:50.167 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.165+0000 7fcff8ff9640 1 -- 172.21.15.57:0/2790316657 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fcfec001930 con 0x7fcffc1005f0 2024-06-11T14:03:50.167 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.165+0000 7fcff8ff9640 1 -- 172.21.15.57:0/2790316657 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4069528183 0 0) 0x7fcffc1aae40 con 0x7fcffc1106e0 2024-06-11T14:03:50.167 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.165+0000 7fcff8ff9640 1 -- 172.21.15.57:0/2790316657 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fcfe4003160 con 0x7fcffc1106e0 2024-06-11T14:03:50.167 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.165+0000 7fcff8ff9640 1 -- 172.21.15.57:0/2790316657 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fcff4001db0 con 0x7fcffc1106e0 2024-06-11T14:03:50.167 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.165+0000 7fcff8ff9640 1 -- 172.21.15.57:0/2790316657 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3616990233 0 0) 0x7fcffc1a89c0 con 0x7fcffc10cbd0 2024-06-11T14:03:50.167 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.165+0000 7fcff8ff9640 1 -- 172.21.15.57:0/2790316657 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fcffc1aae40 con 0x7fcffc10cbd0 2024-06-11T14:03:50.168 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.165+0000 7fcff8ff9640 1 -- 172.21.15.57:0/2790316657 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fcff0001980 con 0x7fcffc10cbd0 2024-06-11T14:03:50.168 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.165+0000 7fcff8ff9640 1 -- 172.21.15.57:0/2790316657 <== mon.2 v1:172.21.15.57:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3265381762 0 0) 0x7fcffc1a9c00 con 0x7fcffc1005f0 2024-06-11T14:03:50.168 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.165+0000 7fcff8ff9640 1 -- 172.21.15.57:0/2790316657 >> v1:172.21.15.107:6789/0 conn(0x7fcffc10cbd0 legacy=0x7fcffc1a3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:03:50.168 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.165+0000 7fcff8ff9640 1 -- 172.21.15.57:0/2790316657 >> v1:172.21.15.57:6789/0 conn(0x7fcffc1106e0 legacy=0x7fcffc1a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:03:50.168 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.165+0000 7fcff8ff9640 1 -- 172.21.15.57:0/2790316657 --> v1:172.21.15.57:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcffc1ac080 con 0x7fcffc1005f0 2024-06-11T14:03:50.168 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.165+0000 7fd001a72640 1 -- 172.21.15.57:0/2790316657 --> v1:172.21.15.57:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fcffc1a8bf0 con 0x7fcffc1005f0 2024-06-11T14:03:50.168 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.165+0000 7fd001a72640 1 -- 172.21.15.57:0/2790316657 --> v1:172.21.15.57:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fcffc1a90c0 con 0x7fcffc1005f0 2024-06-11T14:03:50.168 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.165+0000 7fcff8ff9640 1 -- 172.21.15.57:0/2790316657 <== mon.2 v1:172.21.15.57:6790/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7fcfec001d00 con 0x7fcffc1005f0 2024-06-11T14:03:50.168 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.165+0000 7fcff8ff9640 1 -- 172.21.15.57:0/2790316657 <== mon.2 v1:172.21.15.57:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7fcfec005c90 con 0x7fcffc1005f0 2024-06-11T14:03:50.168 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.166+0000 7fd001a72640 1 -- 172.21.15.57:0/2790316657 --> v1:172.21.15.57:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfe8005080 con 0x7fcffc1005f0 2024-06-11T14:03:50.169 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.167+0000 7fcff8ff9640 1 -- 172.21.15.57:0/2790316657 <== mon.2 v1:172.21.15.57:6790/0 7 ==== mgrmap(e 22) ==== 92138+0+0 (unknown 311317106 0 0) 0x7fcfec0038c0 con 0x7fcffc1005f0 2024-06-11T14:03:50.169 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.167+0000 7fcff8ff9640 1 -- 172.21.15.57:0/2790316657 <== mon.2 v1:172.21.15.57:6790/0 8 ==== osd_map(800..800 src has 265..800) ==== 8219+0+0 (unknown 1081740662 0 0) 0x7fcfec089cb0 con 0x7fcffc1005f0 2024-06-11T14:03:50.169 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.168+0000 7fcff8ff9640 1 -- 172.21.15.57:0/2790316657 --> v1:172.21.15.57:6790/0 -- mon_subscribe({osdmap=801}) -- 0x7fcffc1a9c00 con 0x7fcffc1005f0 2024-06-11T14:03:50.171 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.172+0000 7fcff8ff9640 1 -- 172.21.15.57:0/2790316657 <== mon.2 v1:172.21.15.57:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7fcfec056c20 con 0x7fcffc1005f0 2024-06-11T14:03:50.356 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:50.357+0000 7fd001a72640 1 -- 172.21.15.57:0/2790316657 --> v1:172.21.15.57:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "0"} v 0) -- 0x7fcfe8005370 con 0x7fcffc1005f0 2024-06-11T14:03:50.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:50 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1922224196' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "0"}]': finished 2024-06-11T14:03:50.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:50 smithi057 ceph-mon[29805]: osdmap e799: 8 total, 8 up, 8 in 2024-06-11T14:03:50.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:50 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1922224196' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "0"}]: dispatch 2024-06-11T14:03:50.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:50 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1922224196' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "0"}]': finished 2024-06-11T14:03:50.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:50 smithi057 ceph-mon[40588]: osdmap e799: 8 total, 8 up, 8 in 2024-06-11T14:03:50.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:50 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1922224196' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "0"}]: dispatch 2024-06-11T14:03:50.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:50 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1922224196' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "0"}]': finished 2024-06-11T14:03:50.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:50 smithi107 ceph-mon[33178]: osdmap e799: 8 total, 8 up, 8 in 2024-06-11T14:03:50.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:50 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1922224196' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "0"}]: dispatch 2024-06-11T14:03:51.065 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:51.065+0000 7fcff8ff9640 1 -- 172.21.15.57:0/2790316657 <== mon.2 v1:172.21.15.57:6790/0 10 ==== osd_map(801..801 src has 265..801) ==== 628+0+0 (unknown 4230502670 0 0) 0x7fcfec005c90 con 0x7fcffc1005f0 2024-06-11T14:03:51.065 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:51.065+0000 7fcff8ff9640 1 -- 172.21.15.57:0/2790316657 --> v1:172.21.15.57:6790/0 -- mon_subscribe({osdmap=802}) -- 0x7fcffc1a89c0 con 0x7fcffc1005f0 2024-06-11T14:03:51.069 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:51.070+0000 7fcff8ff9640 1 -- 172.21.15.57:0/2790316657 <== mon.2 v1:172.21.15.57:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 5bd44950-56a4-47a9-a376-eee5f24dbed6 v801) ==== 221+0+0 (unknown 3767320212 0 0) 0x7fcfec05a8d0 con 0x7fcffc1005f0 2024-06-11T14:03:51.187 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:51.187+0000 7fd001a72640 1 -- 172.21.15.57:0/2790316657 --> v1:172.21.15.57:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "0"} v 0) -- 0x7fcfe8000be0 con 0x7fcffc1005f0 2024-06-11T14:03:51.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:51 smithi057 ceph-mon[29805]: pgmap v1783: 188 pgs: 188 active+clean; 866 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T14:03:51.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:51 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/1922224196' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "0"}]': finished 2024-06-11T14:03:51.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:51 smithi057 ceph-mon[29805]: osdmap e800: 8 total, 8 up, 8 in 2024-06-11T14:03:51.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:51 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2790316657' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "0"}]: dispatch 2024-06-11T14:03:51.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:51 smithi057 ceph-mon[29805]: from='client.58988 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "0"}]: dispatch 2024-06-11T14:03:51.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:51 smithi057 ceph-mon[40588]: pgmap v1783: 188 pgs: 188 active+clean; 866 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T14:03:51.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:51 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/1922224196' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "0"}]': finished 2024-06-11T14:03:51.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:51 smithi057 ceph-mon[40588]: osdmap e800: 8 total, 8 up, 8 in 2024-06-11T14:03:51.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:51 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2790316657' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "0"}]: dispatch 2024-06-11T14:03:51.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:51 smithi057 ceph-mon[40588]: from='client.58988 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "0"}]: dispatch 2024-06-11T14:03:51.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:51 smithi107 ceph-mon[33178]: pgmap v1783: 188 pgs: 188 active+clean; 866 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T14:03:51.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:51 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/1922224196' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_bytes", "val": "0"}]': finished 2024-06-11T14:03:51.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:51 smithi107 ceph-mon[33178]: osdmap e800: 8 total, 8 up, 8 in 2024-06-11T14:03:51.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:51 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2790316657' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "0"}]: dispatch 2024-06-11T14:03:51.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:51 smithi107 ceph-mon[33178]: from='client.58988 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "0"}]: dispatch 2024-06-11T14:03:52.086 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:52.086+0000 7fcff8ff9640 1 -- 172.21.15.57:0/2790316657 <== mon.2 v1:172.21.15.57:6790/0 12 ==== osd_map(802..802 src has 265..802) ==== 628+0+0 (unknown 1127097384 0 0) 0x7fcfec0878d0 con 0x7fcffc1005f0 2024-06-11T14:03:52.086 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:52.087+0000 7fcff8ff9640 1 -- 172.21.15.57:0/2790316657 --> v1:172.21.15.57:6790/0 -- mon_subscribe({osdmap=803}) -- 0x7fcfe407ca50 con 0x7fcffc1005f0 2024-06-11T14:03:52.093 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:52.093+0000 7fcff8ff9640 1 -- 172.21.15.57:0/2790316657 <== mon.2 v1:172.21.15.57:6790/0 13 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 5bd44950-56a4-47a9-a376-eee5f24dbed6 v802) ==== 221+0+0 (unknown 1704049222 0 0) 0x7fcfec05f190 con 0x7fcffc1005f0 2024-06-11T14:03:52.093 INFO:tasks.workunit.client.0.smithi057.stderr:set-quota max_objects = 0 for pool 5bd44950-56a4-47a9-a376-eee5f24dbed6 2024-06-11T14:03:52.095 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:52.096+0000 7fd001a72640 1 -- 172.21.15.57:0/2790316657 >> v1:172.21.15.57:6800/2077416415 conn(0x7fcfe406efb0 legacy=0x7fcfe4071470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:03:52.095 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:52.096+0000 7fd001a72640 1 -- 172.21.15.57:0/2790316657 >> v1:172.21.15.57:6790/0 conn(0x7fcffc1005f0 legacy=0x7fcffc10c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:03:52.096 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:52.096+0000 7fd001a72640 1 -- 172.21.15.57:0/2790316657 shutdown_connections 2024-06-11T14:03:52.096 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:52.096+0000 7fd001a72640 1 -- 172.21.15.57:0/2790316657 >> 172.21.15.57:0/2790316657 conn(0x7fcffc077400 msgr2=0x7fcffc10dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T14:03:52.096 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:52.096+0000 7fd001a72640 1 -- 172.21.15.57:0/2790316657 shutdown_connections 2024-06-11T14:03:52.096 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:03:52.096+0000 7fd001a72640 1 -- 172.21.15.57:0/2790316657 wait complete. 2024-06-11T14:03:52.107 INFO:tasks.workunit.client.0.smithi057.stderr:+ sleep 30 2024-06-11T14:03:52.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:52 smithi057 ceph-mon[29805]: from='client.58988 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "0"}]': finished 2024-06-11T14:03:52.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:52 smithi057 ceph-mon[29805]: osdmap e801: 8 total, 8 up, 8 in 2024-06-11T14:03:52.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:52 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/2790316657' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "0"}]: dispatch 2024-06-11T14:03:52.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:52 smithi057 ceph-mon[29805]: from='client.58988 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "0"}]: dispatch 2024-06-11T14:03:52.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:52 smithi057 ceph-mon[40588]: from='client.58988 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "0"}]': finished 2024-06-11T14:03:52.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:52 smithi057 ceph-mon[40588]: osdmap e801: 8 total, 8 up, 8 in 2024-06-11T14:03:52.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:52 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/2790316657' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "0"}]: dispatch 2024-06-11T14:03:52.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:52 smithi057 ceph-mon[40588]: from='client.58988 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "0"}]: dispatch 2024-06-11T14:03:52.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:52 smithi107 ceph-mon[33178]: from='client.58988 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "0"}]': finished 2024-06-11T14:03:52.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:52 smithi107 ceph-mon[33178]: osdmap e801: 8 total, 8 up, 8 in 2024-06-11T14:03:52.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:52 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/2790316657' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "0"}]: dispatch 2024-06-11T14:03:52.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:52 smithi107 ceph-mon[33178]: from='client.58988 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "0"}]: dispatch 2024-06-11T14:03:53.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:53 smithi107 ceph-mon[33178]: pgmap v1786: 188 pgs: 188 active+clean; 878 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-06-11T14:03:53.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:53 smithi107 ceph-mon[33178]: from='client.58988 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "0"}]': finished 2024-06-11T14:03:53.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:53 smithi107 ceph-mon[33178]: osdmap e802: 8 total, 8 up, 8 in 2024-06-11T14:03:53.300 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 14:03:52 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T14:03:53.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:53 smithi057 ceph-mon[29805]: pgmap v1786: 188 pgs: 188 active+clean; 878 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-06-11T14:03:53.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:53 smithi057 ceph-mon[29805]: from='client.58988 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "0"}]': finished 2024-06-11T14:03:53.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:53 smithi057 ceph-mon[29805]: osdmap e802: 8 total, 8 up, 8 in 2024-06-11T14:03:53.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:53 smithi057 ceph-mon[40588]: pgmap v1786: 188 pgs: 188 active+clean; 878 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-06-11T14:03:53.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:53 smithi057 ceph-mon[40588]: from='client.58988 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "field": "max_objects", "val": "0"}]': finished 2024-06-11T14:03:53.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:53 smithi057 ceph-mon[40588]: osdmap e802: 8 total, 8 up, 8 in 2024-06-11T14:03:54.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 14:03:53 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:14:03:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T14:03:54.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:54 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:03:54.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:54 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:03:54.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:54 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:03:54.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:54 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:03:54.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:54 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:03:54.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:54 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:03:55.438 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:55 smithi057 ceph-mon[29805]: pgmap v1788: 188 pgs: 188 active+clean; 878 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.4 KiB/s wr, 1 op/s 2024-06-11T14:03:55.438 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:55 smithi057 ceph-mon[40588]: pgmap v1788: 188 pgs: 188 active+clean; 878 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.4 KiB/s wr, 1 op/s 2024-06-11T14:03:55.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:55 smithi107 ceph-mon[33178]: pgmap v1788: 188 pgs: 188 active+clean; 878 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.4 KiB/s wr, 1 op/s 2024-06-11T14:03:56.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:56 smithi057 ceph-mon[29805]: pgmap v1789: 188 pgs: 188 active+clean; 878 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 2.0 KiB/s wr, 0 op/s 2024-06-11T14:03:56.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:56 smithi057 ceph-mon[40588]: pgmap v1789: 188 pgs: 188 active+clean; 878 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 2.0 KiB/s wr, 0 op/s 2024-06-11T14:03:56.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:56 smithi107 ceph-mon[33178]: pgmap v1789: 188 pgs: 188 active+clean; 878 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 2.0 KiB/s wr, 0 op/s 2024-06-11T14:03:59.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:03:59 smithi057 ceph-mon[29805]: pgmap v1790: 188 pgs: 188 active+clean; 878 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-06-11T14:03:59.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:03:59 smithi057 ceph-mon[40588]: pgmap v1790: 188 pgs: 188 active+clean; 878 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-06-11T14:03:59.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:03:59 smithi107 ceph-mon[33178]: pgmap v1790: 188 pgs: 188 active+clean; 878 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-06-11T14:04:00.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:04:00 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T14:04:00.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:04:00 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T14:04:00.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:04:00 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T14:04:00.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:04:00 smithi057 ceph-mon[29805]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T14:04:00.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:04:00 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T14:04:00.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:04:00 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T14:04:00.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:04:00 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T14:04:00.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:04:00 smithi057 ceph-mon[40588]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T14:04:00.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:04:00 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-11T14:04:00.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:04:00 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-11T14:04:00.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:04:00 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-11T14:04:00.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:04:00 smithi107 ceph-mon[33178]: from='mgr.14604 ' entity='mgr.y' 2024-06-11T14:04:01.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:04:01 smithi057 ceph-mon[29805]: pgmap v1791: 188 pgs: 188 active+clean; 878 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 576 B/s rd, 0 op/s 2024-06-11T14:04:01.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:04:01 smithi057 ceph-mon[40588]: pgmap v1791: 188 pgs: 188 active+clean; 878 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 576 B/s rd, 0 op/s 2024-06-11T14:04:01.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:04:01 smithi107 ceph-mon[33178]: pgmap v1791: 188 pgs: 188 active+clean; 878 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 576 B/s rd, 0 op/s 2024-06-11T14:04:03.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:04:03 smithi057 ceph-mon[29805]: pgmap v1792: 188 pgs: 188 active+clean; 878 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T14:04:03.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:04:03 smithi057 ceph-mon[40588]: pgmap v1792: 188 pgs: 188 active+clean; 878 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T14:04:03.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:04:03 smithi107 ceph-mon[33178]: pgmap v1792: 188 pgs: 188 active+clean; 878 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T14:04:03.300 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 14:04:03 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T14:04:04.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 14:04:03 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:14:04:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T14:04:04.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:04:04 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:04:04.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:04:04 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:04:04.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:04:04 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:04:05.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:04:05 smithi107 ceph-mon[33178]: pgmap v1793: 188 pgs: 188 active+clean; 878 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 862 B/s rd, 0 op/s 2024-06-11T14:04:05.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:04:05 smithi057 ceph-mon[40588]: pgmap v1793: 188 pgs: 188 active+clean; 878 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 862 B/s rd, 0 op/s 2024-06-11T14:04:05.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:04:05 smithi057 ceph-mon[29805]: pgmap v1793: 188 pgs: 188 active+clean; 878 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 862 B/s rd, 0 op/s 2024-06-11T14:04:07.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:04:07 smithi057 ceph-mon[29805]: pgmap v1794: 188 pgs: 188 active+clean; 878 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:04:07.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:04:07 smithi057 ceph-mon[40588]: pgmap v1794: 188 pgs: 188 active+clean; 878 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:04:07.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:04:07 smithi107 ceph-mon[33178]: pgmap v1794: 188 pgs: 188 active+clean; 878 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:04:09.036 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:04:08 smithi057 ceph-mon[40588]: pgmap v1795: 188 pgs: 188 active+clean; 878 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:04:09.037 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:04:08 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:04:09.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:04:08 smithi057 ceph-mon[29805]: pgmap v1795: 188 pgs: 188 active+clean; 878 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:04:09.037 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:04:08 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:04:09.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:04:08 smithi107 ceph-mon[33178]: pgmap v1795: 188 pgs: 188 active+clean; 878 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:04:09.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:04:08 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:04:11.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:04:11 smithi057 ceph-mon[29805]: pgmap v1796: 188 pgs: 188 active+clean; 878 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:04:11.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:04:11 smithi057 ceph-mon[40588]: pgmap v1796: 188 pgs: 188 active+clean; 878 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:04:11.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:04:11 smithi107 ceph-mon[33178]: pgmap v1796: 188 pgs: 188 active+clean; 878 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:04:13.286 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:04:13 smithi057 ceph-mon[29805]: pgmap v1797: 188 pgs: 188 active+clean; 878 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:04:13.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:04:13 smithi057 ceph-mon[40588]: pgmap v1797: 188 pgs: 188 active+clean; 878 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:04:13.299 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:04:13 smithi107 ceph-mon[33178]: pgmap v1797: 188 pgs: 188 active+clean; 878 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:04:13.300 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 14:04:13 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T14:04:14.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 14:04:13 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:14:04:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T14:04:14.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:04:14 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:04:14.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:04:14 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:04:14.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:04:14 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:04:15.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:04:15 smithi107 ceph-mon[33178]: pgmap v1798: 188 pgs: 188 active+clean; 878 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:04:15.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:04:15 smithi057 ceph-mon[29805]: pgmap v1798: 188 pgs: 188 active+clean; 878 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:04:15.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:04:15 smithi057 ceph-mon[40588]: pgmap v1798: 188 pgs: 188 active+clean; 878 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:04:17.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:04:17 smithi057 ceph-mon[29805]: pgmap v1799: 188 pgs: 188 active+clean; 878 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:04:17.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:04:17 smithi057 ceph-mon[40588]: pgmap v1799: 188 pgs: 188 active+clean; 878 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:04:17.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:04:17 smithi107 ceph-mon[33178]: pgmap v1799: 188 pgs: 188 active+clean; 878 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-06-11T14:04:19.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:04:19 smithi057 ceph-mon[29805]: pgmap v1800: 188 pgs: 188 active+clean; 878 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:04:19.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:04:19 smithi057 ceph-mon[40588]: pgmap v1800: 188 pgs: 188 active+clean; 878 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:04:19.549 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:04:19 smithi107 ceph-mon[33178]: pgmap v1800: 188 pgs: 188 active+clean; 878 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:04:21.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:04:21 smithi057 ceph-mon[29805]: pgmap v1801: 188 pgs: 188 active+clean; 878 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:04:21.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:04:21 smithi057 ceph-mon[40588]: pgmap v1801: 188 pgs: 188 active+clean; 878 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:04:21.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:04:21 smithi107 ceph-mon[33178]: pgmap v1801: 188 pgs: 188 active+clean; 878 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-11T14:04:22.108 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph osd pool delete 5bd44950-56a4-47a9-a376-eee5f24dbed6 5bd44950-56a4-47a9-a376-eee5f24dbed6 --yes-i-really-really-mean-it 2024-06-11T14:04:22.203 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:22.203+0000 7f320a6df640 1 -- 172.21.15.57:0/1000116384 >> v1:172.21.15.57:6790/0 conn(0x7f320410e2f0 legacy=0x7f3204110780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:04:22.203 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:22.203+0000 7f320a6df640 1 -- 172.21.15.57:0/1000116384 shutdown_connections 2024-06-11T14:04:22.203 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:22.203+0000 7f320a6df640 1 -- 172.21.15.57:0/1000116384 >> 172.21.15.57:0/1000116384 conn(0x7f3204077400 msgr2=0x7f32040fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T14:04:22.204 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:22.203+0000 7f320a6df640 1 -- 172.21.15.57:0/1000116384 shutdown_connections 2024-06-11T14:04:22.204 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:22.203+0000 7f320a6df640 1 -- 172.21.15.57:0/1000116384 wait complete. 2024-06-11T14:04:22.204 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:22.204+0000 7f320a6df640 1 Processor -- start 2024-06-11T14:04:22.204 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:22.204+0000 7f320a6df640 1 -- start start 2024-06-11T14:04:22.210 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:22.204+0000 7f320a6df640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f32041a8ac0 con 0x7f320410a7e0 2024-06-11T14:04:22.210 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:22.204+0000 7f320a6df640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f32041a9d00 con 0x7f320410e2f0 2024-06-11T14:04:22.210 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:22.204+0000 7f320a6df640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f32041aaf40 con 0x7f32041029e0 2024-06-11T14:04:22.210 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:22.204+0000 7f3203fff640 1 --1- >> v1:172.21.15.57:6790/0 conn(0x7f32041029e0 0x7f3204109fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6790/0 says I am v1:172.21.15.57:47054/0 (socket says 172.21.15.57:47054) 2024-06-11T14:04:22.210 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:22.204+0000 7f3203fff640 1 -- 172.21.15.57:0/543727906 learned_addr learned my addr 172.21.15.57:0/543727906 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T14:04:22.210 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:22.205+0000 7f32017fa640 1 -- 172.21.15.57:0/543727906 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 903322048 0 0) 0x7f32041aaf40 con 0x7f32041029e0 2024-06-11T14:04:22.211 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:22.205+0000 7f32017fa640 1 -- 172.21.15.57:0/543727906 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f31ec003160 con 0x7f32041029e0 2024-06-11T14:04:22.211 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:22.205+0000 7f32017fa640 1 -- 172.21.15.57:0/543727906 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2103901071 0 0) 0x7f32041a8ac0 con 0x7f320410a7e0 2024-06-11T14:04:22.211 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:22.205+0000 7f32017fa640 1 -- 172.21.15.57:0/543727906 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32041aaf40 con 0x7f320410a7e0 2024-06-11T14:04:22.211 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:22.205+0000 7f32017fa640 1 -- 172.21.15.57:0/543727906 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4222489536 0 0) 0x7f32041a9d00 con 0x7f320410e2f0 2024-06-11T14:04:22.211 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:22.205+0000 7f32017fa640 1 -- 172.21.15.57:0/543727906 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32041a8ac0 con 0x7f320410e2f0 2024-06-11T14:04:22.211 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:22.205+0000 7f32017fa640 1 -- 172.21.15.57:0/543727906 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3976609339 0 0) 0x7f32041aaf40 con 0x7f320410a7e0 2024-06-11T14:04:22.211 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:22.205+0000 7f32017fa640 1 -- 172.21.15.57:0/543727906 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f32041a9d00 con 0x7f320410a7e0 2024-06-11T14:04:22.211 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:22.205+0000 7f32017fa640 1 -- 172.21.15.57:0/543727906 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 985365422 0 0) 0x7f31ec003160 con 0x7f32041029e0 2024-06-11T14:04:22.211 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:22.205+0000 7f32017fa640 1 -- 172.21.15.57:0/543727906 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f32041aaf40 con 0x7f32041029e0 2024-06-11T14:04:22.212 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:22.205+0000 7f32017fa640 1 -- 172.21.15.57:0/543727906 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f31f80018b0 con 0x7f320410a7e0 2024-06-11T14:04:22.212 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:22.205+0000 7f32017fa640 1 -- 172.21.15.57:0/543727906 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f31f4001890 con 0x7f32041029e0 2024-06-11T14:04:22.212 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:22.205+0000 7f32017fa640 1 -- 172.21.15.57:0/543727906 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2325773657 0 0) 0x7f32041a8ac0 con 0x7f320410e2f0 2024-06-11T14:04:22.212 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:22.205+0000 7f32017fa640 1 -- 172.21.15.57:0/543727906 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f31ec003160 con 0x7f320410e2f0 2024-06-11T14:04:22.212 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:22.205+0000 7f32017fa640 1 -- 172.21.15.57:0/543727906 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f31fc001870 con 0x7f320410e2f0 2024-06-11T14:04:22.212 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:22.205+0000 7f32017fa640 1 -- 172.21.15.57:0/543727906 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 706942329 0 0) 0x7f32041a9d00 con 0x7f320410a7e0 2024-06-11T14:04:22.212 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:22.205+0000 7f32017fa640 1 -- 172.21.15.57:0/543727906 >> v1:172.21.15.57:6790/0 conn(0x7f32041029e0 legacy=0x7f3204109fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:04:22.212 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:22.205+0000 7f32017fa640 1 -- 172.21.15.57:0/543727906 >> v1:172.21.15.107:6789/0 conn(0x7f320410e2f0 legacy=0x7f32041a7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:04:22.212 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:22.205+0000 7f32017fa640 1 -- 172.21.15.57:0/543727906 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32041ac180 con 0x7f320410a7e0 2024-06-11T14:04:22.212 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:22.205+0000 7f320a6df640 1 -- 172.21.15.57:0/543727906 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f32041a8cf0 con 0x7f320410a7e0 2024-06-11T14:04:22.212 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:22.205+0000 7f320a6df640 1 -- 172.21.15.57:0/543727906 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f32041a91c0 con 0x7f320410a7e0 2024-06-11T14:04:22.212 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:22.206+0000 7f32017fa640 1 -- 172.21.15.57:0/543727906 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f31f8003aa0 con 0x7f320410a7e0 2024-06-11T14:04:22.213 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:22.206+0000 7f32017fa640 1 -- 172.21.15.57:0/543727906 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f31f80049f0 con 0x7f320410a7e0 2024-06-11T14:04:22.213 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:22.206+0000 7f320a6df640 1 -- 172.21.15.57:0/543727906 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31f0005080 con 0x7f320410a7e0 2024-06-11T14:04:22.213 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:22.207+0000 7f32017fa640 1 -- 172.21.15.57:0/543727906 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 22) ==== 92138+0+0 (unknown 311317106 0 0) 0x7f31f8003610 con 0x7f320410a7e0 2024-06-11T14:04:22.213 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:22.208+0000 7f32017fa640 1 -- 172.21.15.57:0/543727906 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(802..802 src has 265..802) ==== 8219+0+0 (unknown 981412255 0 0) 0x7f31f80889e0 con 0x7f320410a7e0 2024-06-11T14:04:22.213 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:22.208+0000 7f32017fa640 1 -- 172.21.15.57:0/543727906 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=803}) -- 0x7f32041a9d00 con 0x7f320410a7e0 2024-06-11T14:04:22.213 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:22.212+0000 7f32017fa640 1 -- 172.21.15.57:0/543727906 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f31f8055930 con 0x7f320410a7e0 2024-06-11T14:04:22.396 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:22.396+0000 7f320a6df640 1 -- 172.21.15.57:0/543727906 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "osd pool delete", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "pool2": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "yes_i_really_really_mean_it": true} v 0) -- 0x7f31f0005370 con 0x7f320410a7e0 2024-06-11T14:04:23.087 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.087+0000 7f32017fa640 1 -- 172.21.15.57:0/543727906 <== mon.0 v1:172.21.15.57:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "pool2": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "yes_i_really_really_mean_it": true}]=0 pool '5bd44950-56a4-47a9-a376-eee5f24dbed6' removed v803) ==== 248+0+0 (unknown 2079118103 0 0) 0x7f31f80595e0 con 0x7f320410a7e0 2024-06-11T14:04:23.096 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.096+0000 7f32017fa640 1 -- 172.21.15.57:0/543727906 <== mon.0 v1:172.21.15.57:6789/0 11 ==== osd_map(803..803 src has 265..803) ==== 296+0+0 (unknown 1982040835 0 0) 0x7f31f80865e0 con 0x7f320410a7e0 2024-06-11T14:04:23.096 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.096+0000 7f32017fa640 1 -- 172.21.15.57:0/543727906 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=804}) -- 0x7f32041aaf40 con 0x7f320410a7e0 2024-06-11T14:04:23.212 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.212+0000 7f320a6df640 1 -- 172.21.15.57:0/543727906 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "osd pool delete", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "pool2": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "yes_i_really_really_mean_it": true} v 0) -- 0x7f31f0005da0 con 0x7f320410a7e0 2024-06-11T14:04:23.212 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.212+0000 7f32017fa640 1 -- 172.21.15.57:0/543727906 <== mon.0 v1:172.21.15.57:6789/0 12 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "pool2": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "yes_i_really_really_mean_it": true}]=0 pool '5bd44950-56a4-47a9-a376-eee5f24dbed6' does not exist v803) ==== 255+0+0 (unknown 977896301 0 0) 0x7f31f805dea0 con 0x7f320410a7e0 2024-06-11T14:04:23.212 INFO:tasks.workunit.client.0.smithi057.stderr:pool '5bd44950-56a4-47a9-a376-eee5f24dbed6' does not exist 2024-06-11T14:04:23.214 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.214+0000 7f320a6df640 1 -- 172.21.15.57:0/543727906 >> v1:172.21.15.57:6800/2077416415 conn(0x7f31ec06eea0 legacy=0x7f31ec071360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:04:23.214 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.214+0000 7f320a6df640 1 -- 172.21.15.57:0/543727906 >> v1:172.21.15.57:6789/0 conn(0x7f320410a7e0 legacy=0x7f32041a3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:04:23.214 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.214+0000 7f320a6df640 1 -- 172.21.15.57:0/543727906 shutdown_connections 2024-06-11T14:04:23.214 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.214+0000 7f320a6df640 1 -- 172.21.15.57:0/543727906 >> 172.21.15.57:0/543727906 conn(0x7f3204077400 msgr2=0x7f3204102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T14:04:23.214 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.214+0000 7f320a6df640 1 -- 172.21.15.57:0/543727906 shutdown_connections 2024-06-11T14:04:23.214 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.214+0000 7f320a6df640 1 -- 172.21.15.57:0/543727906 wait complete. 2024-06-11T14:04:23.223 INFO:tasks.workunit.client.0.smithi057.stderr:+ ceph osd pool delete 3308aae0-a66b-4c2f-a482-8b20f197ecc4 3308aae0-a66b-4c2f-a482-8b20f197ecc4 --yes-i-really-really-mean-it 2024-06-11T14:04:23.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:04:23 smithi107 ceph-mon[33178]: pgmap v1802: 188 pgs: 188 active+clean; 878 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:04:23.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:04:23 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/543727906' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "pool2": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-11T14:04:23.300 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 14:04:23 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug there is no tcmu-runner data available 2024-06-11T14:04:23.310 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.310+0000 7f025a95d640 1 -- 172.21.15.57:0/2655198824 >> v1:172.21.15.57:6790/0 conn(0x7f0254102a50 legacy=0x7f0254102e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:04:23.310 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.310+0000 7f025a95d640 1 -- 172.21.15.57:0/2655198824 shutdown_connections 2024-06-11T14:04:23.310 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.310+0000 7f025a95d640 1 -- 172.21.15.57:0/2655198824 >> 172.21.15.57:0/2655198824 conn(0x7f02540fc230 msgr2=0x7f02540fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T14:04:23.310 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.310+0000 7f025a95d640 1 -- 172.21.15.57:0/2655198824 shutdown_connections 2024-06-11T14:04:23.310 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.310+0000 7f025a95d640 1 -- 172.21.15.57:0/2655198824 wait complete. 2024-06-11T14:04:23.313 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.311+0000 7f025a95d640 1 Processor -- start 2024-06-11T14:04:23.313 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.311+0000 7f025a95d640 1 -- start start 2024-06-11T14:04:23.313 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.311+0000 7f025a95d640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f02541a6870 con 0x7f025410a850 2024-06-11T14:04:23.313 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.311+0000 7f025a95d640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f02541a7ab0 con 0x7f025410e360 2024-06-11T14:04:23.313 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.311+0000 7f025a95d640 1 -- --> v1:172.21.15.57:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f02541a8cf0 con 0x7f0254102a50 2024-06-11T14:04:23.313 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.311+0000 7f0253fff640 1 --1- >> v1:172.21.15.57:6790/0 conn(0x7f0254102a50 0x7f025410a060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6790/0 says I am v1:172.21.15.57:47066/0 (socket says 172.21.15.57:47066) 2024-06-11T14:04:23.313 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.311+0000 7f0253fff640 1 -- 172.21.15.57:0/739464744 learned_addr learned my addr 172.21.15.57:0/739464744 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-06-11T14:04:23.313 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.311+0000 7f02537fe640 1 --1- 172.21.15.57:0/739464744 >> v1:172.21.15.57:6789/0 conn(0x7f025410a850 0x7f02541a1700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:43806/0 (socket says 172.21.15.57:43806) 2024-06-11T14:04:23.313 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.312+0000 7f02517fa640 1 -- 172.21.15.57:0/739464744 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3424116901 0 0) 0x7f02541a6870 con 0x7f025410a850 2024-06-11T14:04:23.313 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.312+0000 7f02517fa640 1 -- 172.21.15.57:0/739464744 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f023c003160 con 0x7f025410a850 2024-06-11T14:04:23.313 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.312+0000 7f02517fa640 1 -- 172.21.15.57:0/739464744 <== mon.2 v1:172.21.15.57:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3843387275 0 0) 0x7f02541a8cf0 con 0x7f0254102a50 2024-06-11T14:04:23.314 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.312+0000 7f02517fa640 1 -- 172.21.15.57:0/739464744 --> v1:172.21.15.57:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f02541a6870 con 0x7f0254102a50 2024-06-11T14:04:23.314 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.312+0000 7f02517fa640 1 -- 172.21.15.57:0/739464744 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4121176054 0 0) 0x7f02541a7ab0 con 0x7f025410e360 2024-06-11T14:04:23.314 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.312+0000 7f02517fa640 1 -- 172.21.15.57:0/739464744 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f02541a8cf0 con 0x7f025410e360 2024-06-11T14:04:23.314 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.312+0000 7f02517fa640 1 -- 172.21.15.57:0/739464744 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3067214940 0 0) 0x7f023c003160 con 0x7f025410a850 2024-06-11T14:04:23.314 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.312+0000 7f02517fa640 1 -- 172.21.15.57:0/739464744 --> v1:172.21.15.57:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f02541a7ab0 con 0x7f025410a850 2024-06-11T14:04:23.314 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.312+0000 7f02517fa640 1 -- 172.21.15.57:0/739464744 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f02480018b0 con 0x7f025410a850 2024-06-11T14:04:23.314 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.312+0000 7f02517fa640 1 -- 172.21.15.57:0/739464744 <== mon.2 v1:172.21.15.57:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 122514159 0 0) 0x7f02541a6870 con 0x7f0254102a50 2024-06-11T14:04:23.314 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.312+0000 7f02517fa640 1 -- 172.21.15.57:0/739464744 --> v1:172.21.15.57:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f023c003160 con 0x7f0254102a50 2024-06-11T14:04:23.314 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.312+0000 7f02517fa640 1 -- 172.21.15.57:0/739464744 <== mon.2 v1:172.21.15.57:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f0244004290 con 0x7f0254102a50 2024-06-11T14:04:23.314 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.312+0000 7f02517fa640 1 -- 172.21.15.57:0/739464744 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3574632263 0 0) 0x7f02541a8cf0 con 0x7f025410e360 2024-06-11T14:04:23.314 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.312+0000 7f02517fa640 1 -- 172.21.15.57:0/739464744 --> v1:172.21.15.107:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f02541a6870 con 0x7f025410e360 2024-06-11T14:04:23.314 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.312+0000 7f02517fa640 1 -- 172.21.15.57:0/739464744 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f024c0013f0 con 0x7f025410e360 2024-06-11T14:04:23.315 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.312+0000 7f02517fa640 1 -- 172.21.15.57:0/739464744 <== mon.0 v1:172.21.15.57:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1140668950 0 0) 0x7f02541a7ab0 con 0x7f025410a850 2024-06-11T14:04:23.315 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.312+0000 7f02517fa640 1 -- 172.21.15.57:0/739464744 >> v1:172.21.15.57:6790/0 conn(0x7f0254102a50 legacy=0x7f025410a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:04:23.315 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.312+0000 7f02517fa640 1 -- 172.21.15.57:0/739464744 >> v1:172.21.15.107:6789/0 conn(0x7f025410e360 legacy=0x7f02541a4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:04:23.315 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.312+0000 7f02517fa640 1 -- 172.21.15.57:0/739464744 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02541a9f30 con 0x7f025410a850 2024-06-11T14:04:23.315 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.312+0000 7f025a95d640 1 -- 172.21.15.57:0/739464744 --> v1:172.21.15.57:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f02541a8f20 con 0x7f025410a850 2024-06-11T14:04:23.315 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.312+0000 7f025a95d640 1 -- 172.21.15.57:0/739464744 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f02541a93f0 con 0x7f025410a850 2024-06-11T14:04:23.315 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.312+0000 7f02517fa640 1 -- 172.21.15.57:0/739464744 <== mon.0 v1:172.21.15.57:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 1216276225 0 0) 0x7f0248001c60 con 0x7f025410a850 2024-06-11T14:04:23.315 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.312+0000 7f02517fa640 1 -- 172.21.15.57:0/739464744 <== mon.0 v1:172.21.15.57:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 4032357520 0 0) 0x7f0248005cf0 con 0x7f025410a850 2024-06-11T14:04:23.315 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.313+0000 7f025a95d640 1 -- 172.21.15.57:0/739464744 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0240005080 con 0x7f025410a850 2024-06-11T14:04:23.316 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.314+0000 7f02517fa640 1 -- 172.21.15.57:0/739464744 <== mon.0 v1:172.21.15.57:6789/0 7 ==== mgrmap(e 22) ==== 92138+0+0 (unknown 311317106 0 0) 0x7f02480038c0 con 0x7f025410a850 2024-06-11T14:04:23.316 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.314+0000 7f02517fa640 1 -- 172.21.15.57:0/739464744 <== mon.0 v1:172.21.15.57:6789/0 8 ==== osd_map(803..803 src has 265..803) ==== 7831+0+0 (unknown 106155722 0 0) 0x7f02480899e0 con 0x7f025410a850 2024-06-11T14:04:23.316 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.314+0000 7f02517fa640 1 -- 172.21.15.57:0/739464744 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=804}) -- 0x7f02541a7ab0 con 0x7f025410a850 2024-06-11T14:04:23.318 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.318+0000 7f02517fa640 1 -- 172.21.15.57:0/739464744 <== mon.0 v1:172.21.15.57:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (unknown 1092875540 0 3075472397) 0x7f0248056c00 con 0x7f025410a850 2024-06-11T14:04:23.509 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:23.509+0000 7f025a95d640 1 -- 172.21.15.57:0/739464744 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "osd pool delete", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "pool2": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0240005370 con 0x7f025410a850 2024-06-11T14:04:23.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:04:23 smithi057 ceph-mon[29805]: pgmap v1802: 188 pgs: 188 active+clean; 878 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:04:23.536 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:04:23 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/543727906' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "pool2": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-11T14:04:23.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:04:23 smithi057 ceph-mon[40588]: pgmap v1802: 188 pgs: 188 active+clean; 878 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-11T14:04:23.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:04:23 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/543727906' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "pool2": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-11T14:04:24.036 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 14:04:23 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: ::ffff:172.21.15.107 - - [11/Jun/2024:14:04:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-06-11T14:04:24.130 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:24.131+0000 7f02517fa640 1 -- 172.21.15.57:0/739464744 <== mon.0 v1:172.21.15.57:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "pool2": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "yes_i_really_really_mean_it": true}]=0 pool '3308aae0-a66b-4c2f-a482-8b20f197ecc4' removed v804) ==== 248+0+0 (unknown 1732433081 0 0) 0x7f024805a8b0 con 0x7f025410a850 2024-06-11T14:04:24.135 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:24.136+0000 7f02517fa640 1 -- 172.21.15.57:0/739464744 <== mon.0 v1:172.21.15.57:6789/0 11 ==== osd_map(804..804 src has 265..804) ==== 296+0+0 (unknown 1496628263 0 0) 0x7f02480521c0 con 0x7f025410a850 2024-06-11T14:04:24.135 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:24.136+0000 7f02517fa640 1 -- 172.21.15.57:0/739464744 --> v1:172.21.15.57:6789/0 -- mon_subscribe({osdmap=805}) -- 0x7f02541a8cf0 con 0x7f025410a850 2024-06-11T14:04:24.255 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:24.256+0000 7f025a95d640 1 -- 172.21.15.57:0/739464744 --> v1:172.21.15.57:6789/0 -- mon_command({"prefix": "osd pool delete", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "pool2": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0240005da0 con 0x7f025410a850 2024-06-11T14:04:24.256 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:24.256+0000 7f02517fa640 1 -- 172.21.15.57:0/739464744 <== mon.0 v1:172.21.15.57:6789/0 12 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "pool2": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "yes_i_really_really_mean_it": true}]=0 pool '3308aae0-a66b-4c2f-a482-8b20f197ecc4' does not exist v804) ==== 255+0+0 (unknown 209759057 0 0) 0x7f024805f170 con 0x7f025410a850 2024-06-11T14:04:24.256 INFO:tasks.workunit.client.0.smithi057.stderr:pool '3308aae0-a66b-4c2f-a482-8b20f197ecc4' does not exist 2024-06-11T14:04:24.258 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:24.258+0000 7f025a95d640 1 -- 172.21.15.57:0/739464744 >> v1:172.21.15.57:6800/2077416415 conn(0x7f023c06ee00 legacy=0x7f023c0712c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:04:24.258 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:24.258+0000 7f025a95d640 1 -- 172.21.15.57:0/739464744 >> v1:172.21.15.57:6789/0 conn(0x7f025410a850 legacy=0x7f02541a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-11T14:04:24.258 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:24.258+0000 7f025a95d640 1 -- 172.21.15.57:0/739464744 shutdown_connections 2024-06-11T14:04:24.258 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:24.258+0000 7f025a95d640 1 -- 172.21.15.57:0/739464744 >> 172.21.15.57:0/739464744 conn(0x7f02540fc230 msgr2=0x7f02540fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-11T14:04:24.258 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:24.258+0000 7f025a95d640 1 -- 172.21.15.57:0/739464744 shutdown_connections 2024-06-11T14:04:24.258 INFO:tasks.workunit.client.0.smithi057.stderr:2024-06-11T14:04:24.258+0000 7f025a95d640 1 -- 172.21.15.57:0/739464744 wait complete. 2024-06-11T14:04:24.267 INFO:tasks.workunit.client.0.smithi057.stdout:OK 2024-06-11T14:04:24.268 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo OK 2024-06-11T14:04:24.269 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-11T14:04:24.269 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-11T14:04:24.337 INFO:tasks.workunit:Stopping ['rados/test.sh', 'rados/test_pool_quota.sh'] on client.0... 2024-06-11T14:04:24.337 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-06-11T14:04:24.400 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:04:24 smithi057 ceph-mon[29805]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:04:24.400 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:04:24 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/543727906' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "pool2": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "yes_i_really_really_mean_it": true}]': finished 2024-06-11T14:04:24.400 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:04:24 smithi057 ceph-mon[29805]: osdmap e803: 8 total, 8 up, 8 in 2024-06-11T14:04:24.400 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:04:24 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/543727906' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "pool2": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-11T14:04:24.400 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:04:24 smithi057 ceph-mon[29805]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:04:24.401 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:04:24 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/739464744' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "pool2": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-11T14:04:24.401 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:04:24 smithi057 ceph-mon[29805]: pgmap v1804: 176 pgs: 176 active+clean; 701 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T14:04:24.401 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:04:24 smithi057 ceph-mon[40588]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:04:24.401 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:04:24 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/543727906' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "pool2": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "yes_i_really_really_mean_it": true}]': finished 2024-06-11T14:04:24.401 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:04:24 smithi057 ceph-mon[40588]: osdmap e803: 8 total, 8 up, 8 in 2024-06-11T14:04:24.401 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:04:24 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/543727906' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "pool2": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-11T14:04:24.401 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:04:24 smithi057 ceph-mon[40588]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:04:24.401 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:04:24 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/739464744' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "pool2": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-11T14:04:24.401 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:04:24 smithi057 ceph-mon[40588]: pgmap v1804: 176 pgs: 176 active+clean; 701 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T14:04:24.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:04:24 smithi107 ceph-mon[33178]: from='client.14556 v1:172.21.15.107:0/1205442497' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-06-11T14:04:24.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:04:24 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/543727906' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "pool2": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "yes_i_really_really_mean_it": true}]': finished 2024-06-11T14:04:24.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:04:24 smithi107 ceph-mon[33178]: osdmap e803: 8 total, 8 up, 8 in 2024-06-11T14:04:24.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:04:24 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/543727906' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "pool2": "5bd44950-56a4-47a9-a376-eee5f24dbed6", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-11T14:04:24.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:04:24 smithi107 ceph-mon[33178]: from='mgr.14604 v1:172.21.15.57:0/773764867' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-11T14:04:24.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:04:24 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/739464744' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "pool2": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-11T14:04:24.550 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:04:24 smithi107 ceph-mon[33178]: pgmap v1804: 176 pgs: 176 active+clean; 701 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-06-11T14:04:24.679 DEBUG:teuthology.parallel:result is None 2024-06-11T14:04:24.679 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-06-11T14:04:24.744 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-06-11T14:04:24.745 DEBUG:teuthology.orchestra.run.smithi057:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-06-11T14:04:24.804 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-06-11T14:04:24.804 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-06-11T14:04:24.814 INFO:tasks.cephadm:Teardown begin 2024-06-11T14:04:24.814 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-11T14:04:24.870 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-11T14:04:24.900 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-06-11T14:04:24.900 DEBUG:teuthology.orchestra.run.smithi057:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-11T14:04:24.926 DEBUG:teuthology.orchestra.run.smithi107:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-11T14:04:24.957 INFO:tasks.cephadm:Stopping all daemons... 2024-06-11T14:04:24.957 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-06-11T14:04:24.957 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@mon.a 2024-06-11T14:04:25.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:04:25 smithi057 ceph-mon[40588]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-06-11T14:04:25.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:04:25 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/739464744' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "pool2": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "yes_i_really_really_mean_it": true}]': finished 2024-06-11T14:04:25.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:04:25 smithi057 ceph-mon[40588]: osdmap e804: 8 total, 8 up, 8 in 2024-06-11T14:04:25.287 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:04:25 smithi057 ceph-mon[40588]: from='client.? v1:172.21.15.57:0/739464744' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "pool2": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-11T14:04:25.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:04:25 smithi057 systemd[1]: Stopping Ceph mon.a for 98a6635e-27f5-11ef-bc9b-c7b262605968... 2024-06-11T14:04:25.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:04:25 smithi057 ceph-mon[29805]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-06-11T14:04:25.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:04:25 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/739464744' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "pool2": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "yes_i_really_really_mean_it": true}]': finished 2024-06-11T14:04:25.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:04:25 smithi057 ceph-mon[29805]: osdmap e804: 8 total, 8 up, 8 in 2024-06-11T14:04:25.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:04:25 smithi057 ceph-mon[29805]: from='client.? v1:172.21.15.57:0/739464744' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "pool2": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-11T14:04:25.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:04:25 smithi107 ceph-mon[33178]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-06-11T14:04:25.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:04:25 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/739464744' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "pool2": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "yes_i_really_really_mean_it": true}]': finished 2024-06-11T14:04:25.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:04:25 smithi107 ceph-mon[33178]: osdmap e804: 8 total, 8 up, 8 in 2024-06-11T14:04:25.300 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:04:25 smithi107 ceph-mon[33178]: from='client.? v1:172.21.15.57:0/739464744' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "pool2": "3308aae0-a66b-4c2f-a482-8b20f197ecc4", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-11T14:04:25.565 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:04:25 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mon-a[29801]: 2024-06-11T14:04:25.412+0000 7fad72f88640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-11T14:04:25.565 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:04:25 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mon-a[29801]: 2024-06-11T14:04:25.412+0000 7fad72f88640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-06-11T14:04:25.565 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:04:25 smithi057 podman[155450]: 2024-06-11 14:04:25.48469983 +0000 UTC m=+0.320428515 container died 8ca4f226c1e690507897c41903d6ff1518ff2f5d0599ee12072d3940837d4a21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mon-a, RELEASE=squid-3f51ea3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.35.3, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528) 2024-06-11T14:04:26.278 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:04:25 smithi057 podman[155450]: 2024-06-11 14:04:25.896465278 +0000 UTC m=+0.732193946 container cleanup 8ca4f226c1e690507897c41903d6ff1518ff2f5d0599ee12072d3940837d4a21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.35.3, RELEASE=squid-3f51ea3, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-06-11T14:04:26.278 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:04:25 smithi057 bash[155450]: ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mon-a 2024-06-11T14:04:26.278 INFO:journalctl@ceph.mon.a.smithi057.stdout:Jun 11 14:04:26 smithi057 podman[155466]: 2024-06-11 14:04:26.123110565 +0000 UTC m=+0.632879255 container remove 8ca4f226c1e690507897c41903d6ff1518ff2f5d0599ee12072d3940837d4a21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mon-a, CEPH_POINT_RELEASE=, RELEASE=squid-3f51ea3, GIT_BRANCH=HEAD, io.buildah.version=1.35.3, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-11T14:04:26.292 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@mon.a.service' 2024-06-11T14:04:26.332 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-11T14:04:26.333 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-06-11T14:04:26.333 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-06-11T14:04:26.333 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@mon.c 2024-06-11T14:04:26.590 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:04:26 smithi057 systemd[1]: Stopping Ceph mon.c for 98a6635e-27f5-11ef-bc9b-c7b262605968... 2024-06-11T14:04:26.841 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:04:26 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mon-c[40582]: 2024-06-11T14:04:26.708+0000 7fbd05592640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-11T14:04:26.841 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:04:26 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mon-c[40582]: 2024-06-11T14:04:26.708+0000 7fbd05592640 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-06-11T14:04:26.842 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 14:04:26 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: [11/Jun/2024:14:04:26] ENGINE Bus STOPPING 2024-06-11T14:04:26.842 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 14:04:26 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: [11/Jun/2024:14:04:26] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-06-11T14:04:26.842 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 14:04:26 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: [11/Jun/2024:14:04:26] ENGINE Bus STOPPED 2024-06-11T14:04:26.842 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 14:04:26 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: [11/Jun/2024:14:04:26] ENGINE Bus STARTING 2024-06-11T14:04:26.842 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 14:04:26 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: [11/Jun/2024:14:04:26] ENGINE Serving on http://:::9283 2024-06-11T14:04:26.842 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 14:04:26 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: [11/Jun/2024:14:04:26] ENGINE Bus STARTED 2024-06-11T14:04:27.176 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:04:26 smithi057 podman[155584]: 2024-06-11 14:04:26.913952533 +0000 UTC m=+0.318209646 container died ee99e6aed6f81e6eed5ff1d138acc4f10fd09445112cd73bd134836ef239b6cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-3f51ea3, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.35.3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528) 2024-06-11T14:04:27.536 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:04:27 smithi057 podman[155584]: 2024-06-11 14:04:27.276358408 +0000 UTC m=+0.680615530 container cleanup ee99e6aed6f81e6eed5ff1d138acc4f10fd09445112cd73bd134836ef239b6cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mon-c, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=squid-3f51ea3, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.35.3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528) 2024-06-11T14:04:27.537 INFO:journalctl@ceph.mon.c.smithi057.stdout:Jun 11 14:04:27 smithi057 bash[155584]: ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mon-c 2024-06-11T14:04:27.714 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@mon.c.service' 2024-06-11T14:04:27.751 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-11T14:04:27.751 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-06-11T14:04:27.751 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-06-11T14:04:27.752 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@mon.b 2024-06-11T14:04:28.050 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:04:27 smithi107 systemd[1]: Stopping Ceph mon.b for 98a6635e-27f5-11ef-bc9b-c7b262605968... 2024-06-11T14:04:28.355 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:04:28 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mon-b[33161]: 2024-06-11T14:04:28.090+0000 7f08cff97640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-11T14:04:28.355 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:04:28 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mon-b[33161]: 2024-06-11T14:04:28.090+0000 7f08cff97640 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-06-11T14:04:28.649 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:04:28 smithi107 podman[75074]: 2024-06-11 14:04:28.356256969 +0000 UTC m=+0.399903420 container died 95117eeb1e103a8bb5e9ff99530bc3442ca96fb5cec22dc29dee295055197180 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mon-b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.35.3, RELEASE=squid-3f51ea3, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-06-11T14:04:29.049 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:04:28 smithi107 podman[75074]: 2024-06-11 14:04:28.824559627 +0000 UTC m=+0.868206070 container cleanup 95117eeb1e103a8bb5e9ff99530bc3442ca96fb5cec22dc29dee295055197180 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mon-b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, RELEASE=squid-3f51ea3, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.35.3, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-06-11T14:04:29.050 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 11 14:04:28 smithi107 bash[75074]: ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mon-b 2024-06-11T14:04:29.206 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@mon.b.service' 2024-06-11T14:04:29.255 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-11T14:04:29.255 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-06-11T14:04:29.256 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-06-11T14:04:29.256 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@mgr.y 2024-06-11T14:04:29.538 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 14:04:29 smithi057 systemd[1]: Stopping Ceph mgr.y for 98a6635e-27f5-11ef-bc9b-c7b262605968... 2024-06-11T14:04:29.881 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 14:04:29 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T14:04:29.589+0000 7febf1030640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-11T14:04:29.882 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 14:04:29 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y[30033]: 2024-06-11T14:04:29.589+0000 7febf1030640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-06-11T14:04:29.882 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 14:04:29 smithi057 podman[155716]: 2024-06-11 14:04:29.620669603 +0000 UTC m=+0.152453199 container died 146c79ecd0c5287ad62144b6265271e2f5ddfc94f790061b2b5da1244c2a8d11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, io.buildah.version=1.35.3, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-3f51ea3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-06-11T14:04:30.286 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 14:04:29 smithi057 podman[155716]: 2024-06-11 14:04:29.98904031 +0000 UTC m=+0.520823910 container cleanup 146c79ecd0c5287ad62144b6265271e2f5ddfc94f790061b2b5da1244c2a8d11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3f51ea3944d39ca13aa40153c9a4ee298306679d, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , RELEASE=squid-3f51ea3, io.buildah.version=1.35.3, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-11T14:04:30.286 INFO:journalctl@ceph.mgr.y.smithi057.stdout:Jun 11 14:04:29 smithi057 bash[155716]: ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-y 2024-06-11T14:04:30.478 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@mgr.y.service' 2024-06-11T14:04:30.513 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-11T14:04:30.513 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-06-11T14:04:30.514 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-06-11T14:04:30.514 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@mgr.x 2024-06-11T14:04:30.800 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 14:04:30 smithi107 systemd[1]: Stopping Ceph mgr.x for 98a6635e-27f5-11ef-bc9b-c7b262605968... 2024-06-11T14:04:31.128 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 14:04:30 smithi107 podman[75194]: 2024-06-11 14:04:30.850265203 +0000 UTC m=+0.132334884 container died cf9f89db8ecd9020a2d4a86caa6a4a8109c37fbab82e00ef4adfe46f2dd10b66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x, ceph=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, io.buildah.version=1.35.3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-3f51ea3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-06-11T14:04:31.549 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 14:04:31 smithi107 podman[75194]: 2024-06-11 14:04:31.195077986 +0000 UTC m=+0.477147668 container cleanup cf9f89db8ecd9020a2d4a86caa6a4a8109c37fbab82e00ef4adfe46f2dd10b66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=squid-3f51ea3, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.35.3, CEPH_POINT_RELEASE=) 2024-06-11T14:04:31.550 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 14:04:31 smithi107 bash[75194]: ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x 2024-06-11T14:04:31.550 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 14:04:31 smithi107 podman[75208]: 2024-06-11 14:04:31.355107212 +0000 UTC m=+0.499581155 container remove cf9f89db8ecd9020a2d4a86caa6a4a8109c37fbab82e00ef4adfe46f2dd10b66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-3f51ea3, org.label-schema.vendor=CentOS, io.buildah.version=1.35.3, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-06-11T14:04:31.550 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jun 11 14:04:31 smithi107 systemd[1]: ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@mgr.x.service: Main process exited, code=exited, status=143/n/a 2024-06-11T14:04:31.574 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@mgr.x.service' 2024-06-11T14:04:31.608 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-11T14:04:31.608 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-06-11T14:04:31.609 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-06-11T14:04:31.609 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@osd.0 2024-06-11T14:04:31.935 INFO:journalctl@ceph.osd.0.smithi057.stdout:Jun 11 14:04:31 smithi057 systemd[1]: Stopping Ceph osd.0 for 98a6635e-27f5-11ef-bc9b-c7b262605968... 2024-06-11T14:04:32.286 INFO:journalctl@ceph.osd.0.smithi057.stdout:Jun 11 14:04:31 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-0[44455]: 2024-06-11T14:04:31.935+0000 7f4adc4a0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-11T14:04:32.313 INFO:journalctl@ceph.osd.0.smithi057.stdout:Jun 11 14:04:31 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-0[44455]: 2024-06-11T14:04:31.935+0000 7f4adc4a0640 -1 osd.0 804 *** Got signal Terminated *** 2024-06-11T14:04:32.313 INFO:journalctl@ceph.osd.0.smithi057.stdout:Jun 11 14:04:31 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-0[44455]: 2024-06-11T14:04:31.935+0000 7f4adc4a0640 -1 osd.0 804 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-11T14:04:37.593 INFO:journalctl@ceph.osd.0.smithi057.stdout:Jun 11 14:04:37 smithi057 podman[155837]: 2024-06-11 14:04:37.307527147 +0000 UTC m=+5.497567428 container died 989b91edbd5d202dedc5c10e1d53836b4308cd89413fe9e049e72ddaf83ac46d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.35.3, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-3f51ea3, org.label-schema.build-date=20240528) 2024-06-11T14:04:37.985 INFO:journalctl@ceph.osd.0.smithi057.stdout:Jun 11 14:04:37 smithi057 podman[155837]: 2024-06-11 14:04:37.7189325 +0000 UTC m=+5.908972782 container cleanup 989b91edbd5d202dedc5c10e1d53836b4308cd89413fe9e049e72ddaf83ac46d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-0, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, io.buildah.version=1.35.3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=squid-3f51ea3, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS) 2024-06-11T14:04:37.986 INFO:journalctl@ceph.osd.0.smithi057.stdout:Jun 11 14:04:37 smithi057 bash[155837]: ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-0 2024-06-11T14:04:38.286 INFO:journalctl@ceph.osd.0.smithi057.stdout:Jun 11 14:04:37 smithi057 podman[155856]: 2024-06-11 14:04:37.986749238 +0000 UTC m=+0.673919326 container remove 989b91edbd5d202dedc5c10e1d53836b4308cd89413fe9e049e72ddaf83ac46d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-0, maintainer=Guillaume Abrioux , io.buildah.version=1.35.3, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240528, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-3f51ea3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-06-11T14:04:38.697 INFO:journalctl@ceph.osd.0.smithi057.stdout:Jun 11 14:04:38 smithi057 podman[155944]: 2024-06-11 14:04:38.257080456 +0000 UTC m=+0.017958274 image pull 2d7459ec65a098c35c774154617f2893b0ad051f56ce89b72ab795f65a97fc22 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310 2024-06-11T14:04:38.697 INFO:journalctl@ceph.osd.0.smithi057.stdout:Jun 11 14:04:38 smithi057 podman[155944]: 2024-06-11 14:04:38.371117303 +0000 UTC m=+0.131995156 container create fcf4fd40c96727788f01647fc7146e456f29aac045217ef47c7ca5c648a20c0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-3f51ea3, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.35.3, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux ) 2024-06-11T14:04:38.698 INFO:journalctl@ceph.osd.0.smithi057.stdout:Jun 11 14:04:38 smithi057 podman[155944]: 2024-06-11 14:04:38.598499203 +0000 UTC m=+0.359377007 container init fcf4fd40c96727788f01647fc7146e456f29aac045217ef47c7ca5c648a20c0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.35.3, ceph=True, RELEASE=squid-3f51ea3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528) 2024-06-11T14:04:38.698 INFO:journalctl@ceph.osd.0.smithi057.stdout:Jun 11 14:04:38 smithi057 podman[155944]: 2024-06-11 14:04:38.601925661 +0000 UTC m=+0.362803472 container start fcf4fd40c96727788f01647fc7146e456f29aac045217ef47c7ca5c648a20c0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-0-deactivate, io.buildah.version=1.35.3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-3f51ea3) 2024-06-11T14:04:38.698 INFO:journalctl@ceph.osd.0.smithi057.stdout:Jun 11 14:04:38 smithi057 podman[155944]: 2024-06-11 14:04:38.698585642 +0000 UTC m=+0.459463469 container attach fcf4fd40c96727788f01647fc7146e456f29aac045217ef47c7ca5c648a20c0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.35.3, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-3f51ea3, org.label-schema.vendor=CentOS) 2024-06-11T14:04:39.036 INFO:journalctl@ceph.osd.0.smithi057.stdout:Jun 11 14:04:38 smithi057 podman[155967]: 2024-06-11 14:04:38.899190466 +0000 UTC m=+0.018757205 container died fcf4fd40c96727788f01647fc7146e456f29aac045217ef47c7ca5c648a20c0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, io.buildah.version=1.35.3, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-3f51ea3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-06-11T14:04:39.438 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@osd.0.service' 2024-06-11T14:04:39.485 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-11T14:04:39.485 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-06-11T14:04:39.485 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-06-11T14:04:39.485 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@osd.1 2024-06-11T14:04:39.726 INFO:journalctl@ceph.osd.1.smithi057.stdout:Jun 11 14:04:39 smithi057 systemd[1]: Stopping Ceph osd.1 for 98a6635e-27f5-11ef-bc9b-c7b262605968... 2024-06-11T14:04:40.036 INFO:journalctl@ceph.osd.1.smithi057.stdout:Jun 11 14:04:39 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-1[50184]: 2024-06-11T14:04:39.829+0000 7fc030781640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-11T14:04:40.037 INFO:journalctl@ceph.osd.1.smithi057.stdout:Jun 11 14:04:39 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-1[50184]: 2024-06-11T14:04:39.829+0000 7fc030781640 -1 osd.1 804 *** Got signal Terminated *** 2024-06-11T14:04:40.037 INFO:journalctl@ceph.osd.1.smithi057.stdout:Jun 11 14:04:39 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-1[50184]: 2024-06-11T14:04:39.829+0000 7fc030781640 -1 osd.1 804 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-11T14:04:45.613 INFO:journalctl@ceph.osd.1.smithi057.stdout:Jun 11 14:04:45 smithi057 podman[156048]: 2024-06-11 14:04:45.344154139 +0000 UTC m=+5.646245053 container died 7554ada8795ef157e37272164727d33b2cef92875baba96bd5c64f4cc5b68cbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-1, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3f51ea3, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.35.3, GIT_BRANCH=HEAD) 2024-06-11T14:04:45.955 INFO:journalctl@ceph.osd.1.smithi057.stdout:Jun 11 14:04:45 smithi057 podman[156048]: 2024-06-11 14:04:45.696672682 +0000 UTC m=+5.998763596 container cleanup 7554ada8795ef157e37272164727d33b2cef92875baba96bd5c64f4cc5b68cbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-1, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240528, RELEASE=squid-3f51ea3, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.35.3, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-06-11T14:04:45.955 INFO:journalctl@ceph.osd.1.smithi057.stdout:Jun 11 14:04:45 smithi057 bash[156048]: ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-1 2024-06-11T14:04:46.286 INFO:journalctl@ceph.osd.1.smithi057.stdout:Jun 11 14:04:45 smithi057 podman[156078]: 2024-06-11 14:04:45.956638563 +0000 UTC m=+0.609983158 container remove 7554ada8795ef157e37272164727d33b2cef92875baba96bd5c64f4cc5b68cbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-1, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-3f51ea3, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.35.3) 2024-06-11T14:04:46.583 INFO:journalctl@ceph.osd.1.smithi057.stdout:Jun 11 14:04:46 smithi057 podman[156164]: 2024-06-11 14:04:46.233760621 +0000 UTC m=+0.017791211 image pull 2d7459ec65a098c35c774154617f2893b0ad051f56ce89b72ab795f65a97fc22 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310 2024-06-11T14:04:46.584 INFO:journalctl@ceph.osd.1.smithi057.stdout:Jun 11 14:04:46 smithi057 podman[156164]: 2024-06-11 14:04:46.340819266 +0000 UTC m=+0.124849847 container create 2c200b3b762d322175737b07bfc29640ed13db332dbdfbecf9dcfab9fe6bc152 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3f51ea3, maintainer=Guillaume Abrioux , io.buildah.version=1.35.3, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-06-11T14:04:46.584 INFO:journalctl@ceph.osd.1.smithi057.stdout:Jun 11 14:04:46 smithi057 podman[156164]: 2024-06-11 14:04:46.584636872 +0000 UTC m=+0.368667462 container init 2c200b3b762d322175737b07bfc29640ed13db332dbdfbecf9dcfab9fe6bc152 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-1-deactivate, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.35.3, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-3f51ea3, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux ) 2024-06-11T14:04:46.847 INFO:journalctl@ceph.osd.1.smithi057.stdout:Jun 11 14:04:46 smithi057 podman[156164]: 2024-06-11 14:04:46.588089373 +0000 UTC m=+0.372119970 container start 2c200b3b762d322175737b07bfc29640ed13db332dbdfbecf9dcfab9fe6bc152 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.35.3, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, RELEASE=squid-3f51ea3, GIT_CLEAN=True) 2024-06-11T14:04:46.847 INFO:journalctl@ceph.osd.1.smithi057.stdout:Jun 11 14:04:46 smithi057 podman[156164]: 2024-06-11 14:04:46.667945595 +0000 UTC m=+0.451976194 container attach 2c200b3b762d322175737b07bfc29640ed13db332dbdfbecf9dcfab9fe6bc152 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.35.3, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, RELEASE=squid-3f51ea3, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-06-11T14:04:47.203 INFO:journalctl@ceph.osd.1.smithi057.stdout:Jun 11 14:04:46 smithi057 podman[156164]: 2024-06-11 14:04:46.848240863 +0000 UTC m=+0.632271452 container died 2c200b3b762d322175737b07bfc29640ed13db332dbdfbecf9dcfab9fe6bc152 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=squid-3f51ea3, io.buildah.version=1.35.3, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True) 2024-06-11T14:04:47.526 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@osd.1.service' 2024-06-11T14:04:47.574 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-11T14:04:47.575 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-06-11T14:04:47.575 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-06-11T14:04:47.575 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@osd.2 2024-06-11T14:04:47.786 INFO:journalctl@ceph.osd.2.smithi057.stdout:Jun 11 14:04:47 smithi057 systemd[1]: Stopping Ceph osd.2 for 98a6635e-27f5-11ef-bc9b-c7b262605968... 2024-06-11T14:04:48.286 INFO:journalctl@ceph.osd.2.smithi057.stdout:Jun 11 14:04:47 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-2[56152]: 2024-06-11T14:04:47.908+0000 7fc96f122640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-11T14:04:48.287 INFO:journalctl@ceph.osd.2.smithi057.stdout:Jun 11 14:04:47 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-2[56152]: 2024-06-11T14:04:47.908+0000 7fc96f122640 -1 osd.2 804 *** Got signal Terminated *** 2024-06-11T14:04:48.287 INFO:journalctl@ceph.osd.2.smithi057.stdout:Jun 11 14:04:47 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-2[56152]: 2024-06-11T14:04:47.908+0000 7fc96f122640 -1 osd.2 804 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-11T14:04:53.575 INFO:journalctl@ceph.osd.2.smithi057.stdout:Jun 11 14:04:53 smithi057 podman[156265]: 2024-06-11 14:04:53.314772146 +0000 UTC m=+5.546054126 container died a5ac63467597f75facf675ed5d8b1ab3ba831f23274d709f104d17265e179e2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-3f51ea3, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, io.buildah.version=1.35.3, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-06-11T14:04:53.958 INFO:journalctl@ceph.osd.2.smithi057.stdout:Jun 11 14:04:53 smithi057 podman[156265]: 2024-06-11 14:04:53.674475843 +0000 UTC m=+5.905757831 container cleanup a5ac63467597f75facf675ed5d8b1ab3ba831f23274d709f104d17265e179e2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-2, org.label-schema.build-date=20240528, io.buildah.version=1.35.3, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3f51ea3, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-06-11T14:04:53.958 INFO:journalctl@ceph.osd.2.smithi057.stdout:Jun 11 14:04:53 smithi057 bash[156265]: ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-2 2024-06-11T14:04:54.286 INFO:journalctl@ceph.osd.2.smithi057.stdout:Jun 11 14:04:53 smithi057 podman[156283]: 2024-06-11 14:04:53.958943412 +0000 UTC m=+0.638046788 container remove a5ac63467597f75facf675ed5d8b1ab3ba831f23274d709f104d17265e179e2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.35.3, org.label-schema.vendor=CentOS, RELEASE=squid-3f51ea3, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240528) 2024-06-11T14:04:54.594 INFO:journalctl@ceph.osd.2.smithi057.stdout:Jun 11 14:04:54 smithi057 podman[156373]: 2024-06-11 14:04:54.218729216 +0000 UTC m=+0.017877238 image pull 2d7459ec65a098c35c774154617f2893b0ad051f56ce89b72ab795f65a97fc22 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310 2024-06-11T14:04:54.595 INFO:journalctl@ceph.osd.2.smithi057.stdout:Jun 11 14:04:54 smithi057 podman[156373]: 2024-06-11 14:04:54.318005587 +0000 UTC m=+0.117153599 container create 131ccc9f1b4915ee4df94b4cbdcd9eda7a0ecf432274d4d1cc34b5933a2911f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3f51ea3, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, io.buildah.version=1.35.3, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-06-11T14:04:54.595 INFO:journalctl@ceph.osd.2.smithi057.stdout:Jun 11 14:04:54 smithi057 podman[156373]: 2024-06-11 14:04:54.520599696 +0000 UTC m=+0.319747716 container init 131ccc9f1b4915ee4df94b4cbdcd9eda7a0ecf432274d4d1cc34b5933a2911f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-2-deactivate, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240528, io.buildah.version=1.35.3, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-3f51ea3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-06-11T14:04:54.595 INFO:journalctl@ceph.osd.2.smithi057.stdout:Jun 11 14:04:54 smithi057 podman[156373]: 2024-06-11 14:04:54.524186127 +0000 UTC m=+0.323334141 container start 131ccc9f1b4915ee4df94b4cbdcd9eda7a0ecf432274d4d1cc34b5933a2911f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-3f51ea3, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.35.3, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-06-11T14:04:54.595 INFO:journalctl@ceph.osd.2.smithi057.stdout:Jun 11 14:04:54 smithi057 podman[156373]: 2024-06-11 14:04:54.595075011 +0000 UTC m=+0.394223031 container attach 131ccc9f1b4915ee4df94b4cbdcd9eda7a0ecf432274d4d1cc34b5933a2911f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-2-deactivate, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-3f51ea3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.35.3) 2024-06-11T14:04:55.036 INFO:journalctl@ceph.osd.2.smithi057.stdout:Jun 11 14:04:54 smithi057 podman[156373]: 2024-06-11 14:04:54.795152193 +0000 UTC m=+0.594300207 container died 131ccc9f1b4915ee4df94b4cbdcd9eda7a0ecf432274d4d1cc34b5933a2911f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-2-deactivate, RELEASE=squid-3f51ea3, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240528, io.buildah.version=1.35.3, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-06-11T14:04:55.341 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@osd.2.service' 2024-06-11T14:04:55.388 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-11T14:04:55.388 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-06-11T14:04:55.388 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-06-11T14:04:55.388 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@osd.3 2024-06-11T14:04:55.687 INFO:journalctl@ceph.osd.3.smithi057.stdout:Jun 11 14:04:55 smithi057 systemd[1]: Stopping Ceph osd.3 for 98a6635e-27f5-11ef-bc9b-c7b262605968... 2024-06-11T14:04:56.036 INFO:journalctl@ceph.osd.3.smithi057.stdout:Jun 11 14:04:55 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-3[61812]: 2024-06-11T14:04:55.686+0000 7f2b7b9e8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-11T14:04:56.036 INFO:journalctl@ceph.osd.3.smithi057.stdout:Jun 11 14:04:55 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-3[61812]: 2024-06-11T14:04:55.686+0000 7f2b7b9e8640 -1 osd.3 804 *** Got signal Terminated *** 2024-06-11T14:04:56.037 INFO:journalctl@ceph.osd.3.smithi057.stdout:Jun 11 14:04:55 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-3[61812]: 2024-06-11T14:04:55.686+0000 7f2b7b9e8640 -1 osd.3 804 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-11T14:04:58.050 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 11 14:04:57 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-4[37950]: 2024-06-11T14:04:57.731+0000 7f3006d0e640 -1 osd.4 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:31.663336+0000 front 2024-06-11T14:04:31.663312+0000 (oldest deadline 2024-06-11T14:04:57.563257+0000) 2024-06-11T14:04:58.550 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 14:04:58 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-5[43766]: 2024-06-11T14:04:58.420+0000 7f698cf5f640 -1 osd.5 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:34.389374+0000 front 2024-06-11T14:04:34.389498+0000 (oldest deadline 2024-06-11T14:04:57.889373+0000) 2024-06-11T14:04:58.550 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:04:58 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:04:58.189+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:31.703677+0000 front 2024-06-11T14:04:31.703581+0000 (oldest deadline 2024-06-11T14:04:57.603720+0000) 2024-06-11T14:04:58.986 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 11 14:04:58 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-4[37950]: 2024-06-11T14:04:58.692+0000 7f3006d0e640 -1 osd.4 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:31.663336+0000 front 2024-06-11T14:04:31.663312+0000 (oldest deadline 2024-06-11T14:04:57.563257+0000) 2024-06-11T14:04:59.300 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:04:59 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:04:59.156+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:31.703677+0000 front 2024-06-11T14:04:31.703581+0000 (oldest deadline 2024-06-11T14:04:57.603720+0000) 2024-06-11T14:04:59.300 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:04:58 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:04:58.987+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:33.012331+0000 front 2024-06-11T14:04:33.012304+0000 (oldest deadline 2024-06-11T14:04:58.312138+0000) 2024-06-11T14:04:59.723 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 14:04:59 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-5[43766]: 2024-06-11T14:04:59.419+0000 7f698cf5f640 -1 osd.5 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:34.389374+0000 front 2024-06-11T14:04:34.389498+0000 (oldest deadline 2024-06-11T14:04:57.889373+0000) 2024-06-11T14:05:00.050 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:04:59 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:04:59.967+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:33.012331+0000 front 2024-06-11T14:04:33.012304+0000 (oldest deadline 2024-06-11T14:04:58.312138+0000) 2024-06-11T14:05:00.050 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 11 14:04:59 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-4[37950]: 2024-06-11T14:04:59.724+0000 7f3006d0e640 -1 osd.4 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:31.663336+0000 front 2024-06-11T14:04:31.663312+0000 (oldest deadline 2024-06-11T14:04:57.563257+0000) 2024-06-11T14:05:00.286 INFO:journalctl@ceph.osd.3.smithi057.stdout:Jun 11 14:04:59 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-3[61812]: 2024-06-11T14:04:59.961+0000 7f2b73ff6640 -1 osd.3 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:36.924393+0000 front 2024-06-11T14:04:36.924609+0000 (oldest deadline 2024-06-11T14:04:59.824544+0000) 2024-06-11T14:05:00.550 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 14:05:00 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-5[43766]: 2024-06-11T14:05:00.398+0000 7f698cf5f640 -1 osd.5 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:34.389374+0000 front 2024-06-11T14:04:34.389498+0000 (oldest deadline 2024-06-11T14:04:57.889373+0000) 2024-06-11T14:05:00.550 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:00 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:00.190+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:31.703677+0000 front 2024-06-11T14:04:31.703581+0000 (oldest deadline 2024-06-11T14:04:57.603720+0000) 2024-06-11T14:05:00.967 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 11 14:05:00 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-4[37950]: 2024-06-11T14:05:00.713+0000 7f3006d0e640 -1 osd.4 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:31.663336+0000 front 2024-06-11T14:04:31.663312+0000 (oldest deadline 2024-06-11T14:04:57.563257+0000) 2024-06-11T14:05:01.300 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:00 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:00.967+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:33.012331+0000 front 2024-06-11T14:04:33.012304+0000 (oldest deadline 2024-06-11T14:04:58.312138+0000) 2024-06-11T14:05:01.300 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:01 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:01.217+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:31.703677+0000 front 2024-06-11T14:04:31.703581+0000 (oldest deadline 2024-06-11T14:04:57.603720+0000) 2024-06-11T14:05:01.461 INFO:journalctl@ceph.osd.3.smithi057.stdout:Jun 11 14:05:01 smithi057 podman[156476]: 2024-06-11 14:05:01.163610816 +0000 UTC m=+5.585405277 container died bccd0c3e73fd3c86be17c6e2b4708508e3b4717f32f5adc2b3974b6681ebe982 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, RELEASE=squid-3f51ea3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.35.3) 2024-06-11T14:05:01.461 INFO:journalctl@ceph.osd.3.smithi057.stdout:Jun 11 14:05:01 smithi057 podman[156476]: 2024-06-11 14:05:01.462372721 +0000 UTC m=+5.884167178 container cleanup bccd0c3e73fd3c86be17c6e2b4708508e3b4717f32f5adc2b3974b6681ebe982 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=squid-3f51ea3, io.buildah.version=1.35.3, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-11T14:05:01.701 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 14:05:01 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-5[43766]: 2024-06-11T14:05:01.358+0000 7f698cf5f640 -1 osd.5 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:34.389374+0000 front 2024-06-11T14:04:34.389498+0000 (oldest deadline 2024-06-11T14:04:57.889373+0000) 2024-06-11T14:05:01.786 INFO:journalctl@ceph.osd.3.smithi057.stdout:Jun 11 14:05:01 smithi057 bash[156476]: ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-3 2024-06-11T14:05:01.786 INFO:journalctl@ceph.osd.3.smithi057.stdout:Jun 11 14:05:01 smithi057 podman[156503]: 2024-06-11 14:05:01.680303621 +0000 UTC m=+0.510021370 container remove bccd0c3e73fd3c86be17c6e2b4708508e3b4717f32f5adc2b3974b6681ebe982 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, RELEASE=squid-3f51ea3, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , io.buildah.version=1.35.3, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-06-11T14:05:01.961 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 11 14:05:01 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-4[37950]: 2024-06-11T14:05:01.702+0000 7f3006d0e640 -1 osd.4 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:31.663336+0000 front 2024-06-11T14:04:31.663312+0000 (oldest deadline 2024-06-11T14:04:57.563257+0000) 2024-06-11T14:05:02.244 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:01 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:01.961+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:33.012331+0000 front 2024-06-11T14:04:33.012304+0000 (oldest deadline 2024-06-11T14:04:58.312138+0000) 2024-06-11T14:05:02.267 INFO:journalctl@ceph.osd.3.smithi057.stdout:Jun 11 14:05:02 smithi057 podman[156598]: 2024-06-11 14:05:02.006492565 +0000 UTC m=+0.113764487 container create 3a6aefca2360d72d001abdaf1ad2316dbbd859ee619658372291c6d296689af9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.35.3, RELEASE=squid-3f51ea3, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-06-11T14:05:02.267 INFO:journalctl@ceph.osd.3.smithi057.stdout:Jun 11 14:05:02 smithi057 podman[156598]: 2024-06-11 14:05:01.910502348 +0000 UTC m=+0.017774289 image pull 2d7459ec65a098c35c774154617f2893b0ad051f56ce89b72ab795f65a97fc22 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310 2024-06-11T14:05:02.267 INFO:journalctl@ceph.osd.3.smithi057.stdout:Jun 11 14:05:02 smithi057 podman[156598]: 2024-06-11 14:05:02.267805567 +0000 UTC m=+0.375077503 container init 3a6aefca2360d72d001abdaf1ad2316dbbd859ee619658372291c6d296689af9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.35.3, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, RELEASE=squid-3f51ea3, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_BRANCH=HEAD) 2024-06-11T14:05:02.536 INFO:journalctl@ceph.osd.3.smithi057.stdout:Jun 11 14:05:02 smithi057 podman[156598]: 2024-06-11 14:05:02.271872868 +0000 UTC m=+0.379144806 container start 3a6aefca2360d72d001abdaf1ad2316dbbd859ee619658372291c6d296689af9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-3-deactivate, RELEASE=squid-3f51ea3, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.35.3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-06-11T14:05:02.536 INFO:journalctl@ceph.osd.3.smithi057.stdout:Jun 11 14:05:02 smithi057 podman[156598]: 2024-06-11 14:05:02.342576566 +0000 UTC m=+0.449848513 container attach 3a6aefca2360d72d001abdaf1ad2316dbbd859ee619658372291c6d296689af9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.35.3, org.label-schema.license=GPLv2, RELEASE=squid-3f51ea3, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240528) 2024-06-11T14:05:02.549 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 14:05:02 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-5[43766]: 2024-06-11T14:05:02.325+0000 7f698cf5f640 -1 osd.5 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:34.389374+0000 front 2024-06-11T14:04:34.389498+0000 (oldest deadline 2024-06-11T14:04:57.889373+0000) 2024-06-11T14:05:02.550 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:02 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:02.244+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:31.703677+0000 front 2024-06-11T14:04:31.703581+0000 (oldest deadline 2024-06-11T14:04:57.603720+0000) 2024-06-11T14:05:02.819 INFO:journalctl@ceph.osd.3.smithi057.stdout:Jun 11 14:05:02 smithi057 podman[156598]: 2024-06-11 14:05:02.547801865 +0000 UTC m=+0.655073811 container died 3a6aefca2360d72d001abdaf1ad2316dbbd859ee619658372291c6d296689af9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-3-deactivate, RELEASE=squid-3f51ea3, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.35.3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240528) 2024-06-11T14:05:03.050 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 11 14:05:02 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-4[37950]: 2024-06-11T14:05:02.677+0000 7f3006d0e640 -1 osd.4 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:31.663336+0000 front 2024-06-11T14:04:31.663312+0000 (oldest deadline 2024-06-11T14:04:57.563257+0000) 2024-06-11T14:05:03.050 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:02 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:02.925+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:33.012331+0000 front 2024-06-11T14:04:33.012304+0000 (oldest deadline 2024-06-11T14:04:58.312138+0000) 2024-06-11T14:05:03.062 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@osd.3.service' 2024-06-11T14:05:03.089 INFO:journalctl@ceph.osd.3.smithi057.stdout:Jun 11 14:05:03 smithi057 podman[156620]: 2024-06-11 14:05:03.037795202 +0000 UTC m=+0.484327464 container remove 3a6aefca2360d72d001abdaf1ad2316dbbd859ee619658372291c6d296689af9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.35.3, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240528, ceph=True, RELEASE=squid-3f51ea3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-06-11T14:05:03.089 INFO:journalctl@ceph.osd.3.smithi057.stdout:Jun 11 14:05:03 smithi057 systemd[1]: ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@osd.3.service: Deactivated successfully. 2024-06-11T14:05:03.089 INFO:journalctl@ceph.osd.3.smithi057.stdout:Jun 11 14:05:03 smithi057 systemd[1]: Stopped Ceph osd.3 for 98a6635e-27f5-11ef-bc9b-c7b262605968. 2024-06-11T14:05:03.089 INFO:journalctl@ceph.osd.3.smithi057.stdout:Jun 11 14:05:03 smithi057 systemd[1]: ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@osd.3.service: Consumed 55.579s CPU time. 2024-06-11T14:05:03.106 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-11T14:05:03.106 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-06-11T14:05:03.106 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-06-11T14:05:03.106 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@osd.4 2024-06-11T14:05:03.453 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 11 14:05:03 smithi107 systemd[1]: Stopping Ceph osd.4 for 98a6635e-27f5-11ef-bc9b-c7b262605968... 2024-06-11T14:05:03.454 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 14:05:03 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-5[43766]: 2024-06-11T14:05:03.276+0000 7f698cf5f640 -1 osd.5 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:34.389374+0000 front 2024-06-11T14:04:34.389498+0000 (oldest deadline 2024-06-11T14:04:57.889373+0000) 2024-06-11T14:05:03.454 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:03 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:03.260+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:31.703677+0000 front 2024-06-11T14:04:31.703581+0000 (oldest deadline 2024-06-11T14:04:57.603720+0000) 2024-06-11T14:05:03.708 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 11 14:05:03 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-4[37950]: 2024-06-11T14:05:03.453+0000 7f300e700640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-11T14:05:03.708 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 11 14:05:03 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-4[37950]: 2024-06-11T14:05:03.453+0000 7f300e700640 -1 osd.4 804 *** Got signal Terminated *** 2024-06-11T14:05:03.708 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 11 14:05:03 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-4[37950]: 2024-06-11T14:05:03.453+0000 7f300e700640 -1 osd.4 804 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-11T14:05:03.708 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 11 14:05:03 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-4[37950]: 2024-06-11T14:05:03.708+0000 7f3006d0e640 -1 osd.4 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:31.663336+0000 front 2024-06-11T14:04:31.663312+0000 (oldest deadline 2024-06-11T14:04:57.563257+0000) 2024-06-11T14:05:04.265 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:03 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:03.972+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:33.012331+0000 front 2024-06-11T14:04:33.012304+0000 (oldest deadline 2024-06-11T14:04:58.312138+0000) 2024-06-11T14:05:04.550 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 14:05:04 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-5[43766]: 2024-06-11T14:05:04.266+0000 7f698cf5f640 -1 osd.5 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:34.389374+0000 front 2024-06-11T14:04:34.389498+0000 (oldest deadline 2024-06-11T14:04:57.889373+0000) 2024-06-11T14:05:04.550 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:04 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:04.301+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:31.703677+0000 front 2024-06-11T14:04:31.703581+0000 (oldest deadline 2024-06-11T14:04:57.603720+0000) 2024-06-11T14:05:05.050 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 11 14:05:04 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-4[37950]: 2024-06-11T14:05:04.741+0000 7f3006d0e640 -1 osd.4 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:31.663336+0000 front 2024-06-11T14:04:31.663312+0000 (oldest deadline 2024-06-11T14:04:57.563257+0000) 2024-06-11T14:05:05.050 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:04 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:04.949+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:33.012331+0000 front 2024-06-11T14:04:33.012304+0000 (oldest deadline 2024-06-11T14:04:58.312138+0000) 2024-06-11T14:05:05.550 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:05 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:05.319+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:31.703677+0000 front 2024-06-11T14:04:31.703581+0000 (oldest deadline 2024-06-11T14:04:57.603720+0000) 2024-06-11T14:05:05.550 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 14:05:05 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-5[43766]: 2024-06-11T14:05:05.227+0000 7f698cf5f640 -1 osd.5 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:34.389374+0000 front 2024-06-11T14:04:34.389498+0000 (oldest deadline 2024-06-11T14:04:57.889373+0000) 2024-06-11T14:05:05.923 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 11 14:05:05 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-4[37950]: 2024-06-11T14:05:05.748+0000 7f3006d0e640 -1 osd.4 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:31.663336+0000 front 2024-06-11T14:04:31.663312+0000 (oldest deadline 2024-06-11T14:04:57.563257+0000) 2024-06-11T14:05:05.924 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 11 14:05:05 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-4[37950]: 2024-06-11T14:05:05.748+0000 7f3006d0e640 -1 osd.4 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:43.364965+0000 front 2024-06-11T14:04:43.364880+0000 (oldest deadline 2024-06-11T14:05:05.664848+0000) 2024-06-11T14:05:05.924 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 14:05:05 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[64736]: ts=2024-06-11T14:05:05.555Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=node msg="Unable to refresh target groups" err="Get \"http://172.21.15.57:8765/sd/prometheus/sd-config?service=node-exporter\": dial tcp 172.21.15.57:8765: connect: connection refused" 2024-06-11T14:05:05.924 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 14:05:05 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[64736]: ts=2024-06-11T14:05:05.558Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=nvmeof msg="Unable to refresh target groups" err="Get \"http://172.21.15.57:8765/sd/prometheus/sd-config?service=nvmeof\": dial tcp 172.21.15.57:8765: connect: connection refused" 2024-06-11T14:05:05.924 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 14:05:05 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[64736]: ts=2024-06-11T14:05:05.558Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.57:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.57:8765: connect: connection refused" 2024-06-11T14:05:05.924 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 14:05:05 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[64736]: ts=2024-06-11T14:05:05.558Z caller=refresh.go:99 level=error component="discovery manager notify" discovery=http config=config-0 msg="Unable to refresh target groups" err="Get \"http://172.21.15.57:8765/sd/prometheus/sd-config?service=alertmanager\": dial tcp 172.21.15.57:8765: connect: connection refused" 2024-06-11T14:05:05.924 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 14:05:05 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[64736]: ts=2024-06-11T14:05:05.558Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.57:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.57:8765: connect: connection refused" 2024-06-11T14:05:06.258 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:05 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:05.923+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:33.012331+0000 front 2024-06-11T14:04:33.012304+0000 (oldest deadline 2024-06-11T14:04:58.312138+0000) 2024-06-11T14:05:06.550 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 14:05:06 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-5[43766]: 2024-06-11T14:05:06.259+0000 7f698cf5f640 -1 osd.5 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:34.389374+0000 front 2024-06-11T14:04:34.389498+0000 (oldest deadline 2024-06-11T14:04:57.889373+0000) 2024-06-11T14:05:06.550 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:06 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:06.321+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:31.703677+0000 front 2024-06-11T14:04:31.703581+0000 (oldest deadline 2024-06-11T14:04:57.603720+0000) 2024-06-11T14:05:07.050 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:06 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:06.921+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:33.012331+0000 front 2024-06-11T14:04:33.012304+0000 (oldest deadline 2024-06-11T14:04:58.312138+0000) 2024-06-11T14:05:07.050 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 11 14:05:06 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-4[37950]: 2024-06-11T14:05:06.750+0000 7f3006d0e640 -1 osd.4 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:31.663336+0000 front 2024-06-11T14:04:31.663312+0000 (oldest deadline 2024-06-11T14:04:57.563257+0000) 2024-06-11T14:05:07.050 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 11 14:05:06 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-4[37950]: 2024-06-11T14:05:06.750+0000 7f3006d0e640 -1 osd.4 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:43.364965+0000 front 2024-06-11T14:04:43.364880+0000 (oldest deadline 2024-06-11T14:05:05.664848+0000) 2024-06-11T14:05:07.549 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 14:05:07 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-5[43766]: 2024-06-11T14:05:07.238+0000 7f698cf5f640 -1 osd.5 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:34.389374+0000 front 2024-06-11T14:04:34.389498+0000 (oldest deadline 2024-06-11T14:04:57.889373+0000) 2024-06-11T14:05:07.550 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:07 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:07.294+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:31.703677+0000 front 2024-06-11T14:04:31.703581+0000 (oldest deadline 2024-06-11T14:04:57.603720+0000) 2024-06-11T14:05:08.050 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 11 14:05:07 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-4[37950]: 2024-06-11T14:05:07.773+0000 7f3006d0e640 -1 osd.4 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:31.663336+0000 front 2024-06-11T14:04:31.663312+0000 (oldest deadline 2024-06-11T14:04:57.563257+0000) 2024-06-11T14:05:08.050 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 11 14:05:07 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-4[37950]: 2024-06-11T14:05:07.773+0000 7f3006d0e640 -1 osd.4 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:43.364965+0000 front 2024-06-11T14:04:43.364880+0000 (oldest deadline 2024-06-11T14:05:05.664848+0000) 2024-06-11T14:05:08.050 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:07 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:07.882+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:33.012331+0000 front 2024-06-11T14:04:33.012304+0000 (oldest deadline 2024-06-11T14:04:58.312138+0000) 2024-06-11T14:05:08.050 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:07 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:07.882+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:43.613326+0000 front 2024-06-11T14:04:43.613130+0000 (oldest deadline 2024-06-11T14:05:07.713132+0000) 2024-06-11T14:05:08.550 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 14:05:08 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-5[43766]: 2024-06-11T14:05:08.197+0000 7f698cf5f640 -1 osd.5 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:34.389374+0000 front 2024-06-11T14:04:34.389498+0000 (oldest deadline 2024-06-11T14:04:57.889373+0000) 2024-06-11T14:05:08.550 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 14:05:08 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-5[43766]: 2024-06-11T14:05:08.197+0000 7f698cf5f640 -1 osd.5 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:43.190370+0000 front 2024-06-11T14:04:43.190303+0000 (oldest deadline 2024-06-11T14:05:07.290390+0000) 2024-06-11T14:05:08.550 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:08 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:08.254+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:31.703677+0000 front 2024-06-11T14:04:31.703581+0000 (oldest deadline 2024-06-11T14:04:57.603720+0000) 2024-06-11T14:05:09.212 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 11 14:05:08 smithi107 podman[75332]: 2024-06-11 14:05:08.975843233 +0000 UTC m=+5.661282229 container died 4f49533ce6b997f138fb8ad306d8023c4a31d0da1b4c2ecf7b97839143fc1b6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-3f51ea3, io.buildah.version=1.35.3, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-06-11T14:05:09.212 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:08 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:08.852+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:33.012331+0000 front 2024-06-11T14:04:33.012304+0000 (oldest deadline 2024-06-11T14:04:58.312138+0000) 2024-06-11T14:05:09.212 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:08 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:08.852+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:43.613326+0000 front 2024-06-11T14:04:43.613130+0000 (oldest deadline 2024-06-11T14:05:07.713132+0000) 2024-06-11T14:05:09.503 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 11 14:05:09 smithi107 podman[75332]: 2024-06-11 14:05:09.303059547 +0000 UTC m=+5.988498533 container cleanup 4f49533ce6b997f138fb8ad306d8023c4a31d0da1b4c2ecf7b97839143fc1b6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-4, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-3f51ea3, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.35.3, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux ) 2024-06-11T14:05:09.504 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 11 14:05:09 smithi107 bash[75332]: ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-4 2024-06-11T14:05:09.504 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 14:05:09 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-5[43766]: 2024-06-11T14:05:09.226+0000 7f698cf5f640 -1 osd.5 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:34.389374+0000 front 2024-06-11T14:04:34.389498+0000 (oldest deadline 2024-06-11T14:04:57.889373+0000) 2024-06-11T14:05:09.504 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 14:05:09 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-5[43766]: 2024-06-11T14:05:09.226+0000 7f698cf5f640 -1 osd.5 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:43.190370+0000 front 2024-06-11T14:04:43.190303+0000 (oldest deadline 2024-06-11T14:05:07.290390+0000) 2024-06-11T14:05:09.504 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:09 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:09.278+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:31.703677+0000 front 2024-06-11T14:04:31.703581+0000 (oldest deadline 2024-06-11T14:04:57.603720+0000) 2024-06-11T14:05:09.800 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 11 14:05:09 smithi107 podman[75347]: 2024-06-11 14:05:09.504619609 +0000 UTC m=+0.524169683 container remove 4f49533ce6b997f138fb8ad306d8023c4a31d0da1b4c2ecf7b97839143fc1b6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-4, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, io.buildah.version=1.35.3, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-3f51ea3, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS) 2024-06-11T14:05:10.144 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 11 14:05:09 smithi107 podman[75436]: 2024-06-11 14:05:09.913785571 +0000 UTC m=+0.101590091 container create 027fa61a5c06bb7c8e6f3c12e157bc3cda8de0c82818a48e673002bd916731eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.35.3, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, RELEASE=squid-3f51ea3, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-11T14:05:10.144 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 11 14:05:09 smithi107 podman[75436]: 2024-06-11 14:05:09.829051192 +0000 UTC m=+0.016855727 image pull 2d7459ec65a098c35c774154617f2893b0ad051f56ce89b72ab795f65a97fc22 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310 2024-06-11T14:05:10.144 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 11 14:05:10 smithi107 podman[75436]: 2024-06-11 14:05:10.141409996 +0000 UTC m=+0.329214519 container init 027fa61a5c06bb7c8e6f3c12e157bc3cda8de0c82818a48e673002bd916731eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-4-deactivate, ceph=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.35.3, RELEASE=squid-3f51ea3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-11T14:05:10.144 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 11 14:05:10 smithi107 podman[75436]: 2024-06-11 14:05:10.144858473 +0000 UTC m=+0.332663000 container start 027fa61a5c06bb7c8e6f3c12e157bc3cda8de0c82818a48e673002bd916731eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-4-deactivate, io.buildah.version=1.35.3, org.label-schema.build-date=20240528, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-3f51ea3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True) 2024-06-11T14:05:10.144 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:09 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:09.891+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:33.012331+0000 front 2024-06-11T14:04:33.012304+0000 (oldest deadline 2024-06-11T14:04:58.312138+0000) 2024-06-11T14:05:10.144 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:09 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:09.891+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:43.613326+0000 front 2024-06-11T14:04:43.613130+0000 (oldest deadline 2024-06-11T14:05:07.713132+0000) 2024-06-11T14:05:10.408 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 11 14:05:10 smithi107 podman[75436]: 2024-06-11 14:05:10.266179869 +0000 UTC m=+0.453984403 container attach 027fa61a5c06bb7c8e6f3c12e157bc3cda8de0c82818a48e673002bd916731eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.35.3, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-3f51ea3, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True) 2024-06-11T14:05:10.409 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 14:05:10 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-5[43766]: 2024-06-11T14:05:10.204+0000 7f698cf5f640 -1 osd.5 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:34.389374+0000 front 2024-06-11T14:04:34.389498+0000 (oldest deadline 2024-06-11T14:04:57.889373+0000) 2024-06-11T14:05:10.409 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 14:05:10 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-5[43766]: 2024-06-11T14:05:10.204+0000 7f698cf5f640 -1 osd.5 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:43.190370+0000 front 2024-06-11T14:04:43.190303+0000 (oldest deadline 2024-06-11T14:05:07.290390+0000) 2024-06-11T14:05:10.409 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:10 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:10.235+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:31.703677+0000 front 2024-06-11T14:04:31.703581+0000 (oldest deadline 2024-06-11T14:04:57.603720+0000) 2024-06-11T14:05:10.409 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:10 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:10.235+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:44.005630+0000 front 2024-06-11T14:04:44.005621+0000 (oldest deadline 2024-06-11T14:05:09.905355+0000) 2024-06-11T14:05:10.752 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 11 14:05:10 smithi107 podman[75436]: 2024-06-11 14:05:10.410033883 +0000 UTC m=+0.597838413 container died 027fa61a5c06bb7c8e6f3c12e157bc3cda8de0c82818a48e673002bd916731eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, io.buildah.version=1.35.3, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-3f51ea3, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-06-11T14:05:11.003 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@osd.4.service' 2024-06-11T14:05:11.004 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:10 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:10.929+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:33.012331+0000 front 2024-06-11T14:04:33.012304+0000 (oldest deadline 2024-06-11T14:04:58.312138+0000) 2024-06-11T14:05:11.004 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:10 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:10.929+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:43.613326+0000 front 2024-06-11T14:04:43.613130+0000 (oldest deadline 2024-06-11T14:05:07.713132+0000) 2024-06-11T14:05:11.004 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 11 14:05:10 smithi107 podman[75460]: 2024-06-11 14:05:10.977759879 +0000 UTC m=+0.561948210 container remove 027fa61a5c06bb7c8e6f3c12e157bc3cda8de0c82818a48e673002bd916731eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, RELEASE=squid-3f51ea3, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_BRANCH=HEAD, io.buildah.version=1.35.3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-06-11T14:05:11.004 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 11 14:05:11 smithi107 systemd[1]: ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@osd.4.service: Deactivated successfully. 2024-06-11T14:05:11.004 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 11 14:05:11 smithi107 systemd[1]: Stopped Ceph osd.4 for 98a6635e-27f5-11ef-bc9b-c7b262605968. 2024-06-11T14:05:11.004 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 11 14:05:11 smithi107 systemd[1]: ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@osd.4.service: Consumed 57.572s CPU time. 2024-06-11T14:05:11.045 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-11T14:05:11.045 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-06-11T14:05:11.045 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-06-11T14:05:11.045 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@osd.5 2024-06-11T14:05:11.273 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 14:05:11 smithi107 systemd[1]: Stopping Ceph osd.5 for 98a6635e-27f5-11ef-bc9b-c7b262605968... 2024-06-11T14:05:11.273 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 14:05:11 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-5[43766]: 2024-06-11T14:05:11.162+0000 7f698cf5f640 -1 osd.5 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:34.389374+0000 front 2024-06-11T14:04:34.389498+0000 (oldest deadline 2024-06-11T14:04:57.889373+0000) 2024-06-11T14:05:11.273 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 14:05:11 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-5[43766]: 2024-06-11T14:05:11.162+0000 7f698cf5f640 -1 osd.5 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:43.190370+0000 front 2024-06-11T14:04:43.190303+0000 (oldest deadline 2024-06-11T14:05:07.290390+0000) 2024-06-11T14:05:11.273 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:11 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:11.186+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:31.703677+0000 front 2024-06-11T14:04:31.703581+0000 (oldest deadline 2024-06-11T14:04:57.603720+0000) 2024-06-11T14:05:11.273 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:11 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:11.186+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:44.005630+0000 front 2024-06-11T14:04:44.005621+0000 (oldest deadline 2024-06-11T14:05:09.905355+0000) 2024-06-11T14:05:11.550 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 14:05:11 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-5[43766]: 2024-06-11T14:05:11.389+0000 7f6994951640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-11T14:05:11.550 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 14:05:11 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-5[43766]: 2024-06-11T14:05:11.389+0000 7f6994951640 -1 osd.5 804 *** Got signal Terminated *** 2024-06-11T14:05:11.550 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 14:05:11 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-5[43766]: 2024-06-11T14:05:11.389+0000 7f6994951640 -1 osd.5 804 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-11T14:05:12.223 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:11 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:11.971+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:33.012331+0000 front 2024-06-11T14:04:33.012304+0000 (oldest deadline 2024-06-11T14:04:58.312138+0000) 2024-06-11T14:05:12.223 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:11 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:11.971+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:43.613326+0000 front 2024-06-11T14:04:43.613130+0000 (oldest deadline 2024-06-11T14:05:07.713132+0000) 2024-06-11T14:05:12.223 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:12 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:12.202+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:31.703677+0000 front 2024-06-11T14:04:31.703581+0000 (oldest deadline 2024-06-11T14:04:57.603720+0000) 2024-06-11T14:05:12.224 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:12 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:12.202+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:44.005630+0000 front 2024-06-11T14:04:44.005621+0000 (oldest deadline 2024-06-11T14:05:09.905355+0000) 2024-06-11T14:05:12.224 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 14:05:12 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-5[43766]: 2024-06-11T14:05:12.189+0000 7f698cf5f640 -1 osd.5 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:34.389374+0000 front 2024-06-11T14:04:34.389498+0000 (oldest deadline 2024-06-11T14:04:57.889373+0000) 2024-06-11T14:05:12.224 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 14:05:12 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-5[43766]: 2024-06-11T14:05:12.189+0000 7f698cf5f640 -1 osd.5 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:43.190370+0000 front 2024-06-11T14:04:43.190303+0000 (oldest deadline 2024-06-11T14:05:07.290390+0000) 2024-06-11T14:05:13.212 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:12 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:12.933+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:33.012331+0000 front 2024-06-11T14:04:33.012304+0000 (oldest deadline 2024-06-11T14:04:58.312138+0000) 2024-06-11T14:05:13.213 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:12 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:12.933+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:43.613326+0000 front 2024-06-11T14:04:43.613130+0000 (oldest deadline 2024-06-11T14:05:07.713132+0000) 2024-06-11T14:05:13.213 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:13 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:13.157+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:31.703677+0000 front 2024-06-11T14:04:31.703581+0000 (oldest deadline 2024-06-11T14:04:57.603720+0000) 2024-06-11T14:05:13.213 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:13 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:13.157+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:44.005630+0000 front 2024-06-11T14:04:44.005621+0000 (oldest deadline 2024-06-11T14:05:09.905355+0000) 2024-06-11T14:05:13.550 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 14:05:13 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-5[43766]: 2024-06-11T14:05:13.212+0000 7f698cf5f640 -1 osd.5 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:34.389374+0000 front 2024-06-11T14:04:34.389498+0000 (oldest deadline 2024-06-11T14:04:57.889373+0000) 2024-06-11T14:05:13.550 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 14:05:13 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-5[43766]: 2024-06-11T14:05:13.212+0000 7f698cf5f640 -1 osd.5 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:43.190370+0000 front 2024-06-11T14:04:43.190303+0000 (oldest deadline 2024-06-11T14:05:07.290390+0000) 2024-06-11T14:05:14.184 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:13 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:13.894+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:33.012331+0000 front 2024-06-11T14:04:33.012304+0000 (oldest deadline 2024-06-11T14:04:58.312138+0000) 2024-06-11T14:05:14.184 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:13 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:13.894+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:43.613326+0000 front 2024-06-11T14:04:43.613130+0000 (oldest deadline 2024-06-11T14:05:07.713132+0000) 2024-06-11T14:05:14.549 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 14:05:14 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-5[43766]: 2024-06-11T14:05:14.228+0000 7f698cf5f640 -1 osd.5 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:34.389374+0000 front 2024-06-11T14:04:34.389498+0000 (oldest deadline 2024-06-11T14:04:57.889373+0000) 2024-06-11T14:05:14.550 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 14:05:14 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-5[43766]: 2024-06-11T14:05:14.228+0000 7f698cf5f640 -1 osd.5 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:43.190370+0000 front 2024-06-11T14:04:43.190303+0000 (oldest deadline 2024-06-11T14:05:07.290390+0000) 2024-06-11T14:05:14.550 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:14 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:14.184+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:31.703677+0000 front 2024-06-11T14:04:31.703581+0000 (oldest deadline 2024-06-11T14:04:57.603720+0000) 2024-06-11T14:05:14.550 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:14 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:14.184+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:44.005630+0000 front 2024-06-11T14:04:44.005621+0000 (oldest deadline 2024-06-11T14:05:09.905355+0000) 2024-06-11T14:05:14.550 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:14 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:14.184+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6811 osd.2 since back 2024-06-11T14:04:49.905680+0000 front 2024-06-11T14:04:49.905830+0000 (oldest deadline 2024-06-11T14:05:14.005760+0000) 2024-06-11T14:05:15.194 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:14 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:14.845+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:33.012331+0000 front 2024-06-11T14:04:33.012304+0000 (oldest deadline 2024-06-11T14:04:58.312138+0000) 2024-06-11T14:05:15.194 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:14 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:14.845+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:43.613326+0000 front 2024-06-11T14:04:43.613130+0000 (oldest deadline 2024-06-11T14:05:07.713132+0000) 2024-06-11T14:05:15.550 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 14:05:15 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-5[43766]: 2024-06-11T14:05:15.223+0000 7f698cf5f640 -1 osd.5 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:34.389374+0000 front 2024-06-11T14:04:34.389498+0000 (oldest deadline 2024-06-11T14:04:57.889373+0000) 2024-06-11T14:05:15.550 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 14:05:15 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-5[43766]: 2024-06-11T14:05:15.223+0000 7f698cf5f640 -1 osd.5 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:43.190370+0000 front 2024-06-11T14:04:43.190303+0000 (oldest deadline 2024-06-11T14:05:07.290390+0000) 2024-06-11T14:05:15.550 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 14:05:15 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-5[43766]: 2024-06-11T14:05:15.223+0000 7f698cf5f640 -1 osd.5 804 heartbeat_check: no reply from 172.21.15.57:6811 osd.2 since back 2024-06-11T14:04:50.191610+0000 front 2024-06-11T14:04:50.191553+0000 (oldest deadline 2024-06-11T14:05:14.891591+0000) 2024-06-11T14:05:15.550 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:15 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:15.194+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:31.703677+0000 front 2024-06-11T14:04:31.703581+0000 (oldest deadline 2024-06-11T14:04:57.603720+0000) 2024-06-11T14:05:15.550 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:15 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:15.194+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:44.005630+0000 front 2024-06-11T14:04:44.005621+0000 (oldest deadline 2024-06-11T14:05:09.905355+0000) 2024-06-11T14:05:15.550 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:15 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:15.194+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6811 osd.2 since back 2024-06-11T14:04:49.905680+0000 front 2024-06-11T14:04:49.905830+0000 (oldest deadline 2024-06-11T14:05:14.005760+0000) 2024-06-11T14:05:16.171 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:15 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:15.893+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:33.012331+0000 front 2024-06-11T14:04:33.012304+0000 (oldest deadline 2024-06-11T14:04:58.312138+0000) 2024-06-11T14:05:16.172 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:15 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:15.893+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:43.613326+0000 front 2024-06-11T14:04:43.613130+0000 (oldest deadline 2024-06-11T14:05:07.713132+0000) 2024-06-11T14:05:16.172 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:15 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:15.893+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6811 osd.2 since back 2024-06-11T14:04:51.214161+0000 front 2024-06-11T14:04:51.214228+0000 (oldest deadline 2024-06-11T14:05:15.314112+0000) 2024-06-11T14:05:16.550 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:16 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:16.172+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:31.703677+0000 front 2024-06-11T14:04:31.703581+0000 (oldest deadline 2024-06-11T14:04:57.603720+0000) 2024-06-11T14:05:16.550 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:16 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:16.172+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:44.005630+0000 front 2024-06-11T14:04:44.005621+0000 (oldest deadline 2024-06-11T14:05:09.905355+0000) 2024-06-11T14:05:16.550 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:16 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:16.172+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6811 osd.2 since back 2024-06-11T14:04:49.905680+0000 front 2024-06-11T14:04:49.905830+0000 (oldest deadline 2024-06-11T14:05:14.005760+0000) 2024-06-11T14:05:16.550 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 14:05:16 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-5[43766]: 2024-06-11T14:05:16.257+0000 7f698cf5f640 -1 osd.5 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:34.389374+0000 front 2024-06-11T14:04:34.389498+0000 (oldest deadline 2024-06-11T14:04:57.889373+0000) 2024-06-11T14:05:16.551 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 14:05:16 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-5[43766]: 2024-06-11T14:05:16.257+0000 7f698cf5f640 -1 osd.5 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:43.190370+0000 front 2024-06-11T14:04:43.190303+0000 (oldest deadline 2024-06-11T14:05:07.290390+0000) 2024-06-11T14:05:16.551 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 14:05:16 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-5[43766]: 2024-06-11T14:05:16.257+0000 7f698cf5f640 -1 osd.5 804 heartbeat_check: no reply from 172.21.15.57:6811 osd.2 since back 2024-06-11T14:04:50.191610+0000 front 2024-06-11T14:04:50.191553+0000 (oldest deadline 2024-06-11T14:05:14.891591+0000) 2024-06-11T14:05:17.140 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 14:05:16 smithi107 podman[75540]: 2024-06-11 14:05:16.866705587 +0000 UTC m=+5.584325841 container died 40295b3eb88cf8c5b56ee6d77e0d7fcc64ec90039a7c952dab57c9bcebe49ba4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.35.3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-3f51ea3) 2024-06-11T14:05:17.140 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:16 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:16.899+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:33.012331+0000 front 2024-06-11T14:04:33.012304+0000 (oldest deadline 2024-06-11T14:04:58.312138+0000) 2024-06-11T14:05:17.140 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:16 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:16.899+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:43.613326+0000 front 2024-06-11T14:04:43.613130+0000 (oldest deadline 2024-06-11T14:05:07.713132+0000) 2024-06-11T14:05:17.140 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:16 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:16.899+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6811 osd.2 since back 2024-06-11T14:04:51.214161+0000 front 2024-06-11T14:04:51.214228+0000 (oldest deadline 2024-06-11T14:05:15.314112+0000) 2024-06-11T14:05:17.488 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 14:05:17 smithi107 podman[75540]: 2024-06-11 14:05:17.248007747 +0000 UTC m=+5.965627995 container cleanup 40295b3eb88cf8c5b56ee6d77e0d7fcc64ec90039a7c952dab57c9bcebe49ba4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-3f51ea3, org.label-schema.build-date=20240528, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.35.3, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-06-11T14:05:17.489 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 14:05:17 smithi107 bash[75540]: ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-5 2024-06-11T14:05:17.489 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:17 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:17.150+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:31.703677+0000 front 2024-06-11T14:04:31.703581+0000 (oldest deadline 2024-06-11T14:04:57.603720+0000) 2024-06-11T14:05:17.489 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:17 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:17.150+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:44.005630+0000 front 2024-06-11T14:04:44.005621+0000 (oldest deadline 2024-06-11T14:05:09.905355+0000) 2024-06-11T14:05:17.489 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:17 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:17.150+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6811 osd.2 since back 2024-06-11T14:04:49.905680+0000 front 2024-06-11T14:04:49.905830+0000 (oldest deadline 2024-06-11T14:05:14.005760+0000) 2024-06-11T14:05:17.800 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 14:05:17 smithi107 podman[75566]: 2024-06-11 14:05:17.489824309 +0000 UTC m=+0.623373719 container remove 40295b3eb88cf8c5b56ee6d77e0d7fcc64ec90039a7c952dab57c9bcebe49ba4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-5, GIT_CLEAN=True, io.buildah.version=1.35.3, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240528, RELEASE=squid-3f51ea3, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-06-11T14:05:18.092 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:17 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:17.852+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:33.012331+0000 front 2024-06-11T14:04:33.012304+0000 (oldest deadline 2024-06-11T14:04:58.312138+0000) 2024-06-11T14:05:18.092 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:17 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:17.852+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:43.613326+0000 front 2024-06-11T14:04:43.613130+0000 (oldest deadline 2024-06-11T14:05:07.713132+0000) 2024-06-11T14:05:18.092 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:17 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:17.852+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6811 osd.2 since back 2024-06-11T14:04:51.214161+0000 front 2024-06-11T14:04:51.214228+0000 (oldest deadline 2024-06-11T14:05:15.314112+0000) 2024-06-11T14:05:18.093 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 14:05:17 smithi107 podman[75655]: 2024-06-11 14:05:17.726241648 +0000 UTC m=+0.017877509 image pull 2d7459ec65a098c35c774154617f2893b0ad051f56ce89b72ab795f65a97fc22 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310 2024-06-11T14:05:18.093 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 14:05:17 smithi107 podman[75655]: 2024-06-11 14:05:17.849174178 +0000 UTC m=+0.140810059 container create f76e48e87e63379d779f983f9d6da48af33c138ac4ba10b44193b66d0244e23e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3f51ea3, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.35.3, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD) 2024-06-11T14:05:18.093 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 14:05:18 smithi107 podman[75655]: 2024-06-11 14:05:18.09319122 +0000 UTC m=+0.384827071 container init f76e48e87e63379d779f983f9d6da48af33c138ac4ba10b44193b66d0244e23e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-3f51ea3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.35.3, org.label-schema.name=CentOS Stream 9 Base Image) 2024-06-11T14:05:18.356 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:18 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:18.136+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:31.703677+0000 front 2024-06-11T14:04:31.703581+0000 (oldest deadline 2024-06-11T14:04:57.603720+0000) 2024-06-11T14:05:18.356 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:18 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:18.136+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:44.005630+0000 front 2024-06-11T14:04:44.005621+0000 (oldest deadline 2024-06-11T14:05:09.905355+0000) 2024-06-11T14:05:18.356 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:18 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:18.136+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6811 osd.2 since back 2024-06-11T14:04:49.905680+0000 front 2024-06-11T14:04:49.905830+0000 (oldest deadline 2024-06-11T14:05:14.005760+0000) 2024-06-11T14:05:18.356 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 14:05:18 smithi107 podman[75655]: 2024-06-11 14:05:18.096744846 +0000 UTC m=+0.388380692 container start f76e48e87e63379d779f983f9d6da48af33c138ac4ba10b44193b66d0244e23e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3f51ea3, ceph=True, io.buildah.version=1.35.3, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-06-11T14:05:18.356 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 14:05:18 smithi107 podman[75655]: 2024-06-11 14:05:18.159662437 +0000 UTC m=+0.451298292 container attach f76e48e87e63379d779f983f9d6da48af33c138ac4ba10b44193b66d0244e23e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-5-deactivate, RELEASE=squid-3f51ea3, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.35.3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True) 2024-06-11T14:05:18.620 INFO:journalctl@ceph.osd.5.smithi107.stdout:Jun 11 14:05:18 smithi107 podman[75655]: 2024-06-11 14:05:18.357551886 +0000 UTC m=+0.649187733 container died f76e48e87e63379d779f983f9d6da48af33c138ac4ba10b44193b66d0244e23e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.35.3, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, RELEASE=squid-3f51ea3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-06-11T14:05:18.887 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:18 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:18.855+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:33.012331+0000 front 2024-06-11T14:04:33.012304+0000 (oldest deadline 2024-06-11T14:04:58.312138+0000) 2024-06-11T14:05:18.887 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:18 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:18.855+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:43.613326+0000 front 2024-06-11T14:04:43.613130+0000 (oldest deadline 2024-06-11T14:05:07.713132+0000) 2024-06-11T14:05:18.887 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:18 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:18.855+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6811 osd.2 since back 2024-06-11T14:04:51.214161+0000 front 2024-06-11T14:04:51.214228+0000 (oldest deadline 2024-06-11T14:05:15.314112+0000) 2024-06-11T14:05:18.907 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@osd.5.service' 2024-06-11T14:05:18.945 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-11T14:05:18.946 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-06-11T14:05:18.946 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-06-11T14:05:18.946 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@osd.6 2024-06-11T14:05:19.183 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:19 smithi107 systemd[1]: Stopping Ceph osd.6 for 98a6635e-27f5-11ef-bc9b-c7b262605968... 2024-06-11T14:05:19.550 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:19 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:19.183+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:31.703677+0000 front 2024-06-11T14:04:31.703581+0000 (oldest deadline 2024-06-11T14:04:57.603720+0000) 2024-06-11T14:05:19.550 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:19 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:19.183+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:44.005630+0000 front 2024-06-11T14:04:44.005621+0000 (oldest deadline 2024-06-11T14:05:09.905355+0000) 2024-06-11T14:05:19.550 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:19 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:19.183+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6811 osd.2 since back 2024-06-11T14:04:49.905680+0000 front 2024-06-11T14:04:49.905830+0000 (oldest deadline 2024-06-11T14:05:14.005760+0000) 2024-06-11T14:05:19.550 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:19 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:19.299+0000 7f9db2e4e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-11T14:05:19.550 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:19 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:19.299+0000 7f9db2e4e640 -1 osd.6 804 *** Got signal Terminated *** 2024-06-11T14:05:19.550 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:19 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:19.299+0000 7f9db2e4e640 -1 osd.6 804 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-11T14:05:20.141 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:19 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:19.885+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:33.012331+0000 front 2024-06-11T14:04:33.012304+0000 (oldest deadline 2024-06-11T14:04:58.312138+0000) 2024-06-11T14:05:20.141 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:19 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:19.885+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:43.613326+0000 front 2024-06-11T14:04:43.613130+0000 (oldest deadline 2024-06-11T14:05:07.713132+0000) 2024-06-11T14:05:20.141 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:19 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:19.885+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6811 osd.2 since back 2024-06-11T14:04:51.214161+0000 front 2024-06-11T14:04:51.214228+0000 (oldest deadline 2024-06-11T14:05:15.314112+0000) 2024-06-11T14:05:20.550 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:20 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:20.140+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:31.703677+0000 front 2024-06-11T14:04:31.703581+0000 (oldest deadline 2024-06-11T14:04:57.603720+0000) 2024-06-11T14:05:20.550 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:20 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:20.140+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:44.005630+0000 front 2024-06-11T14:04:44.005621+0000 (oldest deadline 2024-06-11T14:05:09.905355+0000) 2024-06-11T14:05:20.550 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:20 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:20.140+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6811 osd.2 since back 2024-06-11T14:04:49.905680+0000 front 2024-06-11T14:04:49.905830+0000 (oldest deadline 2024-06-11T14:05:14.005760+0000) 2024-06-11T14:05:21.164 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:20 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:20.910+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:33.012331+0000 front 2024-06-11T14:04:33.012304+0000 (oldest deadline 2024-06-11T14:04:58.312138+0000) 2024-06-11T14:05:21.164 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:20 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:20.910+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:43.613326+0000 front 2024-06-11T14:04:43.613130+0000 (oldest deadline 2024-06-11T14:05:07.713132+0000) 2024-06-11T14:05:21.164 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:20 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:20.910+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6811 osd.2 since back 2024-06-11T14:04:51.214161+0000 front 2024-06-11T14:04:51.214228+0000 (oldest deadline 2024-06-11T14:05:15.314112+0000) 2024-06-11T14:05:21.550 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:21 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:21.165+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:31.703677+0000 front 2024-06-11T14:04:31.703581+0000 (oldest deadline 2024-06-11T14:04:57.603720+0000) 2024-06-11T14:05:21.550 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:21 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:21.165+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:44.005630+0000 front 2024-06-11T14:04:44.005621+0000 (oldest deadline 2024-06-11T14:05:09.905355+0000) 2024-06-11T14:05:21.550 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:21 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:21.165+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6811 osd.2 since back 2024-06-11T14:04:49.905680+0000 front 2024-06-11T14:04:49.905830+0000 (oldest deadline 2024-06-11T14:05:14.005760+0000) 2024-06-11T14:05:22.050 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:21 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:21.906+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:33.012331+0000 front 2024-06-11T14:04:33.012304+0000 (oldest deadline 2024-06-11T14:04:58.312138+0000) 2024-06-11T14:05:22.050 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:21 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:21.906+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:43.613326+0000 front 2024-06-11T14:04:43.613130+0000 (oldest deadline 2024-06-11T14:05:07.713132+0000) 2024-06-11T14:05:22.050 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:21 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:21.906+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6811 osd.2 since back 2024-06-11T14:04:51.214161+0000 front 2024-06-11T14:04:51.214228+0000 (oldest deadline 2024-06-11T14:05:15.314112+0000) 2024-06-11T14:05:22.550 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:22 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:22.156+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:31.703677+0000 front 2024-06-11T14:04:31.703581+0000 (oldest deadline 2024-06-11T14:04:57.603720+0000) 2024-06-11T14:05:22.550 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:22 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:22.156+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:44.005630+0000 front 2024-06-11T14:04:44.005621+0000 (oldest deadline 2024-06-11T14:05:09.905355+0000) 2024-06-11T14:05:22.550 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:22 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:22.156+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6811 osd.2 since back 2024-06-11T14:04:49.905680+0000 front 2024-06-11T14:04:49.905830+0000 (oldest deadline 2024-06-11T14:05:14.005760+0000) 2024-06-11T14:05:23.300 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:22 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:22.897+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:33.012331+0000 front 2024-06-11T14:04:33.012304+0000 (oldest deadline 2024-06-11T14:04:58.312138+0000) 2024-06-11T14:05:23.300 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:22 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:22.897+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:43.613326+0000 front 2024-06-11T14:04:43.613130+0000 (oldest deadline 2024-06-11T14:05:07.713132+0000) 2024-06-11T14:05:23.300 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:22 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:22.897+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6811 osd.2 since back 2024-06-11T14:04:51.214161+0000 front 2024-06-11T14:04:51.214228+0000 (oldest deadline 2024-06-11T14:05:15.314112+0000) 2024-06-11T14:05:23.300 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:22 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:22.897+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6815 osd.3 since back 2024-06-11T14:04:58.215197+0000 front 2024-06-11T14:04:58.215135+0000 (oldest deadline 2024-06-11T14:05:22.315030+0000) 2024-06-11T14:05:23.300 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:23 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:23.114+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:31.703677+0000 front 2024-06-11T14:04:31.703581+0000 (oldest deadline 2024-06-11T14:04:57.603720+0000) 2024-06-11T14:05:23.300 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:23 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:23.114+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:44.005630+0000 front 2024-06-11T14:04:44.005621+0000 (oldest deadline 2024-06-11T14:05:09.905355+0000) 2024-06-11T14:05:23.300 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:23 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:23.114+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6811 osd.2 since back 2024-06-11T14:04:49.905680+0000 front 2024-06-11T14:04:49.905830+0000 (oldest deadline 2024-06-11T14:05:14.005760+0000) 2024-06-11T14:05:24.300 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:23 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:23.866+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:33.012331+0000 front 2024-06-11T14:04:33.012304+0000 (oldest deadline 2024-06-11T14:04:58.312138+0000) 2024-06-11T14:05:24.301 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:23 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:23.866+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:43.613326+0000 front 2024-06-11T14:04:43.613130+0000 (oldest deadline 2024-06-11T14:05:07.713132+0000) 2024-06-11T14:05:24.301 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:23 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:23.866+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6811 osd.2 since back 2024-06-11T14:04:51.214161+0000 front 2024-06-11T14:04:51.214228+0000 (oldest deadline 2024-06-11T14:05:15.314112+0000) 2024-06-11T14:05:24.301 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:23 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6[49278]: 2024-06-11T14:05:23.866+0000 7f9dab45c640 -1 osd.6 804 heartbeat_check: no reply from 172.21.15.57:6815 osd.3 since back 2024-06-11T14:04:58.215197+0000 front 2024-06-11T14:04:58.215135+0000 (oldest deadline 2024-06-11T14:05:22.315030+0000) 2024-06-11T14:05:24.301 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:24 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:24.069+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:31.703677+0000 front 2024-06-11T14:04:31.703581+0000 (oldest deadline 2024-06-11T14:04:57.603720+0000) 2024-06-11T14:05:24.301 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:24 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:24.069+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:44.005630+0000 front 2024-06-11T14:04:44.005621+0000 (oldest deadline 2024-06-11T14:05:09.905355+0000) 2024-06-11T14:05:24.301 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:24 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:24.069+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6811 osd.2 since back 2024-06-11T14:04:49.905680+0000 front 2024-06-11T14:04:49.905830+0000 (oldest deadline 2024-06-11T14:05:14.005760+0000) 2024-06-11T14:05:25.015 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:24 smithi107 podman[75758]: 2024-06-11 14:05:24.73710671 +0000 UTC m=+5.573812804 container died fc39e53c08139beb56dfd10a0bee0bef2dc7508147bc4944688c6d4dcf3ab85f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.35.3, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, RELEASE=squid-3f51ea3, CEPH_POINT_RELEASE=) 2024-06-11T14:05:25.300 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:25 smithi107 podman[75758]: 2024-06-11 14:05:25.106890921 +0000 UTC m=+5.943597015 container cleanup fc39e53c08139beb56dfd10a0bee0bef2dc7508147bc4944688c6d4dcf3ab85f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-3f51ea3, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.35.3) 2024-06-11T14:05:25.300 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:25 smithi107 bash[75758]: ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6 2024-06-11T14:05:25.300 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:25 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:25.062+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:31.703677+0000 front 2024-06-11T14:04:31.703581+0000 (oldest deadline 2024-06-11T14:04:57.603720+0000) 2024-06-11T14:05:25.300 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:25 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:25.062+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:44.005630+0000 front 2024-06-11T14:04:44.005621+0000 (oldest deadline 2024-06-11T14:05:09.905355+0000) 2024-06-11T14:05:25.300 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:25 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:25.062+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6811 osd.2 since back 2024-06-11T14:04:49.905680+0000 front 2024-06-11T14:04:49.905830+0000 (oldest deadline 2024-06-11T14:05:14.005760+0000) 2024-06-11T14:05:25.300 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:25 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:25.062+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6815 osd.3 since back 2024-06-11T14:05:00.307528+0000 front 2024-06-11T14:05:00.307542+0000 (oldest deadline 2024-06-11T14:05:25.007406+0000) 2024-06-11T14:05:25.708 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:25 smithi107 podman[75773]: 2024-06-11 14:05:25.400004369 +0000 UTC m=+0.660516456 container remove fc39e53c08139beb56dfd10a0bee0bef2dc7508147bc4944688c6d4dcf3ab85f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.35.3, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-3f51ea3, ceph=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-06-11T14:05:25.993 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:25 smithi107 podman[75861]: 2024-06-11 14:05:25.708945828 +0000 UTC m=+0.084235877 container create c00df5c0c9e40a4ca65882106ce485db53ba004161058838a4deb6456a309557 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6-deactivate, io.buildah.version=1.35.3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-3f51ea3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux ) 2024-06-11T14:05:25.993 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:25 smithi107 podman[75861]: 2024-06-11 14:05:25.6421883 +0000 UTC m=+0.017478351 image pull 2d7459ec65a098c35c774154617f2893b0ad051f56ce89b72ab795f65a97fc22 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310 2024-06-11T14:05:25.994 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:25 smithi107 podman[75861]: 2024-06-11 14:05:25.911611058 +0000 UTC m=+0.286901089 container init c00df5c0c9e40a4ca65882106ce485db53ba004161058838a4deb6456a309557 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-3f51ea3, io.buildah.version=1.35.3, GIT_CLEAN=True) 2024-06-11T14:05:25.994 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:25 smithi107 podman[75861]: 2024-06-11 14:05:25.915111792 +0000 UTC m=+0.290401833 container start c00df5c0c9e40a4ca65882106ce485db53ba004161058838a4deb6456a309557 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6-deactivate, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.35.3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-3f51ea3, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-11T14:05:25.994 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:25 smithi107 podman[75861]: 2024-06-11 14:05:25.994659126 +0000 UTC m=+0.369949201 container attach c00df5c0c9e40a4ca65882106ce485db53ba004161058838a4deb6456a309557 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6-deactivate, ceph=True, RELEASE=squid-3f51ea3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_BRANCH=HEAD, io.buildah.version=1.35.3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-11T14:05:26.300 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jun 11 14:05:26 smithi107 podman[75861]: 2024-06-11 14:05:26.17808085 +0000 UTC m=+0.553370899 container died c00df5c0c9e40a4ca65882106ce485db53ba004161058838a4deb6456a309557 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-3f51ea3, org.label-schema.build-date=20240528, io.buildah.version=1.35.3, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-06-11T14:05:26.300 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:26 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:26.047+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:31.703677+0000 front 2024-06-11T14:04:31.703581+0000 (oldest deadline 2024-06-11T14:04:57.603720+0000) 2024-06-11T14:05:26.300 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:26 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:26.047+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:44.005630+0000 front 2024-06-11T14:04:44.005621+0000 (oldest deadline 2024-06-11T14:05:09.905355+0000) 2024-06-11T14:05:26.300 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:26 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:26.047+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6811 osd.2 since back 2024-06-11T14:04:49.905680+0000 front 2024-06-11T14:04:49.905830+0000 (oldest deadline 2024-06-11T14:05:14.005760+0000) 2024-06-11T14:05:26.300 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:26 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:26.047+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6815 osd.3 since back 2024-06-11T14:05:00.307528+0000 front 2024-06-11T14:05:00.307542+0000 (oldest deadline 2024-06-11T14:05:25.007406+0000) 2024-06-11T14:05:26.802 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@osd.6.service' 2024-06-11T14:05:26.841 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-11T14:05:26.841 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-06-11T14:05:26.842 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-06-11T14:05:26.842 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@osd.7 2024-06-11T14:05:27.031 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:26 smithi107 systemd[1]: Stopping Ceph osd.7 for 98a6635e-27f5-11ef-bc9b-c7b262605968... 2024-06-11T14:05:27.300 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:27 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:27.031+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:31.703677+0000 front 2024-06-11T14:04:31.703581+0000 (oldest deadline 2024-06-11T14:04:57.603720+0000) 2024-06-11T14:05:27.300 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:27 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:27.031+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:44.005630+0000 front 2024-06-11T14:04:44.005621+0000 (oldest deadline 2024-06-11T14:05:09.905355+0000) 2024-06-11T14:05:27.300 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:27 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:27.031+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6811 osd.2 since back 2024-06-11T14:04:49.905680+0000 front 2024-06-11T14:04:49.905830+0000 (oldest deadline 2024-06-11T14:05:14.005760+0000) 2024-06-11T14:05:27.300 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:27 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:27.031+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6815 osd.3 since back 2024-06-11T14:05:00.307528+0000 front 2024-06-11T14:05:00.307542+0000 (oldest deadline 2024-06-11T14:05:25.007406+0000) 2024-06-11T14:05:27.300 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:27 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:27.159+0000 7f31bbe0a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-11T14:05:27.300 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:27 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:27.159+0000 7f31bbe0a640 -1 osd.7 804 *** Got signal Terminated *** 2024-06-11T14:05:27.300 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:27 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:27.159+0000 7f31bbe0a640 -1 osd.7 804 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-11T14:05:28.300 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:27 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:27.991+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:31.703677+0000 front 2024-06-11T14:04:31.703581+0000 (oldest deadline 2024-06-11T14:04:57.603720+0000) 2024-06-11T14:05:28.300 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:27 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:27.991+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:44.005630+0000 front 2024-06-11T14:04:44.005621+0000 (oldest deadline 2024-06-11T14:05:09.905355+0000) 2024-06-11T14:05:28.300 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:27 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:27.991+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6811 osd.2 since back 2024-06-11T14:04:49.905680+0000 front 2024-06-11T14:04:49.905830+0000 (oldest deadline 2024-06-11T14:05:14.005760+0000) 2024-06-11T14:05:28.300 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:27 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:27.991+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6815 osd.3 since back 2024-06-11T14:05:00.307528+0000 front 2024-06-11T14:05:00.307542+0000 (oldest deadline 2024-06-11T14:05:25.007406+0000) 2024-06-11T14:05:29.300 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:29 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:29.003+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:31.703677+0000 front 2024-06-11T14:04:31.703581+0000 (oldest deadline 2024-06-11T14:04:57.603720+0000) 2024-06-11T14:05:29.300 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:29 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:29.003+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:44.005630+0000 front 2024-06-11T14:04:44.005621+0000 (oldest deadline 2024-06-11T14:05:09.905355+0000) 2024-06-11T14:05:29.300 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:29 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:29.003+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6811 osd.2 since back 2024-06-11T14:04:49.905680+0000 front 2024-06-11T14:04:49.905830+0000 (oldest deadline 2024-06-11T14:05:14.005760+0000) 2024-06-11T14:05:29.300 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:29 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:29.003+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6815 osd.3 since back 2024-06-11T14:05:00.307528+0000 front 2024-06-11T14:05:00.307542+0000 (oldest deadline 2024-06-11T14:05:25.007406+0000) 2024-06-11T14:05:30.300 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:29 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:29.993+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:31.703677+0000 front 2024-06-11T14:04:31.703581+0000 (oldest deadline 2024-06-11T14:04:57.603720+0000) 2024-06-11T14:05:30.300 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:29 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:29.993+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:44.005630+0000 front 2024-06-11T14:04:44.005621+0000 (oldest deadline 2024-06-11T14:05:09.905355+0000) 2024-06-11T14:05:30.300 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:29 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:29.993+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6811 osd.2 since back 2024-06-11T14:04:49.905680+0000 front 2024-06-11T14:04:49.905830+0000 (oldest deadline 2024-06-11T14:05:14.005760+0000) 2024-06-11T14:05:30.300 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:29 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:29.993+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6815 osd.3 since back 2024-06-11T14:05:00.307528+0000 front 2024-06-11T14:05:00.307542+0000 (oldest deadline 2024-06-11T14:05:25.007406+0000) 2024-06-11T14:05:31.300 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:31 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:31.024+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:31.703677+0000 front 2024-06-11T14:04:31.703581+0000 (oldest deadline 2024-06-11T14:04:57.603720+0000) 2024-06-11T14:05:31.300 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:31 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:31.024+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:44.005630+0000 front 2024-06-11T14:04:44.005621+0000 (oldest deadline 2024-06-11T14:05:09.905355+0000) 2024-06-11T14:05:31.300 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:31 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:31.024+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6811 osd.2 since back 2024-06-11T14:04:49.905680+0000 front 2024-06-11T14:04:49.905830+0000 (oldest deadline 2024-06-11T14:05:14.005760+0000) 2024-06-11T14:05:31.300 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:31 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:31.024+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6815 osd.3 since back 2024-06-11T14:05:00.307528+0000 front 2024-06-11T14:05:00.307542+0000 (oldest deadline 2024-06-11T14:05:25.007406+0000) 2024-06-11T14:05:31.300 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:31 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:31.024+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.107:6802 osd.4 since back 2024-06-11T14:05:05.007722+0000 front 2024-06-11T14:05:05.007749+0000 (oldest deadline 2024-06-11T14:05:30.307741+0000) 2024-06-11T14:05:32.300 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:31 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:31.983+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6803 osd.0 since back 2024-06-11T14:04:31.703677+0000 front 2024-06-11T14:04:31.703581+0000 (oldest deadline 2024-06-11T14:04:57.603720+0000) 2024-06-11T14:05:32.300 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:31 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:31.983+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6807 osd.1 since back 2024-06-11T14:04:44.005630+0000 front 2024-06-11T14:04:44.005621+0000 (oldest deadline 2024-06-11T14:05:09.905355+0000) 2024-06-11T14:05:32.300 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:31 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:31.983+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6811 osd.2 since back 2024-06-11T14:04:49.905680+0000 front 2024-06-11T14:04:49.905830+0000 (oldest deadline 2024-06-11T14:05:14.005760+0000) 2024-06-11T14:05:32.300 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:31 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:31.983+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.57:6815 osd.3 since back 2024-06-11T14:05:00.307528+0000 front 2024-06-11T14:05:00.307542+0000 (oldest deadline 2024-06-11T14:05:25.007406+0000) 2024-06-11T14:05:32.300 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:31 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7[55265]: 2024-06-11T14:05:31.983+0000 7f31b4418640 -1 osd.7 804 heartbeat_check: no reply from 172.21.15.107:6802 osd.4 since back 2024-06-11T14:05:05.007722+0000 front 2024-06-11T14:05:05.007749+0000 (oldest deadline 2024-06-11T14:05:30.307741+0000) 2024-06-11T14:05:32.966 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:32 smithi107 podman[75962]: 2024-06-11 14:05:32.641224835 +0000 UTC m=+5.572017646 container died 6e65c9b0e97a4e44a75f48cb4576102d36795b5339faac7afe308e0f365b10b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.35.3, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3f51ea3, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True) 2024-06-11T14:05:32.966 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:32 smithi107 podman[75962]: 2024-06-11 14:05:32.966907757 +0000 UTC m=+5.897700567 container cleanup 6e65c9b0e97a4e44a75f48cb4576102d36795b5339faac7afe308e0f365b10b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3f51ea3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, io.buildah.version=1.35.3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-06-11T14:05:33.300 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:32 smithi107 bash[75962]: ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7 2024-06-11T14:05:33.734 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:33 smithi107 podman[75979]: 2024-06-11 14:05:33.309436952 +0000 UTC m=+0.664314014 container remove 6e65c9b0e97a4e44a75f48cb4576102d36795b5339faac7afe308e0f365b10b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7, ceph=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-3f51ea3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.35.3, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.vendor=CentOS) 2024-06-11T14:05:34.019 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:33 smithi107 podman[76066]: 2024-06-11 14:05:33.735075353 +0000 UTC m=+0.103943386 container create 26bd10481a1da196900e5164db2a2eea25053692d851b74414ecc48abfd8ac9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, RELEASE=squid-3f51ea3, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.35.3, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-06-11T14:05:34.019 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:33 smithi107 podman[76066]: 2024-06-11 14:05:33.648488819 +0000 UTC m=+0.017356862 image pull 2d7459ec65a098c35c774154617f2893b0ad051f56ce89b72ab795f65a97fc22 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310 2024-06-11T14:05:34.019 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:33 smithi107 podman[76066]: 2024-06-11 14:05:33.945670522 +0000 UTC m=+0.314538553 container init 26bd10481a1da196900e5164db2a2eea25053692d851b74414ecc48abfd8ac9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.35.3, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , RELEASE=squid-3f51ea3, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528) 2024-06-11T14:05:34.019 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:33 smithi107 podman[76066]: 2024-06-11 14:05:33.949037686 +0000 UTC m=+0.317905719 container start 26bd10481a1da196900e5164db2a2eea25053692d851b74414ecc48abfd8ac9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, RELEASE=squid-3f51ea3, ceph=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.35.3, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-06-11T14:05:34.020 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:34 smithi107 podman[76066]: 2024-06-11 14:05:34.020514291 +0000 UTC m=+0.389382338 container attach 26bd10481a1da196900e5164db2a2eea25053692d851b74414ecc48abfd8ac9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-3f51ea3, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.35.3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True) 2024-06-11T14:05:34.300 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 11 14:05:34 smithi107 podman[76066]: 2024-06-11 14:05:34.200367722 +0000 UTC m=+0.569235755 container died 26bd10481a1da196900e5164db2a2eea25053692d851b74414ecc48abfd8ac9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-osd-7-deactivate, io.buildah.version=1.35.3, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-3f51ea3) 2024-06-11T14:05:34.691 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@osd.7.service' 2024-06-11T14:05:34.730 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-11T14:05:34.730 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-06-11T14:05:34.731 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-06-11T14:05:34.731 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@rgw.foo.a 2024-06-11T14:05:35.037 INFO:journalctl@ceph.rgw.foo.a.smithi057.stdout:Jun 11 14:05:34 smithi057 systemd[1]: Stopping Ceph rgw.foo.a for 98a6635e-27f5-11ef-bc9b-c7b262605968... 2024-06-11T14:05:35.307 INFO:journalctl@ceph.rgw.foo.a.smithi057.stdout:Jun 11 14:05:35 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-rgw-foo-a[67027]: 2024-06-11T14:05:35.058+0000 7fca2b1fd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-11T14:05:35.308 INFO:journalctl@ceph.rgw.foo.a.smithi057.stdout:Jun 11 14:05:35 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-rgw-foo-a[67027]: 2024-06-11T14:05:35.058+0000 7fca2ea69880 -1 shutting down 2024-06-11T14:05:45.370 INFO:journalctl@ceph.rgw.foo.a.smithi057.stdout:Jun 11 14:05:45 smithi057 bash[156701]: time="2024-06-11T14:05:45Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-06-11T14:05:45.370 INFO:journalctl@ceph.rgw.foo.a.smithi057.stdout:Jun 11 14:05:45 smithi057 podman[156701]: 2024-06-11 14:05:45.081651304 +0000 UTC m=+10.138852078 container died 71407d0c32b0175c9467c2a8c84a30521c97013e6a9b04dd478a08445cb3a902 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-rgw-foo-a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.35.3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-3f51ea3, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True) 2024-06-11T14:05:45.704 INFO:journalctl@ceph.rgw.foo.a.smithi057.stdout:Jun 11 14:05:45 smithi057 podman[156701]: 2024-06-11 14:05:45.470330838 +0000 UTC m=+10.527531588 container cleanup 71407d0c32b0175c9467c2a8c84a30521c97013e6a9b04dd478a08445cb3a902 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-rgw-foo-a, RELEASE=squid-3f51ea3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.35.3, org.label-schema.build-date=20240528, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-06-11T14:05:45.704 INFO:journalctl@ceph.rgw.foo.a.smithi057.stdout:Jun 11 14:05:45 smithi057 bash[156701]: ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-rgw-foo-a 2024-06-11T14:05:45.850 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@rgw.foo.a.service' 2024-06-11T14:05:45.888 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-11T14:05:45.889 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-06-11T14:05:45.889 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-06-11T14:05:45.889 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@prometheus.a 2024-06-11T14:05:46.190 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 14:05:45 smithi107 systemd[1]: Stopping Ceph prometheus.a for 98a6635e-27f5-11ef-bc9b-c7b262605968... 2024-06-11T14:05:46.441 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 14:05:46 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[64736]: ts=2024-06-11T14:05:46.191Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-06-11T14:05:46.441 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 14:05:46 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[64736]: ts=2024-06-11T14:05:46.191Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-06-11T14:05:46.441 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 14:05:46 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[64736]: ts=2024-06-11T14:05:46.191Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-06-11T14:05:46.442 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 14:05:46 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[64736]: ts=2024-06-11T14:05:46.191Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-06-11T14:05:46.442 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 14:05:46 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[64736]: ts=2024-06-11T14:05:46.191Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-06-11T14:05:46.442 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 14:05:46 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[64736]: ts=2024-06-11T14:05:46.191Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-06-11T14:05:46.442 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 14:05:46 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[64736]: ts=2024-06-11T14:05:46.191Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-06-11T14:05:46.442 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 14:05:46 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[64736]: ts=2024-06-11T14:05:46.191Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-06-11T14:05:46.442 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 14:05:46 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[64736]: ts=2024-06-11T14:05:46.191Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-06-11T14:05:46.442 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 14:05:46 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[64736]: ts=2024-06-11T14:05:46.192Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-06-11T14:05:46.442 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 14:05:46 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[64736]: ts=2024-06-11T14:05:46.192Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-06-11T14:05:46.442 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 14:05:46 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a[64736]: ts=2024-06-11T14:05:46.192Z caller=main.go:1147 level=info msg="See you next time!" 2024-06-11T14:05:46.442 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 14:05:46 smithi107 podman[76166]: 2024-06-11 14:05:46.201459445 +0000 UTC m=+0.115293220 container died 70cb60b78e5536cae7dce2813a4825ca346ea1f725a10897aeafb773a29ddfdf (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-11T14:05:46.732 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 14:05:46 smithi107 podman[76166]: 2024-06-11 14:05:46.532686063 +0000 UTC m=+0.446519838 container cleanup 70cb60b78e5536cae7dce2813a4825ca346ea1f725a10897aeafb773a29ddfdf (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-11T14:05:46.732 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 14:05:46 smithi107 bash[76166]: ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a 2024-06-11T14:05:46.968 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@prometheus.a.service' 2024-06-11T14:05:46.991 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 14:05:46 smithi107 podman[76179]: 2024-06-11 14:05:46.733577467 +0000 UTC m=+0.533235235 container remove 70cb60b78e5536cae7dce2813a4825ca346ea1f725a10897aeafb773a29ddfdf (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-11T14:05:46.991 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 14:05:46 smithi107 systemd[1]: ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@prometheus.a.service: Deactivated successfully. 2024-06-11T14:05:46.991 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 14:05:46 smithi107 systemd[1]: Stopped Ceph prometheus.a for 98a6635e-27f5-11ef-bc9b-c7b262605968. 2024-06-11T14:05:46.991 INFO:journalctl@ceph.prometheus.a.smithi107.stdout:Jun 11 14:05:46 smithi107 systemd[1]: ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@prometheus.a.service: Consumed 6.993s CPU time. 2024-06-11T14:05:47.006 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-11T14:05:47.006 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-06-11T14:05:47.006 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-11T14:05:47.207 INFO:teuthology.orchestra.run.smithi057.stdout:Deleting cluster with fsid: 98a6635e-27f5-11ef-bc9b-c7b262605968 2024-06-11T14:05:49.125 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 14:05:48 smithi057 systemd[1]: Stopping Ceph alertmanager.a for 98a6635e-27f5-11ef-bc9b-c7b262605968... 2024-06-11T14:05:49.459 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 14:05:49 smithi057 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-alertmanager-a[76524]: ts=2024-06-11T14:05:49.125Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-06-11T14:05:49.459 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 14:05:49 smithi057 podman[157206]: 2024-06-11 14:05:49.166844057 +0000 UTC m=+0.157786601 container died 2e3bcf3ce9613d04cade40d87295ce529e2695669e59764316820f159a119b43 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-11T14:05:49.786 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 14:05:49 smithi057 podman[157206]: 2024-06-11 14:05:49.559643078 +0000 UTC m=+0.550585579 container cleanup 2e3bcf3ce9613d04cade40d87295ce529e2695669e59764316820f159a119b43 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-11T14:05:49.787 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 14:05:49 smithi057 bash[157206]: ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-alertmanager-a 2024-06-11T14:05:50.093 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 14:05:49 smithi057 podman[157220]: 2024-06-11 14:05:49.829090764 +0000 UTC m=+0.654234234 container remove 2e3bcf3ce9613d04cade40d87295ce529e2695669e59764316820f159a119b43 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-11T14:05:50.093 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 14:05:49 smithi057 podman[157220]: 2024-06-11 14:05:49.91157902 +0000 UTC m=+0.736722497 volume remove 2141269c7cae31e8035729c3ce29d08fa3df011245d62474fc2bfc7a7527adaa 2024-06-11T14:05:50.093 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 14:05:50 smithi057 systemd[1]: ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-06-11T14:05:50.093 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 14:05:50 smithi057 systemd[1]: Stopped Ceph alertmanager.a for 98a6635e-27f5-11ef-bc9b-c7b262605968. 2024-06-11T14:05:50.093 INFO:journalctl@ceph.alertmanager.a.smithi057.stdout:Jun 11 14:05:50 smithi057 systemd[1]: ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@alertmanager.a.service: Consumed 2.448s CPU time. 2024-06-11T14:05:50.516 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 14:05:50 smithi057 systemd[1]: Stopping Ceph node-exporter.a for 98a6635e-27f5-11ef-bc9b-c7b262605968... 2024-06-11T14:05:50.780 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 14:05:50 smithi057 podman[157332]: 2024-06-11 14:05:50.516658315 +0000 UTC m=+0.095466691 container died 2a5082206775b601bfb3bab7fc349896df43985a5c73d6ac31a9730a8b387ac1 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-06-11T14:05:51.036 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 14:05:50 smithi057 podman[157332]: 2024-06-11 14:05:50.889454238 +0000 UTC m=+0.468262613 container cleanup 2a5082206775b601bfb3bab7fc349896df43985a5c73d6ac31a9730a8b387ac1 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-06-11T14:05:51.036 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 14:05:50 smithi057 bash[157332]: ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a 2024-06-11T14:05:51.460 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 14:05:51 smithi057 podman[157345]: 2024-06-11 14:05:51.157511842 +0000 UTC m=+0.640857506 container remove 2a5082206775b601bfb3bab7fc349896df43985a5c73d6ac31a9730a8b387ac1 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-06-11T14:05:51.460 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 14:05:51 smithi057 systemd[1]: ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-06-11T14:05:51.460 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 14:05:51 smithi057 systemd[1]: ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@node-exporter.a.service: Failed with result 'exit-code'. 2024-06-11T14:05:51.461 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 14:05:51 smithi057 systemd[1]: Stopped Ceph node-exporter.a for 98a6635e-27f5-11ef-bc9b-c7b262605968. 2024-06-11T14:05:51.461 INFO:journalctl@ceph.node-exporter.a.smithi057.stdout:Jun 11 14:05:51 smithi057 systemd[1]: ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@node-exporter.a.service: Consumed 5.457s CPU time. 2024-06-11T14:05:51.974 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-11T14:05:52.164 INFO:teuthology.orchestra.run.smithi107.stdout:Deleting cluster with fsid: 98a6635e-27f5-11ef-bc9b-c7b262605968 2024-06-11T14:05:53.550 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 14:05:53 smithi107 systemd[1]: Stopping Ceph iscsi.iscsi.a for 98a6635e-27f5-11ef-bc9b-c7b262605968... 2024-06-11T14:05:54.050 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 14:05:53 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a[59833]: debug Shutdown received 2024-06-11T14:06:04.034 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 14:06:03 smithi107 bash[76585]: time="2024-06-11T14:06:03Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-06-11T14:06:04.035 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 14:06:03 smithi107 podman[76585]: 2024-06-11 14:06:03.66860927 +0000 UTC m=+10.120524916 container died 1be9b8255d66c7b9dd03b9b828b4fb59e3edc5afde3c8d935bbd35211d32ef5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a, maintainer=Guillaume Abrioux , io.buildah.version=1.35.3, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-3f51ea3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.name=CentOS Stream 9 Base Image) 2024-06-11T14:06:04.300 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 14:06:04 smithi107 podman[76585]: 2024-06-11 14:06:04.109679453 +0000 UTC m=+10.561595088 container cleanup 1be9b8255d66c7b9dd03b9b828b4fb59e3edc5afde3c8d935bbd35211d32ef5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.35.3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3f51ea3, org.label-schema.build-date=20240528) 2024-06-11T14:06:04.300 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 14:06:04 smithi107 bash[76585]: ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a 2024-06-11T14:06:04.561 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 14:06:04 smithi107 podman[76600]: 2024-06-11 14:06:04.302761403 +0000 UTC m=+0.633510043 container remove 1be9b8255d66c7b9dd03b9b828b4fb59e3edc5afde3c8d935bbd35211d32ef5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce454e59f5f61a0030b01fb0a6f9012c83522b7e7d1b60b63c90c2a73fa310, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-iscsi-iscsi-a, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3f51ea3, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.35.3) 2024-06-11T14:06:04.562 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 14:06:04 smithi107 systemd[1]: ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-06-11T14:06:04.562 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 14:06:04 smithi107 systemd[1]: ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-06-11T14:06:04.562 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 14:06:04 smithi107 systemd[1]: Stopped Ceph iscsi.iscsi.a for 98a6635e-27f5-11ef-bc9b-c7b262605968. 2024-06-11T14:06:04.562 INFO:journalctl@ceph.iscsi.iscsi.a.smithi107.stdout:Jun 11 14:06:04 smithi107 systemd[1]: ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@iscsi.iscsi.a.service: Consumed 4.386s CPU time. 2024-06-11T14:06:06.050 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 14:06:05 smithi107 systemd[1]: Stopping Ceph grafana.a for 98a6635e-27f5-11ef-bc9b-c7b262605968... 2024-06-11T14:06:06.499 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 14:06:06 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=server t=2024-06-11T14:06:06.209880985Z level=info msg="Shutdown started" reason="System signal: terminated" 2024-06-11T14:06:06.499 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 14:06:06 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=tracing t=2024-06-11T14:06:06.210291472Z level=info msg="Closing tracing" 2024-06-11T14:06:06.500 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 14:06:06 smithi107 ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a[63399]: logger=ticker t=2024-06-11T14:06:06.210324198Z level=info msg=stopped last_tick=2024-06-11T14:06:00Z 2024-06-11T14:06:06.500 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 14:06:06 smithi107 podman[76867]: 2024-06-11 14:06:06.220102287 +0000 UTC m=+0.114292283 container died c9d154c6ff5012639083fc205067f0c4948d6dbb87ee0a380578b88de3e52557 (image=quay.io/ceph/grafana:9.4.12, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a, maintainer=Grafana Labs ) 2024-06-11T14:06:06.500 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 14:06:06 smithi107 podman[76867]: 2024-06-11 14:06:06.500773738 +0000 UTC m=+0.394963733 container cleanup c9d154c6ff5012639083fc205067f0c4948d6dbb87ee0a380578b88de3e52557 (image=quay.io/ceph/grafana:9.4.12, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a, maintainer=Grafana Labs ) 2024-06-11T14:06:06.800 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 14:06:06 smithi107 bash[76867]: ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a 2024-06-11T14:06:06.800 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 14:06:06 smithi107 podman[76880]: 2024-06-11 14:06:06.685548728 +0000 UTC m=+0.467099122 container remove c9d154c6ff5012639083fc205067f0c4948d6dbb87ee0a380578b88de3e52557 (image=quay.io/ceph/grafana:9.4.12, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-grafana-a, maintainer=Grafana Labs ) 2024-06-11T14:06:07.273 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 14:06:06 smithi107 systemd[1]: ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@grafana.a.service: Deactivated successfully. 2024-06-11T14:06:07.273 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 14:06:06 smithi107 systemd[1]: Stopped Ceph grafana.a for 98a6635e-27f5-11ef-bc9b-c7b262605968. 2024-06-11T14:06:07.273 INFO:journalctl@ceph.grafana.a.smithi107.stdout:Jun 11 14:06:06 smithi107 systemd[1]: ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@grafana.a.service: Consumed 22.391s CPU time. 2024-06-11T14:06:07.273 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 14:06:07 smithi107 systemd[1]: Stopping Ceph node-exporter.b for 98a6635e-27f5-11ef-bc9b-c7b262605968... 2024-06-11T14:06:07.545 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 14:06:07 smithi107 podman[76993]: 2024-06-11 14:06:07.273930601 +0000 UTC m=+0.118072901 container died 2332e771e10eb783875ae65259d13f873a0a998b59d44344bbfd02d8757bcc55 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-06-11T14:06:07.800 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 14:06:07 smithi107 podman[76993]: 2024-06-11 14:06:07.62955379 +0000 UTC m=+0.473696081 container cleanup 2332e771e10eb783875ae65259d13f873a0a998b59d44344bbfd02d8757bcc55 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-06-11T14:06:07.800 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 14:06:07 smithi107 bash[76993]: ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b 2024-06-11T14:06:08.197 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 14:06:07 smithi107 podman[77005]: 2024-06-11 14:06:07.897706372 +0000 UTC m=+0.624489989 container remove 2332e771e10eb783875ae65259d13f873a0a998b59d44344bbfd02d8757bcc55 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-98a6635e-27f5-11ef-bc9b-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-06-11T14:06:08.197 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 14:06:07 smithi107 systemd[1]: ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-06-11T14:06:08.197 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 14:06:08 smithi107 systemd[1]: ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@node-exporter.b.service: Failed with result 'exit-code'. 2024-06-11T14:06:08.197 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 14:06:08 smithi107 systemd[1]: Stopped Ceph node-exporter.b for 98a6635e-27f5-11ef-bc9b-c7b262605968. 2024-06-11T14:06:08.197 INFO:journalctl@ceph.node-exporter.b.smithi107.stdout:Jun 11 14:06:08 smithi107 systemd[1]: ceph-98a6635e-27f5-11ef-bc9b-c7b262605968@node-exporter.b.service: Consumed 4.921s CPU time. 2024-06-11T14:06:08.681 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-11T14:06:08.716 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-11T14:06:08.743 INFO:tasks.cephadm:Archiving crash dumps... 2024-06-11T14:06:08.745 DEBUG:teuthology.misc:Transferring archived files from smithi057:/var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-10_21:08:02-orch-squid-distro-default-smithi/7749303/remote/smithi057/crash 2024-06-11T14:06:08.746 DEBUG:teuthology.orchestra.run.smithi057:> sudo tar c -f - -C /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/crash -- . 2024-06-11T14:06:08.825 INFO:teuthology.orchestra.run.smithi057.stderr:tar: /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-11T14:06:08.825 INFO:teuthology.orchestra.run.smithi057.stderr:tar: Error is not recoverable: exiting now 2024-06-11T14:06:08.827 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-10_21:08:02-orch-squid-distro-default-smithi/7749303/remote/smithi107/crash 2024-06-11T14:06:08.828 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/crash -- . 2024-06-11T14:06:08.891 INFO:teuthology.orchestra.run.smithi107.stderr:tar: /var/lib/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-11T14:06:08.892 INFO:teuthology.orchestra.run.smithi107.stderr:tar: Error is not recoverable: exiting now 2024-06-11T14:06:08.893 INFO:tasks.cephadm:Checking cluster log for badness... 2024-06-11T14:06:08.893 DEBUG:teuthology.orchestra.run.smithi057:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph.log | egrep CEPHADM_ | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-06-11T14:06:08.935 INFO:teuthology.orchestra.run.smithi057.stdout:2024-06-11T13:31:23.217094+0000 mon.a (mon.0) 1023 : cluster [WRN] Health check failed: 2 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-06-11T14:06:08.935 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-06-11T14:06:08.935 DEBUG:teuthology.orchestra.run.smithi057:> sudo egrep '\[SEC\]' /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph.log | egrep CEPHADM_ | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-06-11T14:06:09.011 DEBUG:teuthology.orchestra.run.smithi057:> sudo egrep '\[ERR\]' /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph.log | egrep CEPHADM_ | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-06-11T14:06:09.084 DEBUG:teuthology.orchestra.run.smithi057:> sudo egrep '\[WRN\]' /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph.log | egrep CEPHADM_ | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-06-11T14:06:09.156 INFO:teuthology.orchestra.run.smithi057.stdout:2024-06-11T13:31:23.217094+0000 mon.a (mon.0) 1023 : cluster [WRN] Health check failed: 2 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-06-11T14:06:09.156 INFO:tasks.cephadm:Compressing logs... 2024-06-11T14:06:09.157 DEBUG:teuthology.orchestra.run.smithi057:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-11T14:06:09.199 DEBUG:teuthology.orchestra.run.smithi107:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-11T14:06:09.222 INFO:teuthology.orchestra.run.smithi057.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose: No such file or directory -- 2024-06-11T14:06:09.222 INFO:teuthology.orchestra.run.smithi057.stderr: /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph.log 2024-06-11T14:06:09.222 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-osd.2.log 2024-06-11T14:06:09.222 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-11T14:06:09.222 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph.log: /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-osd.3.log 2024-06-11T14:06:09.224 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-osd.1.log 2024-06-11T14:06:09.224 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-mgr.y.log 2024-06-11T14:06:09.224 INFO:teuthology.orchestra.run.smithi057.stderr:gzip/var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-osd.1.log: -5 --verbose -- /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-osd.0.log 2024-06-11T14:06:09.224 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-mon.c.log 2024-06-11T14:06:09.225 INFO:teuthology.orchestra.run.smithi107.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph.log 2024-06-11T14:06:09.225 INFO:teuthology.orchestra.run.smithi107.stderr:: No such file or directory 2024-06-11T14:06:09.226 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-11T14:06:09.226 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-osd.7.log 2024-06-11T14:06:09.226 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-osd.5.log 2024-06-11T14:06:09.226 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-osd.4.log 2024-06-11T14:06:09.230 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-mgr.x.log 2024-06-11T14:06:09.230 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-osd.5.log: /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-osd.6.log 2024-06-11T14:06:09.230 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-06-11T14:06:09.230 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/tcmu-runner.log 2024-06-11T14:06:09.230 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph.cephadm.log: 79.8% -- replaced with /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-06-11T14:06:09.230 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-11T14:06:09.230 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/tcmu-runner.log: 63.3% -- replaced with /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/tcmu-runner.log.gz 2024-06-11T14:06:09.230 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-mon.b.log 2024-06-11T14:06:09.231 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-volume.log: 88.1% -- replaced with /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph.log.gz 2024-06-11T14:06:09.231 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-osd.0.log: 93.4% -- replaced with /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph.log.gz 2024-06-11T14:06:09.236 INFO:teuthology.orchestra.run.smithi107.stderr: 92.3% -- replaced with /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-11T14:06:09.237 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-mon.a.log 2024-06-11T14:06:09.239 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-11T14:06:09.239 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-mon.b.log: 92.4%/var/log/ceph/cephadm.log: -- replaced with /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-mgr.x.log.gz 2024-06-11T14:06:09.244 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-mon.c.log: 95.2% -- replaced with /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-11T14:06:09.244 INFO:teuthology.orchestra.run.smithi107.stderr: 90.9% -- replaced with /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-11T14:06:09.253 INFO:teuthology.orchestra.run.smithi107.stderr: 91.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-11T14:06:09.253 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-06-11T14:06:09.268 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-11T14:06:09.277 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph.cephadm.log: 89.2%gzip -5 --verbose -- /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-client.rgw.foo.a.log 2024-06-11T14:06:09.277 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-volume.log: -- replaced with /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-06-11T14:06:09.278 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-11T14:06:09.306 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-client.rgw.foo.a.log: /var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-11T14:06:09.318 INFO:teuthology.orchestra.run.smithi057.stderr: 90.9% -- replaced with /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-11T14:06:09.424 INFO:teuthology.orchestra.run.smithi057.stderr: 93.5% -- replaced with /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-client.rgw.foo.a.log.gz 2024-06-11T14:06:09.803 INFO:teuthology.orchestra.run.smithi057.stderr: 90.2% -- replaced with /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-mgr.y.log.gz 2024-06-11T14:06:10.808 INFO:teuthology.orchestra.run.smithi057.stderr: 92.0% -- replaced with /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-mon.c.log.gz 2024-06-11T14:06:11.289 INFO:teuthology.orchestra.run.smithi107.stderr: 92.4% -- replaced with /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-mon.b.log.gz 2024-06-11T14:06:12.284 INFO:teuthology.orchestra.run.smithi057.stderr: 91.7% -- replaced with /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-mon.a.log.gz 2024-06-11T14:06:17.524 INFO:teuthology.orchestra.run.smithi107.stderr: 93.9% -- replaced with /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-osd.6.log.gz 2024-06-11T14:06:17.742 INFO:teuthology.orchestra.run.smithi107.stderr: 94.0% -- replaced with /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-osd.7.log.gz 2024-06-11T14:06:18.266 INFO:teuthology.orchestra.run.smithi107.stderr: 94.0% -- replaced with /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-osd.4.log.gz 2024-06-11T14:06:18.519 INFO:teuthology.orchestra.run.smithi057.stderr: 94.0% -- replaced with /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-osd.3.log.gz 2024-06-11T14:06:18.537 INFO:teuthology.orchestra.run.smithi107.stderr: 93.9% -- replaced with /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-osd.5.log.gz 2024-06-11T14:06:18.539 INFO:teuthology.orchestra.run.smithi107.stderr: 2024-06-11T14:06:18.539 INFO:teuthology.orchestra.run.smithi107.stderr:real 0m9.325s 2024-06-11T14:06:18.539 INFO:teuthology.orchestra.run.smithi107.stderr:user 0m36.013s 2024-06-11T14:06:18.539 INFO:teuthology.orchestra.run.smithi107.stderr:sys 0m1.222s 2024-06-11T14:06:18.566 INFO:teuthology.orchestra.run.smithi057.stderr: 93.9% -- replaced with /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-osd.2.log.gz 2024-06-11T14:06:18.890 INFO:teuthology.orchestra.run.smithi057.stderr: 93.9% -- replaced with /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-osd.0.log.gz 2024-06-11T14:06:19.199 INFO:teuthology.orchestra.run.smithi057.stderr: 93.9% -- replaced with /var/log/ceph/98a6635e-27f5-11ef-bc9b-c7b262605968/ceph-osd.1.log.gz 2024-06-11T14:06:19.201 INFO:teuthology.orchestra.run.smithi057.stderr: 2024-06-11T14:06:19.201 INFO:teuthology.orchestra.run.smithi057.stderr:real 0m9.990s 2024-06-11T14:06:19.201 INFO:teuthology.orchestra.run.smithi057.stderr:user 0m42.022s 2024-06-11T14:06:19.201 INFO:teuthology.orchestra.run.smithi057.stderr:sys 0m1.405s 2024-06-11T14:06:19.202 INFO:tasks.cephadm:Archiving logs... 2024-06-11T14:06:19.202 DEBUG:teuthology.misc:Transferring archived files from smithi057:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-10_21:08:02-orch-squid-distro-default-smithi/7749303/remote/smithi057/log 2024-06-11T14:06:19.203 DEBUG:teuthology.orchestra.run.smithi057:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-11T14:06:24.582 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-10_21:08:02-orch-squid-distro-default-smithi/7749303/remote/smithi107/log 2024-06-11T14:06:24.583 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-11T14:06:29.564 INFO:tasks.cephadm:Removing cluster... 2024-06-11T14:06:29.564 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 --force 2024-06-11T14:06:29.767 INFO:teuthology.orchestra.run.smithi057.stdout:Deleting cluster with fsid: 98a6635e-27f5-11ef-bc9b-c7b262605968 2024-06-11T14:06:30.080 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 98a6635e-27f5-11ef-bc9b-c7b262605968 --force 2024-06-11T14:06:30.278 INFO:teuthology.orchestra.run.smithi107.stdout:Deleting cluster with fsid: 98a6635e-27f5-11ef-bc9b-c7b262605968 2024-06-11T14:06:30.581 INFO:tasks.cephadm:Removing cephadm ... 2024-06-11T14:06:30.581 DEBUG:teuthology.orchestra.run.smithi057:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-11T14:06:30.600 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-11T14:06:30.616 INFO:tasks.cephadm:Teardown complete 2024-06-11T14:06:30.616 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-06-11T14:06:30.626 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-06-11T14:06:30.626 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-11T14:06:30.643 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-11T14:06:30.731 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-06-11T14:06:30.741 INFO:teuthology.task.clock:Checking final clock skew... 2024-06-11T14:06:30.741 DEBUG:teuthology.orchestra.run.smithi057:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-11T14:06:30.744 DEBUG:teuthology.orchestra.run.smithi107:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-11T14:06:30.758 INFO:teuthology.orchestra.run.smithi107.stderr:bash: line 1: ntpq: command not found 2024-06-11T14:06:30.758 INFO:teuthology.orchestra.run.smithi057.stderr:bash: line 1: ntpq: command not found 2024-06-11T14:06:30.762 INFO:teuthology.orchestra.run.smithi107.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-11T14:06:30.762 INFO:teuthology.orchestra.run.smithi107.stdout:=============================================================================== 2024-06-11T14:06:30.762 INFO:teuthology.orchestra.run.smithi107.stdout:^* hv01.front.sepia.ceph.com 3 6 377 32 +807us[ +808us] +/- 55ms 2024-06-11T14:06:30.762 INFO:teuthology.orchestra.run.smithi107.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 95 -2082us[-2084us] +/- 65ms 2024-06-11T14:06:30.763 INFO:teuthology.orchestra.run.smithi107.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 29 +783us[ +783us] +/- 65ms 2024-06-11T14:06:30.763 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-06-11T14:06:30.764 INFO:teuthology.orchestra.run.smithi057.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-11T14:06:30.764 INFO:teuthology.orchestra.run.smithi057.stdout:=============================================================================== 2024-06-11T14:06:30.764 INFO:teuthology.orchestra.run.smithi057.stdout:^* hv01.front.sepia.ceph.com 3 6 377 33 +885us[ +864us] +/- 55ms 2024-06-11T14:06:30.764 INFO:teuthology.orchestra.run.smithi057.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 31 -2060us[-2060us] +/- 66ms 2024-06-11T14:06:30.764 INFO:teuthology.orchestra.run.smithi057.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 32 +827us[ +827us] +/- 65ms 2024-06-11T14:06:30.765 INFO:teuthology.orchestra.run.smithi057.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-06-11T14:06:30.765 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-06-11T14:06:30.774 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-06-11T14:06:30.775 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-06-11T14:06:30.853 DEBUG:teuthology.orchestra.run.smithi057:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-11T14:06:30.932 DEBUG:teuthology.orchestra.run.smithi107:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-11T14:06:31.001 DEBUG:teuthology.orchestra.run.smithi057:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-06-11T14:06:31.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-11T14:06:31.031 DEBUG:teuthology.orchestra.run.smithi107:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-06-11T14:06:31.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-11T14:06:31.059 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-06-11T14:06:31.068 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-06-11T14:06:31.106 INFO:teuthology.task.internal:Duration was 3085.012302 seconds 2024-06-11T14:06:31.106 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-06-11T14:06:31.245 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-06-11T14:06:31.245 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-11T14:06:31.248 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-11T14:06:31.287 INFO:teuthology.orchestra.run.smithi107.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-11T14:06:31.288 INFO:teuthology.orchestra.run.smithi057.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-11T14:06:31.591 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-06-11T14:06:31.591 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi057.front.sepia.ceph.com 2024-06-11T14:06:31.591 DEBUG:teuthology.orchestra.run.smithi057:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-11T14:06:31.641 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi107.front.sepia.ceph.com 2024-06-11T14:06:31.641 DEBUG:teuthology.orchestra.run.smithi107:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-11T14:06:31.665 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-06-11T14:06:31.665 DEBUG:teuthology.orchestra.run.smithi057:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-11T14:06:31.683 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-11T14:06:31.732 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-06-11T14:06:31.732 DEBUG:teuthology.orchestra.run.smithi057:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-11T14:06:31.751 DEBUG:teuthology.orchestra.run.smithi107:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-11T14:06:32.209 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-06-11T14:06:32.219 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-06-11T14:06:32.219 DEBUG:teuthology.orchestra.run.smithi057:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-11T14:06:32.245 DEBUG:teuthology.orchestra.run.smithi107:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-11T14:06:32.270 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-06-11T14:06:32.280 DEBUG:teuthology.orchestra.run.smithi057:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-11T14:06:32.287 DEBUG:teuthology.orchestra.run.smithi107:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-11T14:06:32.309 INFO:teuthology.orchestra.run.smithi057.stdout:kernel.core_pattern = core 2024-06-11T14:06:32.334 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = core 2024-06-11T14:06:32.373 DEBUG:teuthology.orchestra.run.smithi057:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-11T14:06:32.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-11T14:06:32.388 DEBUG:teuthology.orchestra.run.smithi107:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-11T14:06:32.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-11T14:06:32.428 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-06-11T14:06:32.439 INFO:teuthology.task.internal:Transferring archived files... 2024-06-11T14:06:32.439 DEBUG:teuthology.misc:Transferring archived files from smithi057:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-10_21:08:02-orch-squid-distro-default-smithi/7749303/remote/smithi057 2024-06-11T14:06:32.439 DEBUG:teuthology.orchestra.run.smithi057:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-11T14:06:32.504 DEBUG:teuthology.misc:Transferring archived files from smithi107:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-10_21:08:02-orch-squid-distro-default-smithi/7749303/remote/smithi107 2024-06-11T14:06:32.504 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-11T14:06:32.540 INFO:teuthology.task.internal:Removing archive directory... 2024-06-11T14:06:32.541 DEBUG:teuthology.orchestra.run.smithi057:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-11T14:06:32.543 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-11T14:06:32.590 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-06-11T14:06:32.600 INFO:teuthology.task.internal:Not uploading archives. 2024-06-11T14:06:32.601 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-06-11T14:06:32.636 INFO:teuthology.task.internal:Tidying up after the test... 2024-06-11T14:06:32.636 DEBUG:teuthology.orchestra.run.smithi057:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-11T14:06:32.638 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-11T14:06:32.652 INFO:teuthology.orchestra.run.smithi057.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 11 14:06 /home/ubuntu/cephtest 2024-06-11T14:06:32.653 INFO:teuthology.orchestra.run.smithi107.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 11 14:06 /home/ubuntu/cephtest 2024-06-11T14:06:32.654 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-06-11T14:06:32.679 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-06-11T14:06:32.931 INFO:teuthology.run:Summary data: description: orch/cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/root mon_election/classic msgr/async-v1only start tasks/rados_api_tests} duration: 3085.0123019218445 failure_reason: '"2024-06-11T13:31:23.217094+0000 mon.a (mon.0) 1023 : cluster [WRN] Health check failed: 2 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log' flavor: default owner: scheduled_teuthology@teuthology success: false 2024-06-11T14:06:32.931 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-11T14:06:33.013 INFO:teuthology.run:FAIL